O/ 510-843-6227 FAX 510 843 6280 /cell 510 219 4255
We made a proposal to the Federal Government in 1995 to develop our approach to structured OJT. That was NSF/SBIR. They turned it down.
Tried again in 2003, this time DARPA/SBIR.
I'm just going to put the whole thing here.
Department of Defense, Small Business Innovation Research (SBIR) Program
Program Solicitation FY03.1 Phase 1 Proposal
Defense Advanced Research Projects Agency (DARPA) Component
"Field Measurement of Soldier's Cognitive Status:
with Emphasis on the Use of Short Unadorned Digital Video Sequences for Development and Calibration of Conceptual Architecture and Framework for Deployment"
Submitted for DARPA 03.1 SBIR Topic SB031-011: "Personnel Monitoring for Assessment and Management of Cognitive Workload"
To be submitted to DARPA/CMO/SBIR Attn: Ms Connie Jacobs
A. COVER SHEET (see attached)
B. TECHNICAL PROPOSAL
B.1. IDENTIFICATION AND SIGNIFICANCE OF THE PROBLEM OR OPPORTUNITY
The objective of this proposal is to demonstrate the feasibility of using multiple modalities for measuring brain function (functional MRI plus functional NIRS plus functional EEG, for example) to assess a soldier's cognitive status. Our assessment and calibration of the measurement modalities will be accomplished by amplifying the soldier's cognitive signal by showing the soldier short (less than one minute) unadorned digital video sequences (GetSkilz videos) depicting militarily relevant tasks, repetitively, so the soldier's cognitive state can then be readily and reproducibly read out by multiple biophysical measurement modalities. The output of certain selected suitable modalities can then be standardized for use in the proposed personnel monitor.
To assess the soldier's cognitive state is in itself a vast opportunity for research. The testing and calibration of the required cognitive measurement tools to assess this state, in a military environment, is also essential to development of a personnel monitor. Data gathered from the proposed studies of soldiers viewing GetSkilzshort unadorned digital videos sequences will be sufficient to promulgate a cognition monitor for assessment and management of cognitive workload.
Over the short term militarily and commercially, the proposed work by GetSkilz Inc. on this Technical Topic will provide an assessment of the primary complementary technologies for measuring cognition in the military environment. By the end of the project, the data collected and analyzed in Phase I will be used to develop a well-characterized personnel monitoring device that will accurately and reproducibly measure the cognitive state of any given soldier performing a given task.
B.2.1 Background
Much research has gone into developing multiple modalities for measuring brain function. Functional MRI, functional NIRS, functional PET, thermography, electroencephalography (Pleydell-Pearce,1994; Pleydell-Pearce et al, 2000)
et seq are physical techniques for reading out physiological information about the brain whilst that brain is functioning, that is, while it is thinking about something, and can be correlated with brain function. All of these physical and electrophysiological techniques have been refined to reduce noise down to levels unimaginable a few short years ago. What we propose is to raise the level of the signal, to gain a more favorable signal to noise ratio. We will raise the cognitive signal by initiating an intense cognitive state in our sample and then measure the cognitive status biophysically, so as to be able to construct a reliable monitor of cognitive status while performing relevant tasks.
An ongoing weakness in the current research is a lack of correlation between what an experimental subject is usually given to think about and what we want to measure in the military milieu.
We would like to know if, and what for, , soldiers are engaging in cognition.
Yo, listen up: I need to interject something here, folks. As this little application winds its way forward, through Act 1 , Act 2, et seq (an intro, then some other stuff, then you ask for the money; you know, a grant application) you find that more and more cooks have been involved in making the soup. And, if your writing project was done with Microsoft Word, all of those bits and pieces of revision and revisionofrevision are STILL THERE! Check this one out: The original sentence said, clearly, we would like to know if and what for soldiers are using their heads. The sentence says, here, We would like to know if and what for soldiers are engaging in cognition. Well, that's a bit confused, isn't it? of course it is. The original original sentence was actually a statement made to me, by Britton Chance in a fax in 1998 or so, that said, (since I was asking him about some experiments I was doing on teaching things to ice hockey players) , "I have often wondered if, and what for, contact sports players use their heads." Brilliant. Turned my work all the way around; it isn't that you don't think to play hockey, it's just that the emphasis is not on using your head; the game is just too fast to make those kind of decisions, I guess; but at any rate, that's just not the emphasis. Chance was just trying to steer me away from working on that, and toward working on something else where the emphasis is more on "using your head". But the way it reads above, in the revised revised version, you've got a tautology. Soldiers are using their heads, allright; they're using them at the very least for a hatrack, and a target, but we just want to know ... well, if and what for, they are using their heads. Whether they are achieving cognition, we want to know that, but it's just putting a too high a degree of certainty , like too many significant figures, on what we want to know at this stage. Just my humble opinion.
The real take home lesson here is that if you want your documents to never have bits and pieces of flotsam and jetsam attached that may surface unexpectedly on somebody's computer (this one did, because I make these webpages with a Mac running Appleworks and Claris HomePage 3) then don't use Microsoft Word. (Or at the very least, turn off that revisions-tracking crap.) Pardon me, feature.
the soldiers are using their heads. An opportunity to address this weakness lies in the use of high-cognition-level soldiers for the assessment of cognitive status. Using highly effective, GetSkilz short, unadorned video sequences of militarily relevant material, to stimulate cognitive activity, allows for calibration and testing at the necessary level of focused, specific cognition. Insight into cognition will come when the soldiers' cognitive status is thus amplified and assessed in a large cohort of soldiers.
B.2.1. Conceptualization -- Short Unadorned Digital Video Sequences for Training and Calibration of Cognition
The soldier's cognitive state during training should be characterized by alertness and focus. The concept here is that this state of mind can be measured biophysically by well-characterized methods in a milieu that is absolutely relevant to activities that are unambiguously and characteristically military. To this end, we shall prepare short unadorned digital video sequences depicting activities that fall within the scope of military activities, either in basic training, specialist training or deployment in the field.
Short Unadorned Digital Video Sequences (GetSkilz Videos) Defined
Generally,
short means less than a minute running time1 with practically no exceptions2
unadorned means no titles, no credits, no talking heads (where a "talking head" is a narrator whose visage appears onscreen)3;
digital video means that the movie, or "moving images", or "motion picture", are being played on a computer monitor and are controlled by the keyboard, or mouse, or other input device of the computer and can be distributed efficiently through a computer network4;
sequence5 means, by filmic convention, recorded action of a single subject in that takes place in one placelocation at one time, and during which the camera is free to take up different camera angles and points of view6.
The genre of GetSkilz short unadorned digital video sequences is not at all common nowadays. In fact, movies consisting of a single sequence haven't been released for a long time. They were the lingua franca and stock in trade of the movie business at the very beginning, with the invention of the movie camera and movie projector around the turn of the twentieth century. Since DW Griffiths produced Birth of a Nation in 1915, however, it has been the accepted purpose of film production to "tell a story" (though a number of excellent storytelling productions featuring sexual adventures had been produced, distributed, and exhibited before Griffiths' movie.) The typical training material consists of long talking heads, illustrations or animations of activities, or videos of activities broken down into disjoint steps and performed at very slow speed. None of these methodologies demonstrate the task as it should be done and at the pace that it should be done. GetSkilz video sequences take advantage of the human brain's response to situated cognition by showing the real activity in the real environment in real time. Preliminary fieldwork on training with GetSkilz videos in sports, restaurant, and medical industries indicate that this training is highly-effective (both in time and in performance) in teaching hands-on skills. Experiments at the University of Pennsylvania have concluded that a system of learning physical tasks developed by GetSkilz, Inc. of Berkeley significantly improves the subject's learning capacity. GetSkilz's video instruction method (patent pending) proved to be a highly sophisticated method of teaching subjects how to do hands-on tasks such as tying a knot. Dr Britton Chance's laboratory at Penn's Medical School measured oxygen flowing to the cortex of the brain when subjects learned to tie some common knots in a length of rope. The Chance lab's equipment revealed that the brain achieved a high level of cognition when the subjects watched GetSkilz short videos of how to tie bowline and clove hitch knots The purpose of GetSkilz sequences is not so much to tell a story, as to tell the truth.7,8.
B.2.2 Proposed Use of GetSkilzShort Unadorned Digital Video Sequences in Phase I
We propose to use short unadorned digital GetSkilz video sequences of small arms weapons training to establish and calibrate a system for the measurement of the soldier's cognitive state. A soldier watches the short video of, say, disassembly, while hooked up to instrumentation that measures cognition, by NIRS, or fMRI, or EEG, et seq. Hands-on implementation follows. The Drill Instructor(s) mentor as is traditional.
The GetSkilz short unadorned digital videos sequences we propose to use will be movies shot in real time of real people, people like the Berkeley Police Department Training Officer who disassembles a service pistol; or the ship's engineer who fieldstrips a deck winch; or the surgeon performing an ultrasound-guided surgical procedure; or the preparateur at a fast food restaurant, all depicted on the GetSkilz Skillware CD-ROM that accompanies this Proposal, or by connecting to the URL http://www.getskilz.com and linking to DarpaVideos (You need to have QuickTime® installed to view these movies) .. GetSkilz will create content by documenting the work activities of expert performers, such asThese are people who don't need to rehearse. A any Instructor in Small Arms Weapons training in any branch of the United States military will do fine. These expert performers do not need to rehearse and can perform the specific work activity at the correct speed and in the correct manner. We will use this content to develop a training system based on situated cognition
What we propose is simply to put the eons-old neural capability of viewing images realistically, in the service of learning the ropes; and to make a system of it, as an adjunct to developing a system for personnel monitoring.9
B.2.3 Measurement
In Phase I we propose to measure cognitive activity while recruits learn to handle small arms weapons, to establish the proof of the concept in the military arena and analyze cognition measurement methodologies. Digital videos depicting in real time solid performers disassembling, assembling, cleaning, and selectively firing service weapons regularly issued to troops will be produced. These movies will be deployed in the accustomed basic training milieu, integrated side by side noncompetitively with the customary drill officers' methods. Brain activity data will be acquired for the troops during the learning process via all of the modalities mentioned above, and an assessment made of the trainees' alertness and focus as indicated by tissue oxygenation, MRI, PET, USND, EEG, and thermography, as well as the customary determination of proficiency and time to proficiency.
One of the cognition indicators is developed by Dr. Britton Chance's group. Dr. Chance's group has characterized the response by the brain to cognitive stimuli by measuring increased flow and volume of oxygenated hemoglobin to specific areas of the brain. Recent work at the Chance lab in Philadelphia indicates quite strongly that repetitive viewing of GetSkilz videosshort, unadorned digital video sequences depicting physical tasks results in swift acquisition of the knowledge of how to perform the depicted task; and that this is accompanied by unambiguous recordings of increased oxygenation in specific areas of the brain, observed noninvasively and in real time using a phased array detector at the wavelengths of visible and nearinfrared light. Preliminary experiments were conducted with video depiction of physical tasks of knot tying exercises. The regimen for training in knot tying is according to the patent-pending process originated by GetSkilz Inc. Brain response to the knot regimen would predictably be measured under fMRI; PET; Ultrasound; enhanced EEG; and thermography, similarly to Chance et al's cognometry device.
Our research will show that It remains to be seen whether physical tasks other than knot tying elicit measurable brain response with the Chance lab equipment during repetitive video training with GetSKilz videosshort unadorned digital video sequences.. Specifically, we willit would be useful to determine if tasks that require skills that are imparted during military basic training elicit this measurable response. In addition, we will test if it would be useful to bethis response can be able to measured this response remotely, in a wireless or "untethered" arrangement, under MRI; PET; USND; EEG, and thermography.
In Phase I we propose to measure cognitive activity while recruits learn to handle small arms weapons, to establish the proof of the concept in the military arena. Digital videos depicting in real time solid performers disassembling, assembling, cleaning, and selectively firing service weapons regularly issued to troops will be produced. These movies will be deployed in the accustomed basic training milieu, integrated side by side noncompetitively with the customary drill officers' methods. Brain activity data will be acquired for the troops during the learning process via all of the modalities mentioned above, and an assessment made of the trainees' alertness and focus as indicated by tissue oxygenation, MRI, PET, USND, EEG, and thermography, as well as the customary determination of proficiency and time to proficiency. There is, of course, no possibility that any trainee will not attain proficiency during basic training in small arms weapons.
Running times for short unadorned digital the video skill acquisition movies shawill be optimized, once the rudiments of the military-oriented training regimen hasve been established. The optimization goal is to provide the most effective training in the shortest time. An ordnance expert, someone at the level of expertise of Officer Marangoni of the Berkeley Police Department, has very little trouble performing routine tasks like disassembly, cleaning, and assembly of a weapon at any speed requested of him. A neophyte trainee requires as much as ten times as long to perform the same task. A series of movies will needs must then be prepared to ascertain the optimal relationship between movieshape of the curve for running time (of the movie depicting the task performed by the firearms expert) vs. time to proficiency of trainees; and whether this curve is shifted as trainees gain experiencemove along the path from novelty to familiarity with the tasks.10.
B.2.4. Applicability to Phase II
The concept would be extended in Phase II to a ruggedized and widely applicable version of the hardware for assessing cognitive status via near infrared spectroscopy (NIRS) as outlined above. Other modalities that can be used outside the laboratory/hospital situation would similarly be employed to measure cognitive status, indicating alertness and focus. The ruggedized hardware would be deployed in a wider number of training situations, involving additional movie production (production of digital assets). Both basic training in addition to small arms weapons; and advanced or specialist training, would be areas of measurement where the ruggedized hardware and additional videos would be deployed.
B.2.5. Applicability to Phase III
Training to perform tasks in the non-military sector is analogous to military training, when one considers that military training encompasses learning how to perform tasks that are simple (disassembling a carbine or pistol); or complex (larger ordinance, aircraft maintenance, et seq); or decisionmakingdecision-making (selective firing; commanding troops); and that all of these types of expertise must be imparted to quite a few people in the organization, be it military or non-military. Especially in the military, but increasingly so in the nonmilitary sector as well, the emphasis has come to be on ramping up quickly, on acquiring skills and information rapidly and effectively.
Managers in the nonmilitary world thus have as much of a reason as their military counterparts to assess the state of their trainees and field personnel biophysically, to assure themselves that their brains are adequately mobilizing their resources for cognition to be taking place; and they would want to do this in a nearly automated, or at a minimum scalable, fashion. They would want to accomplish their training aims in a reproducible and nonidiosyncratic fashion as well. Finally, they would like to capitalize on the smashing success shown by United States military (and civilian) use of simulators ("trainers"), up to and including the evocative simulation of the enemy by troops fielding the enemy's equipment and flying the enemy's aircraft. These three attributes constitute the architecture of the concept we are proposing for assessing cognitive status: A reproducible nonidiosyncratic highly evocative training milieu mediated by digitized repetitive video sequences depicting images at a high level of verisimilitude, with the trainee's relevant metabolism evaluated by physical methods, and the outcomes verified in the military arena, where failure to attain proficiency is unthinkable and improvement in a method of attaining proficiency is always commendable. Acceptance, and earnings, should be vast.
Finally, in Phase III we envision the standardization of measurement of cognitive status by several complementary modalities, as requested in DARPA component Topic SB031-011; and the packaging of such measurement equipment in a readily deployable device outside the training milieu. We expect It is possible that the civilian world willto as well as the highly effectiveembrace such a device, in the same way that the civilian world will surely embrace the proposed vastly improved training and education methodology and machinery. That acceptance may take many years; it will be success enough, however, if the fruits ofFurther, the research and development will result in a military adjuncttool whichthat magnifies the contribution of every soldier.11
B.2. PHASE I TECHNICAL OBJECTIVES
The Phase I proposal has the following technical objectives:
Develop militarily relevant-training content to provide a standardized framework for monitoring cognitive status.
Test a variety of cognitive status measurement modalities within the standardized training format.
Assess the performance of the different modalities and measurement tools.
Calibrate the measurement tools according to the results of the militarily-relevant training.
Assess the "portability potential" of the status measurement tools.
Author a summary of all testing, calibration, and portability results.
Accomplishment of all the technical objectives will allow GetSkilz to proceed to Phase II, which includes the development of a "rugged", effective, portable, cognitive status monitor.
B.3. PHASE I WORK PLAN
The following tasks will occur in serial immediately upon award of grant:
Task 1: Create GetSkilz videos short unadorned video content of small arms training at military base. Complete all production and post production of content. (Estimated time: 1 month)
Task 2: Test and collect data using a cognometer in a lab setting using the video content. Our first modality will be Dr. Chance's cognitive measurement device. Contact alternative measurement labs. (Estimated time: 1 month)
Task 3: Test and collect data using other modalities. (Estimated time: 2 months)
Task 4: Analyze data from the multiple modalities. (Estimated time: 1 month)
Task 5: Prepare Final Report (Estimated time: 1 month)
B.4. RELATED WORK
GetSkilz work withusing short unadorned digital video sequences in science and industry has included collaborations or contracts with Huntington Investments (a Burger King franchisee) in a beta test of SkilWare and GetSkilz compliance software package; the University of California at Berkeley Microlab; Andre Lacroix and the Oakland Ice Rink study of hockey fundamentals; the knot-tying studies referenced above with Professor Chance's group at the University of Pennsylvania; and extensive collaborations with Jay Harness, MD on presenting ultrasound guided surgical practice. The emphasis in all of the work to date has been on performance as a metric of cognitive activity.12
GetSkilz welcomes the opportunity to extend this work to military-related activitiesthe within the related field of biophysical measurement as assessment of cognitive state, and apply it to the development oextended to the completion of a cognitive personnel monitoring system.
The work with EEG measurement of subjects watching analog longform videotapes provided a surrising insight that is no doubt correlated with cognitive assessments following GetSkilz short digital video sequence viewing. Electroencephalography is indeed a powerful tool for assessing cognitive state, and probably the only avenue to pursue in these bipolar observed aspects of fascination/inattention to video, or perhaps any information presented on a monitor.
B.5. RELATIONSHIP WITH FUTURE RESEARCH OR RESEARCH AND DEVELOPMENT
The methodology developed by GetSkilz for the calibration and testing of different modalities to measure cognitive status will be easily adaptable to any future modalities. The proposed methodology developed under Phase I is generalizable across modalities.
B.6. COMMERCIALIZATION STRATEGY
GetSkilz seeks to develop a portable cognitive status monitor that can be marketed to any industry that invests in training. Monitoring cognitive status during training will allow GetSkilz to evaluate a firm's Return on Investment for training and quantify the cognitive assets of the firm. Additionally, GetSkilz will develop and market the training methodology developed in this proposal.
B.7. KEY PERSONNEL
Richard Katz
The PI has been involved in investigations into bioenergetics and cognition beginning 30 years ago (Katz, Kilpatrick and Chance 1971), with an emphasis on spectrophotometric methods. With the invention of the personal computer twenty years ago and the nonlinear digital movie ten years ago, he commenced investigations into the use of computerized media for the elucidation and explication of human cognition, and the learning of industrial tasks in particular. GetSkilz Inc, of which he is Chief Creative Officer, was formed three years ago to commercialize the use of short unadorned digital video sequences in industrial training and as an adjunct to commercial compliance and recordkeeping requirements.
Educated at the University of Pennsylvania and the University of California Berkeley.
Ben Campbell, Ph.D.
Dr. Campbell holds a Ph.D. in Economics from the University of California at Berkeley and is an expert in training for Hhigh-Sskill workers. He has presented his research on training methodology and efficacy in peer reviewed journals (for example, Brown and Campbell, 2001) and at international conferences13. He has operationalized his research on training through the development and implementation of training programs for the University of California at Berkeley Microfabrication Laboratory.
James Helms
Mr. Helms is an Information Technologist with expertise on digital content production on the Macintosh platform. He has extensive experience on the production of software and content for the management and creation of digital media.
Brian Choate
Mr. Choate is an Information Technologist specializing in the creation, deployment, and distribution of web-based content. In addition to his content delivery experience, he has user interface and design expertise.
B.8. FACILITIES/EQUIPMENT
All proposed work will be completed at the GetSkilz office in Berkeley, CA. GetSkilz maintains a fully functional digital video production and post-production studio. GetSkilz has the facilities and equipment to capture any real-time event on digital media, edit and post-produce the video content, embed the content within an effective user interface environment, and publish the content to all common distribution technologies.
B.9. SUBCONTRACTORS/CONSULTANTS
GetSkilz has established a volunteer advisory board for this project:
Clair Brown, Ph.D., founder and CEO of GetSkilz, Inc, is Professor of Economics and Director of the Center for Work, Technology, and Society at the University of California Berkeley. An internationally recognized expert on training, Dr. Brown, developed (and neologized) the American concept of "Structured OJT" (Brown, et al. 1997).
Britton Chance, Ph.D. For. Mem. RS, Eldridge Reeves Johnson University Professor Emeritus of Biophysics, Physical Biochemistry and Radiologic Physics, University of Pennsylvania. Professor Chance's current interests include the study of the basic theory of photon migration through tissues; the use of picosecond pulsed and high frequency modulation of near infrared (NIR) light in human brain, breast and muscle, to characterize tissue optical properties; and the use of imaging systems to detect human brain function in cognitive activity.
Mr. Alexander M. Hehmeyer, has extensive experience as an executive and entrepreneur in several industries, including financial services, postsecondary education, law, real estate, food service, and franchising. He was President, CEO and Director of the California Culinary Academy, Inc., a leading degree-granting postsecondary culinary school, which is publicly traded, and President and Co-founder of Career Choices, Inc., degree-granting postsecondary schools offering curricula in applied technology and allied health. Hehmeyer was Richard C. Blum & Associates, Inc., a merchant bank. He also served as Director of Acc-U-Tune, a multi-unit franchiser of automotive service centers. Hehmeyer practiced corporate securities law at Pillsbury, Madison & Sutro and then served as general counsel and director at the investment bank, Sutro & Co.entrepreneur with extensive experience in the training sector, Judge Advocate General's Corps, Lt. retired.
John Peters, Ph.D. licensed clinical psychologist with offices in Berkeley, California.
The members of this advisory support have will offered their expertise without cash remuneration.
B.10. PRIOR, CURRENT, OR PENDING SUPPORT OF SIMILAR PROPOSALS AND AWARDS
GetSkilz has no prior, current, or pending support for a similar proposal.
C. COST P ROPOSAL
*
D. COMPANY COMMERCIALIZATION REPORT
Submitted on-line.
Richard Katz
January 2003
REFERENCES
Brown, C. and Campbell B. A., 2001. "Technical Change, Wages, and Employment in Semiconductor Manufacturing." Industrial and Labor Relations Review 54(2): 450-65.
Brown, C., NYakata Y., Reich, M., and Ulman, L. 1997. " Work and Pay in the United States and Japan. Oxford University Press, New York.
Chance, B. Anday, E., Nioka, S., Zhou, S., Long, H., Worden, K., Li, C., Turray, T., Ovetsky, Y., Pidikiti, D. and Thomas, R. 1998. "A novel method for fast imaging of brain function noninvasively with light." Optics Express 2(10):411-423.
Cliver, D. O., and Riemann, H. P., eds. 2002. Foodborne Diseases, 2d ed., Academic Press, London.
Decety, J. and Michel, F., 1989, Comparative analysis of actual and mental movement times in two graphic tasks. Brain and Cognition 11:87-97.
Katz R., Kilpatrick L., Chance B. (1971). Acquisition and loss of rotenone
sensitivity in Torulopsis utilis. Eur. J. Biochem. 21, 301-307.
Pleydell-Pearce C.W. (1994) DC potential correlates of attention and cognitive load. Cognitive Neuropsychology 11 (2) 149-166.
Pleydell-Pearce, C., Dickson, B. Whitecross, S. (2000). Cognition Monitor: a system for real time pilot state assessment. In P.T. McCabe, M.A. Hanson, S.A. Robertson Ed.s Contemporary Ergonomics 2000. Taylor and Francis, London. Pp 65-69
NOTES
1 A film sequence like one of these GetSkilz sequences, consisting of a single subject in real time, and having a running time of a minute or so, conflicts with most people's understanding of what one means by "a video" or "a movie". If it's a "video", it's a half hour or so; if it's a "movie", it is expected to be at least an hour and a half. In mentally constructing the category "film" (or movie or video), if it's too short, it just doesn't fit the definition. Yet it is precisely the short unadorned movie sequence that is the lingua franca and stock in trade of the human brain.
A modern human brain can come up with a phantasmagorical dream sequence with a runtime in the tens of minutes, or the tenths of seconds; or, consciously, it can come up with an accurately modeled rendition of a person heaving a basketball through a hoop, obligatorily in real time (Decety and Michel. 1989).
2 How "short" is the short unadorned digital video sequence? In an odd catachresis to "lon g enough to reach the ground," it's too long if the viewer stops responding to it in a recognizable fashion. Our own preliminary (unpublished) experiments indicate that if a person watch a video for much longer than 90 seconds, their EEG signal exhibits less and less activity. A simple clap on the shoulder or a cheerful "Hello!" restores the EEG signal in less than a second to something approaching the more normal state. This long-form-video-induced signal does not resemble any of the EEGs of sleep; nor does it in any way resemble the EEG observed when watching short unadorned GetSkilz Quicktime movies, which resemble an ordinary EEG over their running time of a minute or so.
There is a significant semantic singularity in this video-watching situation: We don't have an accustomed term for this long-form-video induced behavior as read out by electroencephalography. Various colloquial terms come to mind: "Spaced out"; "Not punched in"; glassy-eyed, or eyes-glazed-over. The EEG traces resemble those obtained accidentally when electrodes become partly unattached, until the shoulder clap or the hearty "Hello", when the brain springs back into action.
3 No mention has been made here of "special effects" in these short digital sequences. It's very much in everyone's common nocturnal experience to have the brain composite all kinds of special effects on moving images in its purview. One of the movies used as a demonstration of GetSkilz video sequences (the Ultrasound in Surgical Applications movie) has had it's ultrasound imaging tweaked with video color-correction software to equalize the histogram and make the ultrasound-imaged anatomical information much easier to see. What the brain sees is only a version of what appears upside down on the retina, leaving lots of maneuvering room on both the input and output ends for "special effects". It's the real time aspect that makes GetSkilz short unadorned digital video sequences a certain category of movie, not how special or how unvarnished the footage is, nor how much trouble it was to get it into viewable shape.
4 The genre of short unadorned movie sequences is restricted to that which can be shown on a computer simply because it isn't practical to show them any other way. Tape decks and movie projectors are, in general, too clunky to accurately play, rewind, and replay a sequence of recorded action running less than a minute. The use of short unadorned movie sequences was made possible only a decade ago, when movies were reinvented by Apple Computer Corporation with the introduction of the Quicktime multimedia architecture. For the present purpose of developing a Personnel Monitoring Workstation, amongst the many positive attributes of digital movies such as Quicktime movies, note that a proper digital movie can be started, stopped, and rewound accurately and reproducibly; and can run several tracks (including a timecoded text track) in addition to the one audio track and one video track of film, television, and videotape. This digital capability for frame-accurate "rewind" and replay is absolutely essential to the GetSkilz method of skill acquisition through repetitive unfettered viewing and reviewing; and is the basis for the GetSkilz system of cognition and recognition.
5 By filmic convention, several "shots", or just one shot, can make up a sequence, where a shot is recorded action that takes place in one place at one time from one camera position.
6 Most commonly, though NOT in this Proposed Work, a series of sequences is then edited into a "scene"; each scene is all of the action taking place in one location at one time; and the scenes are later edited into what is generically called a movie, or motion picture, which can take the form of a television production, a feature film for the theaters, or a documentary for educational purposes. These are long-form productions. These are NOT what we shall be using for the study of cognition, or for the development of a cognitive status monitor, or for training and education.
The GetSkilz Training System has been implemented by several fast food restaurants and is being used by engineers in semiconductor fabrication. It is underdevelopment to train physicians how to use ultrasound in surgical procedures.
7 A century ago, when movie cameras and projectors and film were first invented, and before the devolution of film into fantasy, filmed sequences were produced of quite ordinary events, which, when shown to people, came across with astonishing verisimilitude. A Westerner, for example, would draw his revolver on the screen, and fire in self defense at the bad guy in the black hat; ordinary folks would run as fast and as far as they could from a locomotive approaching head on. It was thus accidentally discovered, a hundred years ago, that there was a powerful and apparently unexpected interaction between projected moving images -- mere "shadows on a wall" -- and something that goes on in the human brain. People found short unadorned movie sequences especially fascinating, and seemed to adopt them as their own. Something very much like them appears to be the lingua franca and stock in trade of the human brain itself; with the image generation and projection equipment in your head, they are the stuff which dreams are made of.
8 A television commercial is a short video. Generally a television advertisement is composed of SEVERAL very short sequences strung together, depicting action in multiple locales and at different times. Very occasionally a television commercial will indeed be a single sequence (and is then generally very effective at creating product awareness.) At the time of this Proposal (December 2002), a brand awareness campaign by Amazon.com is using short unadorned video sequences.
9 Learning the Ropes:
Four centuries ago, the Royal Navy commenced to explore the globe. The Royal Navy's requirement for Officers to serve on extended voyages resulted in the establishment of Naval Training Vessels at Liverpool, and thus to the institution of Naval Training. Officers and crew had always had to learn the ropes, one way or the other; from now on, learning the ropes would be a system. By this simulation, one could learn it here and use it there. Britannia ruled the waves.
The British Navy's system of naval training was indeed that: A system. Most of what had transpired as "training", in that time and to this day, is simply learning by doing, or more simply put, just trial and error, much as sailors had been trained by simply shipping out and picking it up as they went along. The essence of the Royal Navy's method (and of the United States Navy later on) was that a particular officer was in charge of and responsible for the seaman's knowledge of and familiarity with the details of seamanship. There was a book, wherein the curriculum was detailed out, and milestones had to be achieved. Progress was committed to a writing; records were kept. The training and education of the officers, certainly, and of the hands in most cases, was no longer a haphazard matter left to chance.
The Technical Topic for Personnel Monitoring extends this concept, even beyond the cognitive state during training and into the thick of battle. ShortGetSkilz videos unadorned digital movie sequences make the unfamiliar content familiar and contextually relevant; while the digital packaging of them allows for the modularization of learning and efficient recordkeeping of the results. It is the nature of digital information that it tends to reorganize the information around it. By digitizing the learning of tasks, we shall accomplish the more efficient performance of the tasks that surround that learning. We digitize the learning of small arms weapons training, for example, and we gather efficiencies in the entire milieu in which this training is embedded.
10 The elucidation of the role of novelty in assessing cognitive status may turn out to be a major effort under this Proposal. Paradoxically, it would appear from preliminary experiments that cognition measured by increased oxygenation of specific areas of brain tissue declines as a lesson is learned, as a skill is acquired. Presumably, this occurs during routine activities for already-learned performances as well. Here is the crux of assessment of cognitive status: To elucidate the nature of novelty, as measured reproducibly and accurately and noninvasively and quickly by biophysical means, even though Novelty and Reproducibility would appear to be somewhat at odds with each other! But the job here is not to sort out a matter of semantics; the mission is to assess cognitive status, and we propose to sort it out, using GetSkilz videos short unadorned digital video sequences as a probe.
11 The Cognitively Self-Aware Organization: It is interesting to speculate on the relationship between a personnel monitoring workstation envisioned by DARPA component Topic SB031-011 and the saga of workstation one at the valve manufacturing company. That relationship is key to implementing the concept of "… supervisors having the ability to quickly assess a soldier's current workload and fitness for upcoming tasks …" in a broad sense; that if the soldiers' Command measures accurately, reproducibly, noninvasively, and quickly the soldiers' cognitive status, a very cognitively aware organization will result, an organization that is not only aware, but proprioceptively self-aware.
A Note on the Personnel Monitoring Workstation, as Ontogeny Recapitulates Phylogeny for both Men and Machines:
The United States Department of Defense has labored mightily over the years to evolve not only its physical power but also its Command; when that Command adapts, in a manner analogous to the brain having adapted its larger size and evolved cortex from targetting projectiles and strategizing to abstract and linguistic cognitive activity, there will appear an Armed Forces that will be as hard to imagine as modern humans would be to pre-linguistic cave-dwellers. Education has already become a staple of soldiering at all levels, and training has become a matter of simulation at the most sophisticated levels --troops hold training exercises in mock combat with forces who fly, shoot, and employ tactics just like their next enemy, temporally proximal to the armed encounter. The Personnel Monitoring Workstation is going to incorporate all of that and feed back just a little bit more.
12 Within the past two years, GetSkilz proposed that a manufacturer of high pressure valves make a set of movies -- short unadorned digital video sequences -- depicting accepted recommended procedures for each workstation, as the raw stock moved along the line toward becoming finished valves. The GetSskilz involvement in the company was curtailed when the valve company filed for bankruptcy. A GetSkilz short unadorned digital video sequence had been prepared of the steps performed at the first station, which is the washing station, where blocks of stock are washed in solutions of alcohol/water, rinsed, washed, rinsed, and dried. No doubt some incremental value to the company would have accrued had they been able to tune up the performance of the operators who were washing the stock; but bankruptcy intervened. Instead, and quite unpredictably, the valve company's Customer, who acquired the assets of the company and kept it running, noticed that the procedure depicted on the video sequence was simply incorrect. The rinsing solution was being blown onto the cleaned stock stacked below it, routinely. An engineer noticed this, in the video. So instead of an incremental improvement in the plant's performance, there was effected a 90% (ninety percent) reduction in waste at station one. The consequence of this was profound: In this type of manufacturing, the contaminants deposited at station one only surfaced, as defects, near the end of the line, during electrostatic welding. The acquiring customer of the valve company's assets achieved high yields and nearly instant internal profitability. The digitization of a small part of the information that constituted the valve company's expertise, through the production of GetSkilz short unadorned digital video sequences, had reorganized the information around it. The lesson for the valve company was profound: If you value your independence, cast a digital eye on your knowledge and expertise.
Note that this story relies on anecdotal information supplied confidentially.
Correcting ground-level problems like the valve company's contamination problem might very well be achieved on purpose, just as it was achieved inadvertently in high pressure valve manufacturing. For example, the military prepares many meals; perhaps some kitchens use plastic cutting boards. Impeccable research by Professor Cliver's group (Cliver and Riemann, 2002) proves that bacteria proliferate on plastic cutting boards and are a vector of food contamination, while wooden cutting boards dissolve bacteria and make them disappear. In the course of preparing short unadorned digital movie sequences of food preparation, similar to Whopper.mov, for cutting, chopping, and slicing, GetSkilz would be particularly careful to depict wooden cutting boards, and to point out the benefits to personnel health of wooden cutting boards in the text track of the movie(s).
13 "Technological Change, Training, and Job Tasks in the Semiconductor Industry," presented at the International Industrial Relations Association 12th World Congress, Tokyo, May 29-June 2, 2000.