||||||||||||||||| US005204703A

United States Patent (19) 11) Patent Number: 5,204,703 Hutchinson et al. (45) Date of Patent: Apr. 20, 1993

54 EYE MOVEMENT AND PUPIL DIAMETER 4,755,045 7/1988 Borah et al. ........................ 351/210 APPARATUS AND METHOD 4,789,235 12/1988 Borah et al. .. ... 351A210

4,836,670 6/1989 Hutchinson ... ... 35/210 75) Inventors: Thomas E. Hutchinson, Hardendale; 4,950,069 8/1990 Hutchinson ........................ 351/210

Kevin S. Spetz, Lynchburg; Nirav R. Desai, Richmond, all of Va. Primary Examiner-Rodney B. Bovernick

Assistant Examiner-Thong Nguyen 73) Assignee: The Center for Innovative Attorney, Agent, or Firm-Staas & Halsey

Technology, Herndon, Va. 57) ABSTRACT

(21) Appl. No.: 713,969 A subject's eye movement and pupil diameter are mea 22 Filed: Jun. 11, 1991 sured while the subject is exposed to visual stimuli. The

apparatus and method utilizes an eye-look detector to 51 Int, C. ........................... A61B 3. f % determine a subject's visual examination of a presented 52 U.S. Cl. ..................................... image and by computer recording an analysis superim (58) Field of Search ................ 351/212, 243, 206-210, poses icons representing the lookpoints over the image.

351/246-247, 354/62; 128/731-745, 782 The apparatus and method are not only of value in 56) References Cited running routine tests but also may be used for new

experiments. . U.S. PATENT DOCUMENTS 4,676,6ll 6/1987 Nelson et al. ....................... 351/243 20 Claims, 5 Drawing Sheets

SUBJECT CAMERA ANDLED COMPUTER SCREEN

BESTO COMPUTER 28

N-COMPUTER (WITHFRAMEGRABBER)

U.S. Patent Apr. 20, 1993 Sheet 1 of 5 5,204,703

A/G/ SUBJECT CAMERA ANDLED COMPUTER SCREEN

N CABLESTO COMPUTER S28

NCOMPUTER (WITH FRAMEGRABBER)

A/G 2 COMPUTERSCREEN

CLINT ... b

BRIGHT-EYE

U.S. Patent Apr. 20, 1993 Sheet 2 of 5 5,204,703

FIG. 4 F.G. 5

F.G. 6

Please Make a Selection

Bigtiz:a::Sgf8::page Generate and Save a Text Screen Perform Saccadic motion test Analyze Results Register a Subject Quit

U.S. Patent Apr. 20, 1993 Sheet 3 of 5 5,204,703

FIG. 7

0 1 2 3 4 56789 10 12131415

FIG. 8

U.S. Patent Apr. 20, 1993 Sheet 4 of 5 5,204,703

FIG. 9

X LOOKPOINT Y LOOKPOINT 640

320 175

125 250 25 250

IMAGES USED INTEST

\SACCADE\PICTURESVSALLYPIC \SACCADE\PICTURES\ESTELLE,PIC

250 DATA POINTS TAKEN

-HORIZ --WERT

U.S. Patent Apr. 20, 1993 Sheet 5 of 5 5,204,703

FIG. 11

Subject Name: Subject Number: Image files: \SACCADEPICTURESSALLY,PIC

Sally Brown, Supermodel and Autoracer \SACCADE\PICTURESESTELLEPIC

Estelle Sharpe, Scholar

n s258 L(pd) -45 2(pd), -11361 (pd2) =528635

S 7.04

s143 =45 6457 296.333

5,204,703 1.

EYE MOVEMENT AND PUPIL DIAMETER APPARATUS AND METHOD

This invention relates to the determination of the movement of an eye, where it is looking and/or the pupil diameter of a person as he or she is stimulated with the presentation of various pictures and/or text.

BACKGROUND OF THE INVENTION The reactions and behavior of the eye have been

under study for many years. In fact, as early as 1765 it was known that dilations and constrictions of the pupil were not all attributable to lighting changes. It was not until the early 1970s that the psychological community began taking a stronger interest in both pupillary move ments and eye saccades. The technology of the time was limited but the researchers managed to acquire a significant knowledge base relating to the eye. Aside from the obvious effect that light intensity has

on pupil diameter, many emotional reactions can cause dilation or constriction. The primary emotional states associated with pupil dilation are excitement, comfort, pleasure, and displeasure. Pupil dilation also occurs in response to impending threat or pain. In general, more unappealing stimuli are responsible for pupil constric tion. An example of this type of stimulus would be a cross-eyed baby. Some stimuli, such as concentration camp scenes, initially cause pupil dilation, but with repeated exposure cause pupil constriction.

In general, the pupil can range in diameter from 1.5 to over 9 millimeters, and can react to stimuli in as little as 0.2 seconds.

Reported methods of measuring pupil diameter in clude: using a short-focus telescope with adjustable cross hairs, or taking pictures of the eye as a stimuli is presented then later measuring the pupil diameter with a ruler. Although each of these methods will produce valid results, the extensive time investment required and potential for human error are significant problems. With the increased interest in pupillometry, there has

also been a corresponding increase in the interest di rected towards saccadic eye movements. The different movements made by the eyes can be broken down into two main categories: voluntary fixation movements and involuntary fixation movements. The fixation move ments, or jumps, are called saccades, while the move ments are called opticokinetic movements. The sac cadic jumps made by the eye are so quick that only 10 percent of the time is the eye engaged in movement, the rest is spent fixating. The voluntary movements, called macro-saccades,

are responsible for selecting the visual target of interest. Once a target has been acquired, the involuntary fixa tion mechanism assumes control. These involuntary movements, called micro-saccades, are responsible for maintaining target fixation. Many different methods of tracking saccades have

been reported in the literature. These methods range in technology from using the electrooculogram (EOG) to using magnetic fields to induce a current in a coil imbed ded into a contact lens. Of the saccade tracking tech niques reported in the literature, almost all required some sort of invasive interface with the subject. These invasions are often uncomfortable and inhibiting to the subject.

10

5

20

25

30

35

45

50

55

65

2 SUMMARY OF THE INVENTION

The present invention is a specific application of the type of apparatus disclosed in U.S. Pat. Nos. 4,950,069 and 4,836,670 which disclosures are hereby incorpo rated herein by reference and made a part hereof. The present invention utilizes a relatively inexpensive

computer based apparatus and method for determining the lookpoint of a subject's eyes as well as the pupil diameter while the subject is viewing an image such as picture or text or a combination of a picture and text. These lookpoints and pupil diameters are nomintrusively recorded and may be correlated with a meaning thereof. The apparatus and their method may be used for testing the images such as a choice for new packag ing or testing the psychological state of mind or the manner by which a person reads text. The basic plat form of the invention can be a measuring apparatus and method for performing a variety of psychological ex periments or test of stimuli and, based on empirical data or other information, have various meanings assigned to the results. The most notable advantage is the non-inva sive method used to monitor the subject. A second major advantage provided with the technology is real time processing. The system can gather information concerning both pupil diameter and lookpoint and then react to this information as it is gathered. This capability opens the door for many types of testing experiments previously unavailable. The system provides not only a non-invasive method for eye tracking, but also an auto mated environment for stimuli presentation. The appa ratus can track a subject's eye-gaze with a resolution of 1 to 1.5 of eye rotation and a speed of 15 frames/second. Both the resolution and speed is expected to improve as the underlying technology is improved. Many things haven't been explored further because of

the measurement apparatus used to measure pupil diam eter are either too slow, don't measure accurately or frequently enough, are too uncomfortable, are invasive, or don't measure in real time, or they aren't portable. All of these things inhibit the use of pupil diameter in further research of psychophysiological phenomenon. The present system provides all of these extra advan

tages and permits looking into areas further which ha ven't been explored. The use of the invention is primarily in psychological,

technological, industrial, military, ethical, and political areas. As an example, pupil diameter is related to mental workload. Mental workload estimation quantifies the information processing demands upon a human opera tor. Pupil diameter is linked to mental workload. By assessing a person's mental workload, tasks and equip ment can be designed to better suit the workload of a user and tasks can be reallocated between humans and machines so that a human is not overloaded with work. Also, decisions can be made on who you want to select to operate certain machines. Other applications include lie detection, drug use marketing applications, pictures involving what packaging arouses people's interest, etc. When people are aroused their pupil diameter usually dilates. Still other applications include economic im pacts. As an example, if someone is overloaded then they are more likely to get sick or and the employer is going to have to pay for the sickness, etc.

In marketing applications, the apparatus permits the determation of the lookpoints on specific images. One example uses two pictures, both pictures containing the same product with different packages and the apparatus

5,204,703 3

determines which picture the person prefers to look at, or which picture their pupil diameter is larger when the packages are viewed. Another example, usable for determining intelligence

of children in the toddler age, is to have a picture of 5 mother and a stranger and see who the baby looks at first and what part of the mother or stranger the baby looks at and which one they look at more often. Also, a determination can be made to what extent there is a correlation with pupil diameter. Areas of interest on the screen may be defined by a

box technique. The box is defined as that area you are interested in seeing how often somebody is looking into. So, if you want to analyze that area, then you need to define it. A box file is created. The box file contains up to 15 different boxes and each of those 15 box defini tions are described by the upper left and the bottom right corner of each box. A box is defined by entering the coordinates of the top left corner and of the bottom right corner. The box can be placed around any area of 20 the computer screen. Also, because box coordinates are based on the screen, and not on the picture, box coordi nates can be used for different pictures. This is helpful if it is desired to see if somebody looks at the left side of the screen more often than the right side of the screen. 25 By defining boxes on the left side and the right side and they would be independent of the pictures that were being viewed on the screen. The system permits having a model and a package in each image, and determine whether a person looked at the model more or at the package more and also would permit a comparison between the two images. Originally it would only be determined how often somebody looked at one image or the other but now it can determined if the person is paying attention to the model or if they are paying attention to the package. This permits a determination of what to improve. If the total picture is viewed, this necessarily would not show what specific things were going on. I mean, what specific areas were the look points. The lookpoints can be superimposed but in order 40 to analyze the number of lookpoints, a manual count of the lookpoints in a certain area would be required. The use of boxes is a way of simplifying the analysis of what a person is looking at.

DESCRIPTION OF THE PREFERRED EMBODIMENT

In this description, including the accompanying drawings, there is shown and described a preferred embodiment of the invention. Others skilled in the art 50 will understand the invention and the principals thereof and will be able to modify the preferred embodiment and embody the invention in a variety of forms, each as may be suited for the conditions of a particular case. BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 shows schematically a typical set up of the

apparatus; FIG. 2 shows schematically the test and camera ar

rangement as viewed by the subject; FIG.3 shows a typical relationship between the pupil

as outlined by the bright-eye effect and the glint; FIG. 4 shows a test stimuli where two different pack

ages are presented; FIG. 5 shows the test stimuli of FIG. 4 with look

points superimposed thereon; FIG. 6 shows the computer screen with the main

menu pulled down;

10

15

30

35

45

55

65

4. FIG. 7 shows a test stimuli having two models pres

ented; FIG. 8 shows the test stimuli of FIG. 7 with a part

outlined in a box; FIG. 9 shows the test stimuli of FIG. 7 with a look

point superimposed on the left model and the pupil diameter shown in the left lower box; FIG. 10 shows the display of an analysis of X and Y

lookpoints and pupil diameter; FIG. 11 shows a summary of an entire test; and FIG. 12 shows individual summaries.

DETAILED DESCRIPTION OF THE INVENTION

With reference to FIGS. 1, 2 and 3 there is shown the basic apparatus and technology of this invention in schematic form. This is more fully explained in U.S. Pat. Nos. 4,950,069 and 4,836,670 mentioned supra. The gaze routines as more fully set forth in the appen

dix and below provide the lookpoint and pupil diameter of a user to applications software in real time. The for mat of information exchange between applications pro grams and the gaze routines is standardized so the same gaze engine is common to all applications. Individual application packages decide how or if the lookpoint and pupil diameter information is to be used. The hardware utilized by the gaze routines to pro

duce the lookpoint information includes generally a computer with frame-grabber board 24, a infrared sensi tive standard surveillance camera and lens 20 and an image presentation device such as a computer monitor 26, and a near infrared light emitting diode (LED) with a light condensing lens 22. The surveillance camera 20 is situated underneath the computer monitor with ca bles 28 feeding the signal to the frame grabber-board in the computer. Mounted coaxially with the lens is the LED and light condensing lens. The arrangement is assembled into a desk or console 30 which is at a conve nient height for a subject 32 to sit on a chair 34 in front of the monitor 26. When the camera is pointed at one of the user's eyes 36, the LED 22 floods the user's face with near infrared light, producing the effect commonly called the bright eye. The bright eye outlines the pupil 38 is the same sort of effect that causes a cat's eyes to glow at night or a person's eyes to glow red in a flash picture. Two features are immediately apparent in the infrared image of the eye: the glowing bright eye effect and the virtual reflection of the LED off of the surface of the eye, which is called the glint 38. It is the presence of the bright-eye and the glint which prompts the tech nology to determine lookpoints as the distance and direction between the center of the pupil and the glint is used in making the determination. When the gaze routines are told to deliver a lookpoint

(i.e. answering a request from applications software for the current screen location at which the subject is look ing), the first operation is to grab a frame. Grabbing a frame simply involves taking the image currently in the frame buffer and holding it for analysis. Taking advan tage of the image contrast, the gaze routines can scan the frame and locate the pupil and the glint. The vector relationship between the pupil center and the glint cen ter is used by the gaze routines to calculate a lookpoint. If this lookpoint is determined to be outside of the bounds of the computer display the applications pro gram is notified and the data processed accordingly (depending upon the application, off-screen lookpoints may be processed or ignored). Since the edges of the

5,204,703 5

pupil are used to determine the location of the pupil center, the pupil diameter is generated in the process.

In order to use the vector connecting the pupil and glint centers to calculate a lookpoint, the gaze routines must first be calibrated. Calibration involves placing icons (targets a which the user is instructed to look) on the computer display and, while the user is looking at them, processing image frames and isolating the pupil/- glint vector. As the icon is moved to different locations on the screen, the vector is continually calculated. When all of the calibration icons have been presented, the calibration data is fit to a curve thus producing the equations to be used in calculating the gaze point. The final operation the gaze routines perform is to

save the previous locations of the pupil and glint center. When the routines receive the next request for look point information, the previous pupil and glint center locations are used to speed up the search. The gaze routines complete the search for the pupil

and glint centers in approximately 10 ms (milliseconds). Due to limitations in the frame-grabber hardware, a new frame, and therefore a lookpoint, can only be pro duced every 33 ms but this will improve with future hardware. The gaze routines achieve this speed by tak ing advantage of information contained in the previ ously processed frame to find the new pupil and glint centers. If for some reason the eye is lost, global search routines are activated which scan the entire frame for the pupil and glint. Once the global routines have been successful in lo

cating the eye local routines assume the task of process ing subsequent frames. The local search routines first perform a spiral search out from the previous glint location to find the new glint center. This method takes advantage of the fact that the glint moves very little between frames. If the search routines are successful in finding the glint, the local pupil search routines are called. Otherwise, the global routines are reactivated. The local pupil search routines use the previous pupil

center and expected pupil diameter to quickly find the

10

5

20

25

30

35

edge of the new pupil image. By taking advantage of 40 pupil diameter information, the software can jump to the expected location of the pupil edge and eliminate the need to scan all the way across the eye. The number of pixels scanned can be reduced up to 70% using this procedure. Since accessing pixels in the frame buffer is relatively slow, reducing the number of pixels scanned results in a proportional increase in Speed. The technology can detect eye rotations of as little as

1.5°. This corresponds to approximately 84 boxes on a screen. The actual accuracy of the system varies with head position and may vary with gaze direction, de pending upon which curve fit is used and the positions of the calibration points. The calibration curve fit adopted with the gaze rou

tines was a first order model with a y dependence. The equation defining this fit is as follows:

where, in calculating the x component of the lookpoint, j is the x component of the pupil/glint vector and j2 is they component of the pupil glint vector.j and j2 are reversed when calculating they component of the look point. In order for they dependence to be stable, the shape of the region bounded by the calibration point positions must be the same as the shape of the area where the user will be looking (i.e. the calibration point positions extend to the borders of the screen). If a more

45

50

55

65

6 simple calibration procedure is required (i.e. less cali bration points for the subject to follow), a curve fit with no y dependence may be used. The current curve fit of choice for this situation is a simple second order model,

k = Bo-Blil--Bi.

The target accuracy for the system is 0.5' of eye rotation or better. With this accuracy, the system would be on the threshold of detecting micro-saccades. These micro-saccades are important for some testing and fu ture improvements will permit an accuracy and speed that they Will be detected using basically the same apparatus and methods used in the present invention. One of the primary variables of interest in measuring

response is pupil diameter. It is for this reason that the gaze routines were adapted to produce not only look point information, but pupil diameter as well. The sys tem does not use any sort of range finding capability (no information concerning the distance from the head to the camera), so an absolute value for pupil diameter is not calculated; however, a relative value is produced. Since relative changes in pupil diameter are the main interest and not actual pupil size, this has proven to be sufficient but a range finding capability can be added if desired. One of the most critical steps in making the eye-gaze

engine work is the calibration. If the calibration is not performed correctly, or the software fails to find the pupil and glint accurately during the calibration, the coefficients generated can be useless. The gaze routines test the calibration for accuracy; if any calibration icon position calculated with the calibration equations differs from the actual position by more than 5%, the calibra tion is rejected. The result of verifying the calibration provides reliability in the data produced by the eye gaze engine. The basic cornerstone of the testing system of the

present invention is that people will react to certain stimuli and the system detects these reactions. Since the testing system itself is usually the presenter of the stim uli, it is a relatively easy task for the computer to corre late the reaction data to the stimuli. If, however, the stimuli originates from another source, such as a record ing, projected slide, or picture, the system will still allow data points of significance to be labelled. These labelled data points can be used to match the reaction data to the correct stimulus. An example of this practice would be using a recorded auditory stimulus and label ling the data point taken when recording was started. The types of stimuli the system is capable of present

ing can be divided into two categories text and pictures. Each of these stimuli types are usually presented to the subject on the computer screen. The picture classifica tion can be further broken down into those picture which fill an entire screen and those pictures which are displayed in different boxes or a divided screen. The single picture display has been provided so that a

subject's reaction to a single scene can be monitored. The larger image is more intense and provides greater detail, possibly leading to stronger subject reaction. The images which are displayed as groups such as groups of four are correspondingly smaller. With this reduction in size comes a proportionate loss of detail. The advantage of this display format is that a subject can not only register a reaction to the individual images, but a prefer ence of certain images over others can be indicated.

5,204,703 7

The design of the system does not limit it to perform ing tests only with stimuli presented by the system. Tests can be given where other forms of stimuli presen tation are used. Performing tests which use senses in addition to sight for stimulation (i.e. smell and auditory stimuli). Once a test has been designed and the stimuli entered

into the system, a sequence of subject tests can be run. The subject is lead into the testing environment (usually a small, quiet room) and placed in front of the computer monitor. If the test has already been completely set up, all that will be displayed on the screen is the instruction, "Press a Key'. At this point the administrator of the test, sometimes referred to herein as the operator, will explain to the subject how the test will be run, and the requirements of calibration. Once briefed, the adminis trator will start the test (by pressing a key) and the first stimulus will be presented to the subject. The system will automatically move from stimulus to stimulus as the test is administered. After the test is complete the software returns to a simple menu of options. At this point, the administrator can elect to present another test, or proceed to the next subject. All gathered data and data analysis are hidden from the subject. Once a subject has been tested, it is a relatively

straight forward process to analyze the results and ex tract useful information. The system provides reliable information concerning the subject's lookpoint and relative pupil diameter. The gaze engine may also be used to extract information concerning saccadic veloci ties and other parameters. For individual or unrelated stimuli screens, the result

ing data files are analyzed one at a time. The data is processed to find average pupil diameter, and, when appropriate, broken down into lookpoints lying on the displayed images. In this way a subject's preference for a certain image or portion of the screen can easily be discovered. The results from stimulus screens can be processed as

groups and multi-factor tests can be designed which are sensitive to possible stimulus effects. Using the group processing capability, the tests can be designed such that the same figure appears in different screen locations (assuming that the multiple picture screen model is being used) at different times. When the results are being processed the system can analyze each individual screen, examining quadrant effects, and then process the screens as a group, focussing on stimulus effects. The software system developed for testing was de

signed as a general package capable of presenting a wide variety of stimuli to a test subject. The stimuli are grouped into two different categories: text and pictures. Obviously, both text and pictures can be used simulta neously. Extended test sequences can be generated which are composed of any combination of stimuli in these categories. Tests can also be run with nothing displayed to the subject. With the screen left blank, the operator is free to test a subject's response to other categories of stimuli.

Since many of the potential users (i.e. experimenters) of the software system have fields of expertise that do not include computer operation, a user friendly envi ronment is very important. To realize this environment, the software is designed using menus and windows to clearly and efficiently provide the experimenter or op erator with information. In the event of an error, the software informs the user and allows the mistake to be corrected if possible.

O

15

20

25

30

35

45

50

55

65

. 8

The menus are configured in a simple tree structure. Menu selections can be chosen by moving a selection bar with the keyboard arrow keys, or by typing the hot character associated with each menu option. After a menu option has been selected the user is either pres ented with another menu, or the selected operation is performed. If at any time the user selects an operation that will result in the destruction of data, the system requests confirmation from the user. The tree structure used in the menus partitions the software's tasks into three different sections: entering data, performing tests, and analyzing data. The forms of stimuli that the system is capable of

presenting to the subject are text and pictures or a con bination thereof. These stimuli are displayed on a stan dard EGA graphics screen for a personal computer. Since the system allows the experimenter or operator to create the text and pictures used for testing, the differ ent possible stimuli are unlimited. When entering text stimuli into the system the experi

menter is presented with a blank area of the screen in which to type. The screen is treated much like a blank piece of paper in a typewriter, imposing no format rules on the entered text. The characters are entered through the keyboard and can be displayed in any of the com puter's available colors. The graphic stimuli may be entered into the system

through the surveillance camera used by the eye-gaze algorithms to follow the user's eye. The desired image is simply held in front of the camera and the frame frozen. Once captured, a variety of image processing functions are available for image enhancement. The image can be transferred to the computer's graphics screen and stored to later be displayed during a test. The images transferred to the computer screen, have

been classified into size categories: full screen images, part Screen and box images. Full screen images cover almost the entire computer screen and are displayed one at a time. The part screen images and boxes cover only part of the screen and can be displayed as groups. As one example, the images captured by the present

frame-grabber have a spatial resolution of 512x480 and a contrast resolution of 256 grey levels. The image quality of this type of image is consistent with that of a standard black and white television. An EGA (En hanced Graphics Adapter-a standard graphics card) graphics card in the computer, however, has a spatial resolution of 640x350 and can display 16 colors at one time. This will give 16 different shades of blue that can be assigned as the EGA's color palette. Image intensi ties from the frame-grabber are translated to one of these grey level values and displayed on the EGA Scree.

Even though the frame-grabber images have the po tential to span the 256 gray levels, they normally use only part of the range. As a result, the image transferred to the EGA screen only uses a comparable portion of the 16 grey levels available on the EGA. This low image contrast, while not a problem with 256 grey levels, can cause significant image degradation with only 16 grey levels. Other reasons for pre-processing the images sent to

the EGA screen include: image simplification, and image reduction. Some of the images to be processed may be simple line drawings. Camera noise and lighting gradients can make the images appear more complex than is necessary. For the stimuli to be as effective as

5,204,703 possible, these simple images should be as artifact free as possible.

In order to maintain good image contrast when trans ferring pictures to the EGA screen, an image equaliza tion is done. This process involves altering the image's histogram to have a flat power density function (PDF), and therefore one which extends through the entire range of grey levels. After equalization, an image trans ferred to the EGA graphics screen will utilize all 16 of the available grey levels. When displaying simple line drawings on the EGA

screen, it is normally preferable to use only two grey levels. For these cases, software functions allow the experimenter or operator to convert an image to only black and white (no grey levels in between). When displayed on the EGA screen these images resemble the original drawing much more closely.

Finally, in order to make the images small enough to display on the EGA screen in groups such as four, the frame-grabber images must be reduced. To accomplish this a simple reduction algorithm is used which reduces a frame to one ninth of the original size. The software uses an averaging method for reduction so camera noise is reduced in the process. Once test stimuli have been entered into the system,

they can be presented to a subject during a test. While the stimulus is being displayed on the screen the system will monitor and store the lookpoint and pupil diameter of the subject. Several system features allow for slightly different testing environments. The experimenter or operator can vary the number of screens presented to the subject during the test, vary the length of time each screen is displayed, and turn off or on lookpoint feed back to the subject. Usually, lookpoint feedback to the subject is not used during the test.

10

5

25

30

35 In order to increase the speed of testing and make

reproducing tests easier, the software provides the ca pability of generating test batch files. A test batch file contains a list of test stimuli to be presented to the sub ject, and a list of files in which the results are stored. A batch file may be programmed to present as many or as few stimuli screens as desired. Under certain conditions it may be desirable to pro

vide the subject with lookpoint feedback. Before the system executes a test, the experimenter or operator is asked whether or not the subject's lookpoint should be displayed. When feedback is desired the computer will place a small round icon at the screen position where the gaze routines have calculated the subject to be look ing. The subject's behavior will be influenced by this lookpoint feedback. Another operational parameter of the testing proce

dure is the test length. Before the test sequence is pres ented the experimenter is asked how long each screen should be displayed. Screen display time is usually mea sured in number of lookpoints calculated instead of seconds; in other words it takes data until so many data points are taken instead of for a given amount of time. This ensures that system delays in finding the pupil are not taken away from test time. Time may also be mea sured in conventional units (i.e. seconds). Appropriate data analysis functions of the results

obtained with the testing software are important and a variety of methods for presenting the test results have been provided. These functions have been designed to both present the raw data to the experimenter or opera tor and to summarize the information contained in the data file.

45

50

55

65

10 When the experimental sets of stimuli presented to

the subject do not have any direct relation to each other (i.e. the same image does not appear on more than one screen), the results can be processed separately. The different possible processing functions include: listing the raw lookpoint and pupil diameter data, graphing the raw data verses time, superimposing the data back onto the stimuli, and summarizing the results. Listing the results involves writing to the computer screen the x and y lookpoint information, the pupil diameter and any data point labels added by the experimenter.

Superimposition is useful when the experimenter wants to correlate the subject's lookpoint and pupil diameter to a particular part of an image. With this function, the stimuli are first re-displayed on the com puter screen. Each lookpoint from the data is then dis played as an icon at the determined location on the stimulus image, and the pupil diameter is shown in a corner of the screen. The data can be presented in real time, or stepped through under keyboard control. Also, the program can be modified to number the icons to show the sequence the lookpoints were generated and the dwell time at a location. . Another method of data display, graphing, is useful

for following trends in the data. The data are separated into X lookpoints, y lookpoints and pupil diameters. Each parameter is then graphed with respect to time. Pupil diameter changes and strong saccades are easily seen with this display method.

Still another method of reviewing the results is with a data summary. This option is most useful when stimuli are presented as groups of pictures. The summary basi cally involves categorizing each lookpoint by the quad rant or section or box in which it appears. The mean and standard deviation of the pupil diameter are then calcu lated for the data points falling on each separate picture, and finally for the whole data set. This method makes readily apparent the subject's specific reactions to dif ferent stimuli. Using the multiple images on a screen test model such

as four different images, experiments and tests can be designed in which the same stimulus is presented on a number of different screens. On each of these screens, the stimulus may be located in a different quadrant, eliminating any effects from quadrant preference. In these cases it may be desirable to extract from each data file (a data file is made for each screen displayed during the test) those lookpoints which fell upon this certain image. To do this, the administrator may select a group of related files and instruct the software to summarize all of the lookpoints falling on all of the images repre sented in that list of files. The summary generated is the same as for the separately processed results. For each image (one of the four images presented on a screen), the total number of lookpoints and the mean and stan dard deviation of the pupil diameter is calculated.

Certain quadrants attracted more lookpoints from subjects than others, possibly stemming from reading habits. The upper two quadrants are significantly more preferred than the lower two. There is also a strong correlation between the number of lookpoints a stimu lus received and the stimulus complexity. The least complex object receives the fewest lookpoints, while figures having the highest complexity receives the most. The relationship of pupil diameter to stimulus com

plexity and stimulus quadrant has been evaluated. In none of the examined relationships were any significant effects found. Pupil diameter did not have any strong

5,204,703 11

connection to complexity, presentation sequence, or quadrant. An easy method of entering pictures into the system

involves holding the desired stimuli in front the surveil lance camera and freezing the image. Once frozen, the image is enhanced and transferred to the EGA graphics screen. The major problem with this method is the relatively poor quality of images that the system is capa ble of capturing. Image quality suffers because of varia tions in lighting and glares off of the original stimuli held in front of the camera. A more robust method of picture entry utilizes a scanner or some other method of picture entry which is designed for image capture of higher quality images. The major advantage, however, of the current method is that no extra equipment is necessary. Some experiments o tests involve target tracking. In

these scenarios the subject is shown a screen with a

10

5

moving target. After the subject is told to follow the . target as closely as possible, the system would track the subject's lookpoint. The performance rating generated from the test results would be based upon the subject's ability to track the target. This may be used for drug testing, fatigue, some illnesses, etc. One of the primary capabilities of the system is real

time data analysis and the decisions made based on that analysis. The normal batch file organization used for administering the tests is purely linear; regardless of what reaction the subject has the stimuli are always presented in a defined order. Processing the results as they are gathered allows the system to evaluate how the subject is reacting to certain classifications of stimuli and according to his reaction display more or less from this class. Described simply, the test that a subject re ceives would depend on that subject and his reactions during the test. The testing system may use a second screen for con

puter output. The system shown in FIG. 1 displays all messages to the experimenter or operator and all stimuli to the subject on the same screen. With only one screen, trying to hide information about how the experiment is run is often difficult and cumbersome. If, however, the experimenter (operator) uses a private screen which the software use for test control then the required setup procedures easily be hidden from the subject. The addi tion of the second screen also allows the software to provide the experimenter with real time data analysis, while the subject's screen is presenting the test stimuli. To test a subject, the first step is to register the sub

ject. The next step would be to calibrate the subject. Load in the batch file or test file to be used for the test. The batch file contains a series of images, which is the pictures and the text which are to be displayed for the test. A key to begin the test is pressed and that's when the calibration begins. Once the subject goes through the calibration and the calibration is a good calibration, then the apparatus goes on and starts displaying the stimuli. Each screen could contain one large image or could contain two or more smaller images. Each screen is basically designed to collect a certain amount of look points. Once it collects a predetermine number such as 250 or so data points, the system goes on to the next screen and displays different images. This repeats itself until all the desired images are viewed and the test completed. About 250 set points are collected in just a few seconds. FIG. 6 is the main menu from which a selection is

made. The first choice is "Digitize and Store an Image"

20

25

30

35

45

50

55

65

12 which is highlighted. That is where the camera itself can be utilized to take a picture in front of the camera, digitize it and store that image. Also, the image can be digitized and stored separately and placed in the mem ory to be pulled up or during the test.

"Generate and Save a Text Screen' is when the text material can be entered rather than being photo graphed. That can be entered from the keyboard.

"Perform Saccadic Motion Test' is when the test itself is being created. The "Analyze Results' is where all of the results files can be viewed. Lookpoints can be superimposed over images based on the results files.

"Register Subject' is the first step where a new sub ject is entered at the beginning of a test. Finally, "Quit" is utilized to leave the main menu.

FIG. 7 defines a box file. Two models are shown as being represented in a picture and a box file is being formed for later review. Basically, the option called "Define a Box Option' is selected and it first asks the user to define the upper left corner of the box by using the mouse. The mouse is moved to the desired area of the screen and the left mouse button is actuated to de fine the top left corner. Then the mouse is to define the bottom right corner. The 15 boxes represent the storage capacity of this particular program and the mouse 40 is pointing at box. 7 in FIG. 7. After that box number 7 has been selected, then in FIG. 8 a certain area for that box to represent is selected around the left shoulder of the model on the right. The mouse position is shown by an icon 40 in the shape of a hand. The picture may be scanned for a predetermined period of time which is usually dependent upon the number of data points de sired keeping in mind that usually it takes 30 millisec onds per data point. A typical scan may be 5 seconds. FIG. 9 shows the superimposition of a lookpoint and

the pupil diameter on the picture just scanned. It shows that the subject looked at the right eye of the model on the left with the pupil diameter being 43 computer screen graphic pictels wide.

FIG. 10 is an overall analysis of the results of the subject looking at a picture. The upper left quadrant and the upper right quadrant show the X lookpoint and Y lookpoint, respectively, where 250 lookpoints in total are viewed. The X lookpoint vertical numbers are the lines of this

particular EGA graphics screen which has 640 vertical lines so the top would be the far right line (640th line) and one would be the first line on the left. The Y lookpoint vertical numbers represents the

horizontal lines of 350 with the number 1 being the top line and number 350 being the bottom horizontal line. For screens having greater resolution, this would change accordingly with the screen and graphics uti lized, The lower right hand corner of FIG. 10 shows the

change in pupil diameter with the light line representing the vertical (or whatever line is used in the graphics) and the dark line representing the horizontal. The verti cal line in the lower right hand quadrant represents the number of pictels and the horizontal line represents the various lookpoints which are 250. The lower left corner shows the specific identification of the subject whose pictures are analyzed. It shows Sally and Estelle who are the two models shown and that 250 data points have been taken. FIG. 11 shows a summary of the entire test, the image

files being the two models. The number of lookpoints are 250, u(pd) is pupil diameter and is 45 pictels. Sigma

5,204,703 13 14

(pd) equals in effect the sum of the pupil diameters The invention has been described herein and the which is 11,361 and Sigma (pd?) is the sum of the pupil appendix included with the specification completes the diameter squared which is 528,635 and this is used to disclosure. The invention provides a flexible, inexpen calculate 's', which is the standard deviation and sive, user-friendly apparatus and method that can be equals 7.04. Standard deviation is an indication of how 5 used as a basic platform or engine for testing and experi steady the pupil diameter is and would show to some menting with a wide variety of psychophysiological extent the degree of excitability of th subject. reactions to visual stimuli which may be augmented FIG. 12 shows the individual summaries between two with other sensory information such as sound. Many

boxes. The one on the left being a box for Sally and the embodiments may be created using the invention and it one on the right being the box for Estelle. They shows 10 is to be understood that such embodiments and modifi that in this case there were 143 lookpoints for Estelle vs. cations can be created without departing from the spirit 103 lookpoints for Sally indicating the box surrounding and scope of the claims below. the total picture that the subject found Estelle more attractive to look at than Sally. However, there was a APPENDIX higher degree of excitement in the case of Sally because 15 This appendix contains a listing of the software usable the standard deviation was higher. in the present invention and includes an introductory FIGS. 4 and 5 shows a comparison test. The left side user's guide.

of the screen shows a new package of one design and This appendix is hereby incorporated in and made a the right side shows the new package of a second de- part of the specification of said invention. Any and all sign. When tested, the subject found the package of the 20 copyrights relating to this appendix except those neces second design more interesting and would normally be sary for fulfilling any patent law or rule requirements the design selected. are retained by owners.

Part 1: Introduction

ERICA stands for Eyegaze Response Interface Computer Aid. The ERICA system is an eye gaze computing system which estimates where a user or subject is looking and measures pupil diameter. It can also be used to identify eye blinks. The setup of the system is shown in Figure 1 below.

Computer Monitor

Video Monitor

Stimulus

Computer G) OOOO //

Camera and LED Keyboard

A variety of applications have already been considered for this technology. Among these are visual scanning patterns, visual acuity, and mental workload. The purpose of this manual is to explain the use of the software written to analyze visual scanning patterns (also

5,204,703 15 16

called Saccadic eye movements) and pupillary responses to visual stimuli. Part 2 describes the requirements of the ERICA system and how the system measures eye behavior. Part 3 describes the purpose of the software, providing an overview of its functions and capabilities. Part 4 describes how to create psychological test stimuli. Part 5 describes how to define important areas of stimuli and how to combine stimuli into executable tests. Part 6 explains how to register subjects for and how to run these tests. Finally, Part 7 provides explanation of how to analyze and interpret the results of the tests.

System requirements Hardware.

In order to use the ERICA psychological testing software, you need the following hardware:

1) An IBM or IBM-compatible personal computer with an MS/DOS operating environment and at least 1 megabyte of hard disk space.

2) An EGA (or better) graphics monitor. 3) A keyboard and mouse.- : 4) A video surveillance camera, e.g. Sanyo VD360 5) A framegrabber board, e.g. Imaging Technology's PCVision Plus

framegrabber 6) A light emitting diode (LED) 7) A black and white video monitor

Installation of Software Before copying the files into the computer, the following directories must be created on

your hard drive:

SACCADE SACCADENBATCH SACCADENBOX SACCADENPICTURES SACCADENRESULTS SACCADENTEXT

Using the make directory, change directory, and copy commands, all of the software contained on your floppy disk (usually in the A or B drive) can be installed. In order to create these directories, at the MSNDOS prompt (a greater than sign D), type the following sequence of commands, pressing the "Return' or "Enter” key in between commands. The prompt will appear every time you press either of those keys.

D cd\ Dnd Saccade D cd saccade Dnd batch

5,204,703 17 18

D ind box > md pictures > md results Dnd text

Now to copy all of the files into the appropriate directories, enter the following sequence of Commands:

D (If your floppy drive is B, type b:) > copy "...btic c\saccade\batch (The c: on this line indicates that the hard

drive you will be working with is the C drive. If it is D, you would used:, etc. Similarly, in the following commands, use the appropriate letter for the drive you are working with)

> copy .box c:Waccade\box > copy "...pic c:\saccade\pictures > copy "...res c:\saccade\results > copy .txt c\saccade\text > copy '..exe c:\saccade > C. > cd\saccade

The main executable files are called SACCADE.EXE and SANALYZE.EXE.

How the System operates: : Figure 2 illustrates a hardware configuration of the system. An infrared light emitting

diode (LED) is mounted on the center of the camera lens. As this light shines on the user's face three basic types of reflections occur. The camera captures these reflected images every 30 milliseconds. The light reflecting off the face looks relatively dark compared to the light reflections off the eyes. One reflection, known as the glint comes from light reflecting off the corneal surface of the eye. It looks like a very small and bright dot. The larger, but somewhat less intense reflection is known as the bright eye, a reflection of light which has entered the pupil, bounced off the retinal surface, and passed back out through the pupil.

When these images are passed by the camera to the framegrabber they are digitized, that is the intensities of the different reflections are converted to numbers. Each digitized image is routed from the framegrabber to the gaze routines. It is the gaze routines which use the differences in light intensities to locate the user's eye in the image. They use the relative positions of the glint and the bright eye to measure each lookpoint, an x-y coordinate indicating where someone is looking in the area around the camera. The diameter of the circular bright eye is interpreted as the pupil diameter, and a quick decrease and increase in this diameter is identified as an eyeblink. The x-y coordinates represent graphics coordinates on the computer screen. Pupil diameter is measured in terms of the number of pixels in the image counted along a diameter of the bright eye. During these different eye movements, the video monitor shows the relative positions and sizes of the bright eye and glint as shown below.

5,204,703 19

Dilation Constriction

Part 3: Introduction to the Software

The purpose of this psychological software is to provide a means of investigating scanning patterns and preferences of images and text. The software allows you to

1) Create stimuli in textual and graphics form 2) Combine stimuli in any number of ways into a batch test 3) Define important areas of interest on stimulus screens 4) Run a batch test 5) Review raw and summarized data in written and graphic form 6) Write summarized results to a DOS file, LOTUS file, or printer

From the creation of the stimuli and test to the compilation and reduction of the results, an example will be used to demonstrate the software.

Starting the program In order to run the main executable software, you would enter the SACCADE

directory and type in the name of the executable file (filename ends in .EXE) at the MSWDOS prompt. For example, to run the saccade results file writing program (SANALYZE.EXE), you would type

D cd\saccade (not necessary if you are already in the Saccade directory) > sanalyze

For right now, however, it is necessary to create and run a complete test from scratch. The SACCADE.EXE executable file provides these options. So type

5,204,703 21 22

> cd\saccade > Saccade

At this point the screen should be blue with a white menu in the center. The line on the top left indicates the name of the menu. The line on the top right beginning "Coreleft = " indicates how much dynamic memory is left in the computer to use during the program. Memory will be used by the program when loading in graphics, reading in results files, etc. At such times this memory would decrease.

The bottom line of the screen tells you if there are any messages (there should not be any right now).

You should be looking at the Main Menu of the program. The following choices (with their functions in parentheses) are listed in the Main

Menu:

Digitize and Store an Image (Creating pictorial stimuli using ERICA's camera) Generate and Save a Text Screen (Creating textual stimuli) Register a Subject (Entering information about the subject to be tested) Perform saccadic motion test (Creating and running tests on subjects) Analyze Results (Looking at results of tests) Quit (Exiting the saccade program back to DOS)

Selecting Options Notice that using the left, right, up, and down cursor arrows on the keyboard move the

red bar which highlights the current menu choice. A menu choice can be selected either by pressing the Center> key when the bar is highlighting your menu choice or by typing the hotkey corresponding to your choice. For example, if you wanted to create textual stimuli to be used later for testing, you would want to select the option "Generate and Save a Text Screen" either by 1) Typing the hotkey T (shown in bold face in the manual, and in yellow on the screen) O

2) Pressing return after using the arrows to position the highlight bar over that menu option.

Part 4: Creating Psychological Test Stimuli

Creating Pictorial Stimuli From the Main Menu, select the first option: "Digitize and Store an Image." The

Picture Menu should appear with the following options:

Grab a Frame (Take a photo snapshot of object in front of camera) Continuous (Allow camera to bring in new images) Make image Binary (Converts images to black and white) Draw the Image on a graphics screen (View the stimulus as it would appear in the test) Equalize the image (Increases black/white contrast of the image) Reverse the intensities, Negative - (Produce the inverse of the image)

5,204,703 23 24

Perform image Reduction (Make image smaller) Average image intensities, smoothing (Reduce contrasts in the image) Box on image (Cut image in half) Load image from disk (View a previously created image) Save image on disk (Name the image and store it) Quit (Return to the main menu)

The current version of the psychological testing software allows one type of pictorial stimulus to be displayed: the type which has been created using this menu. (Scanner-created images will be incorporated in later versions). When you first enter this menu, the only available options are "Grab a Frame", "Continuous", "Load image from disk", and "Quit". This is because at the moment there are no pictures are available in the program's memory to work with, and working with an image is what the remaining options are used for.

ing in in to view (the "Load an image from disk" option

To view pictures which have already been created, select the "Load image..." option. If any usable images are in the c\saccade\pictures directory, their names will be listed in the box on the right. Using the cursor arrows, you can select the picture you wish to view (there are no hotkeys for the pictures). As an example, use the arrow keys to highlight the picture entitled farside1. As soon as you press Center->, the menu screen will disappear and on the left side of the screen, you should see a picture from "The Far Side" comic strip. Pressing any key will return you to the Picture Menu. Similarly, you can select this option as often as you like in order to view the images you have available. (Note: At this moment there is no option for working with pictures already saved on the disk. A later version should allow for this option as well).

ing Pict he "Contin rab a Frame" option

In order to create your own image, the framegrabber must be able to receive images from the camera. To check if the framegrabber is already in continuous mode (meaning it is continuously receiving pictures), place your hand about twenty centimeters in front of the camera and move it around. If the image on the black-and-white monitor remains still, the framegrabber is not in continuous mode. Select the "Continuous" option to get into this mode. This option releases any grabbed frames in the framegrabber and enters continuous imaging node.

Choose something small that you want to take a picture of your hand, your face, a newspaper clipping, a magazine photo, etc. Move it back and forth in front of the camera until the image in the black-and-white monitor is as large and focused as you want. For big objects, this may require unscrewing the extender lens from the front of the camera. When you have the object where you want it, select the first option "Grab a Frame", which tells the framegrabber to hold the current image coming in from the camera. If at this moment, the image on the black-and-white screen is not as you wanted, go into continuous mode again and grab a frame as described before until the image on the screen is as you would like it. This is the image to be worked with.

5,204,703 25 26

Viewing Pictures (the "Draw an image on the graphics Screen" option) To see how this image would look in a test, choose "Draw the image...". The program

will ask if you wish to see the Reduced or Full-Sized image. Select Full-Size for now. The image should appear on the graphics screen in different shades of blue. It is likely that the image has not come out in graphics mode as well as desired. That is why the image processing choices are available at this time. Press a key to return to the Picture Menu.

Improving pictures through post-processing The " i "Average image intensiti moothing" option

Generally, the image will come out much better after you equalize it. Watch the image on the black-and-white video monitor as you select the "Equalize the image" option. In effect, this function performs contrast enhancement and allows for greater separation in the drawing on the graphics screen. When the yellow bar on the computer monitor moves to 100% the image has been full equalized. Try the "Draw the image..." option again; first view it "Reduced" and then choose the "Draw the image..." option again to see it "Full-sized".

The Full-Size image is larger and rougher looking then the Reduced version. If you are interested in saving a Full-sized image, you might want to use the "Average image intensities, smoothing" option which will make the graph look less rough. You can smooth the image as many times as you wish. It is advisable to view the image after every Smoothing (or after any image processing) just to make sure you have done all you want to the image. Every time you smooth or equalize your image, you cannot get back the previous version of the image.

Binary" and "Reverse the intensities (Negative)" option

Two of the processing options which may be of interest are "Make image Binary" and "Reverse the intensities...". The first is also an option which is irreversible. It will take each pixel (or point) in the image and make it either black or white, depending on which is more similar. This option is good for newspaper clippings, coloring book pictures, and other pictures for which shading is not of interest. In essence, it is good for cleaning up line drawings held in front of the camera. Reversing the intensities is a reversible option, so you can try it on your current picture (twice to get back the original image). In general, it makes lighter pixels darker, and darker pixels lighter. In other words, it negates the images.

vin work (the "Save image on disk" option

Once you have created an image you would like to keep, select "Save image...". You will be asked to enter the name of the file. Enter a name which is at most eight characters in length, hopefully something which will describe the picture. After pressing Center) describe the picture in further detail if you wish (no more then 60 characters, however). Finally the program will ask if you wish to save the enlarged or reduced version. Think about how you want to use this picture in testing, and how it will look in either size. If you want the picture

5,204,703 27 o r . 28

to take up an entire screen during testing, select "Enlarged." If you want the picture to show up only about half the screen, choose "Reduced."

Congratulations, you have created your first test stimulus. Using "Continuous", "Grab a Frame", "Equalize", the desired image processing options, and "Save image on disk", you can create an entire library of pictorial stimuli to be used for psychological testing.

When you are finished creating pictorial stimuli, choose the "Quit" option to return to the Main Menu.

Creating Tertual Stimuli From the Main Menu, select the option entitled "Generate and Save a Text Screen".

You will be presented with a screen for text entry. The text on the screen can be entered in any format with any mix of colors desired. To use this selection, just move the cursor to the position at which you would like to enter text and type. The arrow keys move the cursor around. It performs much like a typewriter in that there is no wrap around, etc.

Part 5: Creating Tests

Before creating batch files which will run tests, it is first necessary to define which areas of the images you are interested in analyzing (at least current areas of interest). The idea here is to set up the screen the same way you would have it in a test. Looking at the screen, you would then define boxes which surround different regions of the screen. After the test is run, the analysis portion of the software indicates how many times the subject looked at the specified regions, what the average diameter of the pupil was as the user gazed into that region, and the variance of the pupil diameter. Eventually, the analysis will also include information about duration of gaze within each box, time between lookpoints in the box, etc. The analysis options will be described further in Part 7. For now, we will create a new box file and a batch test file.

The simplest way to learn this is by example. Let's suppose you have at least three pictures: the reduced version of farside1.pic, a reduced size picture which you have created, and the enlarged size picture ccmickey-pic. Suppose you also have a text file called fred.txt. The enlarged picture and the text will each take up full screens, so you cannot compare then with other stimuli simultaneously. You wish to run a test on the subject to determine:

1) How much he prefers your picture to the farside1 picture 2) The subject's reaction to the textual stimuli in fred.txt and 3) The subject's reaction to the large picture of Mickey Mouse.

You would want to create a test which will display these three screens in that sequence. Furjermore, you may also want to know:

5,204,703 29 30

1) How often does the subject look at the caption under the Far Side cartoon? 2) How often does the subject look at Mickey's eyes?

For this sort of analysis you also need to create a box file. The box file specifies areas of interest on a screen.

Defining Areas of Interest on the Screen From the Main Menu select the "Execute picture/test batch file" option which will

bring you to the Test Menu. There are a number of options in this menu. Select the option entitled "Define Important areas of images".

The directory listing of pictures will appear on the right side of the screen. Now, remembering the test you are planning on creating (it is helpful to have it written down in front of you), the first stimulus screen should contain two reduced size pictures: a picture of the Far Side cartoon on the left, and on the right, the picture of your choice (one you have created yourself, one which is already provided, or the farside1 picture again). Because you want the Far Side picture on the left, you choose that first by pressing <enter> when the bar is over that name.

* At this point (because of the version you own) you might be asked if the picture is active or inactive... it does not matter what you choose. Choose active for now. Next the program will ask for another picture. If you do not want another picture in that stimulus screen, press <ESC> at this time; otherwise, position the bar over the reduced size picture of your choice and select that picture.

The graphics screen will now appear with the Far Side picture on the bottom left and the other picture on the bottom right. On the top third of the screen is the menu for box definitions. This is the menu which will allow you to define the screen areas you are interested in analyzing. The menu choices are: Remove a box Save a box file Edit a box file,

Define a box View a box Quit

As indicated by the white message below this menu, the menu choices can be activated by 1) typing the number written in each menu box 2) typing the letter with which a menu choice begins O

3) by using the left button on the mouse. IF you do not see a mouse cursor on the screen you cannot create or edit box

definitions from within this program. "Quit" out of the entire program and make Sure your mouse is active and connected.

View a Bor

Click on this option. A strip of 15 boxes appears. Under the strip is a request for you to enter the number of the box you wish to view. You can enter the box number by clicking

5,204,703 31 32

the mouse over it or by typing the character in the box. If you have decided that you no longer want to define a box, you can press '0' or 'Q' or <ESC> to avoid viewing a box.

For our example, Select box 1. You should hear a beep and see a message indicating that box 1 is not defined. Click the mouse or press a key to continue. The point of making this mistake first was to learn that boxes must somehow be defined before they can be viewed. You can define them either by calling up a box file which already contains box definitions (the Edit box file command) or by defining a box for a given number. Try viewing other boxes... you should get the same error message.

If a box is actually defined, a yellow box will appear on the screen. The yellow boundary outlines an area of interest which the user has defined.

Define a Bor You may be wondering how a box is defined, and what is involved in the definition.

Simply enough, to define a box, you need only mark the upper left and bottom right corners of the area you are interested in. Although the description for defining a box appears lengthy, it actually consists of three simple steps:

Step 1) Indicate which box you wish to define Step 2) Click on the upper left corner of the region of interest Step 3) Click on the bottom right corner of the region of interest

Click on the "Define a Box" option. A strip of 15 boxes appears. Under the strip is a request for you to enter the number of the box you wish to define. You can enter the box number by clicking the mouse over it or by typing the character in the box. If you have decided that you no longer want to define a box, you can press '0' or 'Q' or <ESC> to avoid defining a box.

For our example, choose to define box number 1. The white message will then ask you to define the upper left corner of the box. Position the mouse cursor such that the pointed end of the arrow is at the upper left corner of your region of interest. Click the mouse button when you are at this point (remember to use the left button). A small cross appears at the center of the upper left corner. Also, a message appears asking you to define the bottom right corner. Define this corner the same way (positioning the mouse cursor and then clicking the button).

If you have not defined the bottom right corner further down and right of the upper left, you will get a message asking you to define the box. It is actually useful to misplace the bottom right corner if the cross showing the upper right corner is not where you want it to be. That way you can redefine your box.

If you have defined a box properly, the box appears on the screen, outlining the image. In the message area, you will see the coordinates ulx, uly, brx, and bry. These are the x and y graphics screen coordinates of the upper left and bottom right corners of the box you have defined. Click the mouse again to remove the box from the screen. The program now knows

5,204,703 33 34

the coordinates for box 1. If you use the "View a Box" option now, you would not get an error message for box 1. However, you would still get the error for undefined boxes.

Continuing with the "Define a Box" option you can create a set of up to 15 boxes defining important areas of the computer screen.

Save a Bor File

After making definitions you will need to store them in the civsaccade\box directory on the computer. Select the menu's "Save a box file" option. A message will ask you to enter the name of the file under which these definitions are to be saved. You can use any letters or numbers for the filename. You can also use the backspace key to erase typing errors. Press the return key when you have typed in the filename. The program will save the definitions under the name. The file containing these definitions can now be used for tests.

Erasing box definitions A box definition can be removed in one of two ways: 1) Using the "Define a Box" option to redefine a previously defined box

This option is better if you actually need to redefine the area of interest. 2) Using the "Remove a Box" option to remove from the program's memory, a previously defined box. This is better if you do not want to associate a region of interest with the number of the box you choose to remove. To "Remove a Box" you need only select this option and then click on the number of

the box you wish to erase. If the box is not defined anyway, a message will indicate that you have really done nothing by trying to erase a box definition which does not exist. If the box is defined, the program displays the box on the screen so you can see what you are erasing. Hopefully, you will have viewed the box before deciding to delete it. When the program displays the box to be removed, click the mouse and the definition will be erased.

Editing a box file In some ways, this is one of the most powerful features of the box files. Supposing

you run a test and you view the results based on some box file definitions. What if you find that a lot of the lookpoints are in a certain area, an area which you did not define in the box file? By reloading the box file and editing it, you can redefine the areas which you are interested in. If you have only defined about ten of the possible fifteen boxes, you could use the remaining five slots to define the areas which you now find interesting. Also, you could remove some box definitions, and make them larger or smaller, depending on where your interests lie.

Calling up a box file for editing is similar to the save option. Selecting the "Edit a box file" option, type in the name of the file you wish to edit. If it exists, you will be able to work with the definitions it contains.

5,204,703 35 36

Creating a batch test file From the Main Menu, select the third option: "Perform saccadic motion test." The

Tes, Menu should appear with the following options:

Define Important areas of image (Use pictures to create box files) Create a batch test (Make test with pictorial and textual stimuli) Edit batch file (Change names of results files for a batch test) Test configure (Change testing parameters) Perform a single test (Create and run a test for one stimulus Screen) Execute a picture test Batch file (Run a batch test) Quit (Return to the Main menu)

The first three options listed above allow you to choose the pictures and text you wish to display in a test and change the name of the files which the results are written to.

Perform a single test This option allows you to test saccadic motion for a single stimulus screen. It

performs a test with text, one enlarged picture, or two reduced-size pictures. If you select this option, the program will request a list of file names to be used in the test. The program will also ask for a file name to store the results in and for a box file to process the results with. You must enter a results file name, the box file name is optional. When the user is ready he/she or the experimenter can press a key to begin the test. The program will perform a calibration and display the selected image(s) or text screen. The program will grab and store lookpoints until either the desired number of lookpoints have been taken or a key is pressed.

Create a batch test

This is the standard option to use in creating a test which includes more than one stimulus screen. You can create a batch file to execute later as many times as you wish. The user is presented with a screen to enter a list of files to be displayed, enter the name of the results file, and enter the name of the corresponding box file (for analysis). After each screen is filled in, a new one will be presented. As many tests as desired can be included into a batch file. To end the batch file just leave the filename at the first prompt blank and press C.

Edit batch file The last option mentioned that you could create a batch file to be executed as many

times as you wished. But recalling that when the batch file was created, the results files had to be specified, every time the batch test is run, the results from the previous running of the test will be erased and written over. This becomes a real problem when many subjects are to be tested on the same set of stimuli using the same batch test. One way to avoid this hassle is to recreate the batch test for each subject and give different names for the results filed for

5,204,703 37 38

that subject. Another way is to exit the program, enter the c\saccade\results directory and rename the old results files before they are written over. Both of these methods are clearly tedious and time consuming.

This option saves you the trouble of trying to rename your batch files by doing it for you. When you select this option, you will be prompted to select a batch file. Once you make a selection, you can input a 5-character seed which will be used in modifying the results files used by the batch file. The new results files are constructed as follows: seedO00.res, seed001.res, seed002.res, etc. up to the number of screens in the batch test. By using this option, you can input the subject's name, number, or other relevant data as the 5-character seed so that later you will be able to distinguish the results files for different test cases.

Part 6: Running Tests

Registering the subject From the Main Menu, select the fifth option: "Register a subject." This option simply

allows you to enter information about the subject: their name, number, and any other information of interest. The information of interest cannot take up more than 80 characters. Although it is not necessary to register a subject, it may come in useful, when analyzing results, to know whose eye behavior data you are evaluating. The analyze results section of the software will often display information about the subject if it is available in the results file. By registering information about the subject to be tested before the test, this information will be included in the results files created by running a test on the subject.

Configuring the test environment This option is particularly important in terms of making a test compatible with the

subject. If, for example, you have found from previous tests that a particular calibration does not work well for a subject or that you would like to gather more or less data points in your tests, this is the option to use to make such alterations. Also, when you quit the program, any changes you have made in terms of configuring the test environment will be stored in the file CONFIG.CAL and reused when the program is run again.

From the Test Menu, when you select the "Test configure" option you will see a screen with information that looks quite puzzling. The screen will have the following configure options in the left column followed by their value in parentheses:

Number of Calibration points Choices are 4, 5, and 8, the default is 4. The more points used in the calibration before the test, the more accurately the software can estimate lookpoints. The tradeoff is that it is more difficult to calibrate with more points.

Number of Filter points The upper limit is 5, the default value is 1. When gathering lookpoints, the software gets as many lookpoints as there are filter points, and then

5,204,703 39 40

averages the lookpoints to come up with one point. The more filter points used, the smoother the data. The tradeoff is that the software gathers data slower as the number of filter points increases.

Active box lookpoint feedback This can be either On or Off, the default is On. The idea behind this is that during a test, the software has the ability to place a small circle at the point where it thinks the subject is looking. This is an excellent means of seeing how accurate the program is in determining lookpoints. In cases where you do think the user may be distracted by the feedback cursor, you should have this option turned Off.

Recall that when the test is created, after a picture is chosen, the user is asked whether the image is active or not. This active box idea is actually a remnant of an older version of the program in which if a subject looks at an active image a certain number of times, the program goes on to the next stimulus screen. At this point, such processing can be done on any part of the image, and so the active/inactive boxes are really unnecessary. Still, the idea behind the active box feedback is that you can choose if the feedback cursor appears when the subject is looking at the image. Similarly, the inactive box lookpoint feedback, when turned On, allows you to see where the subject is looking when they are looking at an inactive portion of the screen (either an inactive image or the screen's background).

Inactive box lookpoint feedback This can be either On or Off, the default is On. See the "Active box lookpoint feedback" option for more explanation.

Active boxes status This can be either On or Off as well, the default is Off. If it is On, then if the software collects X lookpoints within a certain box (X is defined by the option "Number of lookpoints to select box"). This option could be useful when you are experimenting with interactive images -- the screen could change from one stimulus Screen to the next if the subject looks at a specific region a certain number of times.

Number of lookpoints to select box The range is from 1 to 32000; this option is useless if "Active boxes status" is Off. See active boxes status for further explanation. Also, note that if the number of lookpoints required to select a box is greater than the Total number of lookpoints allowed per stimulus screen, then this option will be ineffective.

Total number of lookpoints allowed The range is from 1 to 32000. The default is at 250 lookpoints taken per stimulus screen. If the software is sampling well (about 30 samples per second), then each screen will be displayed for about 8 seconds. 900 lookpoints take about 30 seconds to gather. Also, note that the software takes slightly more time to gather lookpoints if the feedback cursor's are On.

Number of screens between calibration. The range is between 0 and 4, the defaults is 0. This indicates how many stimulus screens will be presented before the program perform a

5,204,703 41 42

another calibration on the subject's eye. By introducing a calibration after a specified number of Screens, the experiment can better estimate the subject's lookpoints if the subject has moved, and it can provide a change of pace from the monotony of staring at stimulus screens. An animated calibration that runs every few screens may be quite useful, in fact, for testing infants. When this option is set to.0, only one calibration is performed -- at the beginning of the test.

Performing a test As described earlier the "Perform a single test" option allows you to create and run a

test in one step -- except the test must be for only one screen. The option for longer tests is "Execute a picture test batch file." This name may be misleading in that implies only pictures can be used in the test. In fact, the option actually allows you to select a previously created batch file, and as you may recall, a batch file can contain both pictures and text as stimuli.

The testing step is very simple. When the option is selected, a batch file must be Selected to be run. You can press ESCAPE if you decide not to make a selection. If you do choose a test to run, the screen will go black and a message will appear in the center: "Press any key to begin test." Get the subject situated comfortably in front of the camera so that his/her eye appears focused on the video monitor. You may also want to adjust the F-stop on the camera in order to enhance the contrasts between their bright eye, glint, and face. Press the key to begin, or let them press it.

The program will first perform a calibration with the number of points you chose in the Test Configure menu. If the calibration is animated, a turtle will travel from one calibration point to the next (perhaps with noise), and a smiley face will flash at each point. If the calibration is normal, a bullseye will appear at each calibration point. The subject should stare at each calibration point as it is identified by the smiley face or the bullseye. It is also helpful if the subject does not move around or blink during calibration. If the calibration is bad, the program will indicate this and try again until it works. If the program continues to get only bad calibrations, you must stop the program by pressing CTRL-C twice. If the calibration is good, the program will continue with the test Stimuli.

As the test is running, if a test stalls on one screen for too long, it may be that the software is having trouble finding the subject's eye. Pressing a key will terminate the stimulus screen and allow the software go on to the next stimulus screen.

Part 7: Analyzing Results

In order to analyze the results of a test, you can either use the "Analyze results" option in the saccade.exe program or use the sanalyze.exe program. The first choice allows you to look at the results in graphical and textual form in several different ways. The second option, only providing textual summaries of the results, writes the results to a DOS file which can be analyzed later using other post-processing programs.

5,204,703 43 44

It is easier to understand the use and limits of the sanalyze.exe program once you understand the rest of the saccade.exe program.

Analysis using saccade.exe From the Main Menu, select the fourth option: "Analyze results." The Analysis Menu should appear with the following options:

Set File Specification (Use * and 2 to load results files) Get New results file for analysis (Load in a new results file) List results on screen (Lists each data point in current results file) Superimpose lookpoints over images (Change names of results files for a batch test) Graphical summary of results (Graphs x and y lookpoints, and horizontal and

vertical pupil diameters for each data point in current results file)

Information Summary of results (Summarizes data in current results file) Analyze results using Box definitions (Use box files to summarize results information

over user-defined areas of interest) Quit (Return to the Main menu) a

Loading a results file In order to view the results of the tests, it is first necessary to load the results into

memory. If you choose the option, "Get New results file for analysis," the program checks for results files corresponding to the pathname saccade\results\"...res. If you want to load results files which have extensions other than "...res' or which are located in other directories, you must specify the corresponding pathname using the option entitled "Set File Specification."

99

For example, suppose you saved results files in the saccade\results directory with the same file name but different extensions (e.g. result.1 result.2 result.3). You would set the file specification as "result.". Suppose these files were located in another directory called my results\part1, then you would ues the file specification "\myresults\partlvesult.".

Once you have set the appropriate file specification and chosen the option "Get New results file for analysis," the program presents, in alphabetical order, a list of the results files corresponding to the pathname you have chosen. You can select only one file from the list provided. Use the arrow keys to move the highlight bar over the desired results file, and then press return to load that file.

Simple options List results on screen -

This option is very basic. It lists the raw data on the screen ten data points at a time until it reaches the end of the file. You must press a key in between each set of ten data points in order to see the subsequent set of points. Each listed data point contains 4 pieces of information: the x-coordinate and y-coordinate of the subject's eyegaze, the horizontal pupil diameter, and the time at which the data was taken.

5,204,703 45 46

The X- and y-coordinates are based on the resolution of the screen (specified by a function called set-gaze_res). This means that if the resolution screen was specified as an 80 by 25 (80 horizontally by 25 vertically), then if the person were staring directly at the center of the computer screen, the x- and y-coordinates would be 40 and 13 (12.5 rounded up), respectively. The pupil diameter is in terms of pixels on the framegrabber. These measures have not totally accounted for changes in diameter resulting from head motion. Therefore, when you compare diameters from data point to data point, you should consider only the relative change in the measure and not the absolute change. Finally, the time stamp is given in milliseconds. The time stamp starts from zero at the beginning of each stimulus screen.

Superimpose lookpoints over image This option places on the screen the image which was presented to the subject when

the data was recorded. For each data point, a tiny circle is placed over the image at a point corresponding to the x- and y-coordinates of the subject's lookpoint. In the bottom left corner of the screen is a large circle whose diameter is the same as the horizontal pupil diameter for that data point. The diameter of the circle is also written inside the circle. The succeeding lookpoints will be shown either automatically ("Continuous lookpoint display") or through subsequent keystrokes ("Wait for key between lookpoints"), depending on which option you choose immediately after you select "Superimpose lookpoints over images." Anytime while the lookpoints are being displayed, you can type 'Q' to stop the display and return to the Analysis Menu.

NOTE: If a picture does not appear on the screen where it should be, the program was unable to find it in the directory in which it should be located. If you wish to see the picture, you must place it in that directory. It is likely that this directory is the \saccade\pictures directory.

Graphical summary of results This option is a small step above the previous two. It provides for a complete display

of the test results. When the option "Graphical summary of results" is selected, a graphics screen appears. The bottom left corner identifies the subject who was tested and describes the image that was on the screen during testing. If there were two pictures on the screen, for example, then the first picture listed is the name of the picture on the left and the second is the name of the picture on the right.

The other three boxes on the screen contain graphs. If there were 250 data points gathered during the test, then the horizontal axis of each graph should begin with 1 and end with 250. (The horizontal axis will later be change to a time axis, since the time interval between sequential data points is not constant). As indicated by the titles of the graphs, the graphs show the complete set of x-coordinates and y-coordinates for each lookpoint, and they show both the horizontal and vertical pupil diameters for each data point.

5,204,703 47 48

An ing lookpoint

Af The labels of the vertical axis of the x-coordinate graph are based on the horizontal resolution of the graphics or text screen used during the test. For example, since an EGA high-resolution graphics screen has 640 horizontal pixels for the graphics screen, the vertical axis for the x-coordinates ranges from 1 to 640. Similarly, the labels of the vertical axis of the y-coordinate graph are based on the vertical resolution of the stimulus screen. On each graph you can see how the subject's horizontal and vertical gaze changed as the total number of data points (e.g. 250) were taken. For instance, if the graph on the x-coordinate always ranges in the upper half of the screen, the behavior indicates that the individual preferred to look at the right side of the image screen.

nalyzing di r link

The graph in the bottom right corner of the screen shows the change in horizontal and vertical pupil diameters as the subject looks at the stimulus. Notice that in the beginning there is a sharp drop in both diameters because the subject's eye initially constricts in reaction to the brightness of the stimulus. Also, eye blinks can be identified by a near vertical drop in the vertical pupil diameter.

Information summary of results When you select this option, the first screen which appears displays information about

the subject and about the stimulus screen. This information includes the subject's name and number as well as other relevant information which was entered before the subject took the test. It also includes the name of the text file which was used or the names of the pictures which were displayed during the test. Below this information is a brief statistical summary of the entire test. It contains the following information:

1) Total number of lookpoints, in 2) Mean horizontal pupil diameter, pu 3) Sum of all horizontal pupil diameters 4) Sum of all squared horizontal pupil diameters 5) Standard deviation of horizontal pupil diameters, s The most important of these measures are the first two and the last statistical

measure. The first is significant in that it tells you how many lookpoints were gathered in all. The second measure provides a basis for comparison with other stimulus screens. The average pupillary response to the stimulus screen can be compared with the level of pupillary arolysal elicited by other stimulus screens. This measure is not significant in itself, but is instead, a basis for comparison. Similarly, the standard deviation provides an idea of how much the pupillary response varied for that stimulus screen. This may provide a basis for determining which stimuli cause more pupillary activity to occur.

Sometimes you may be interested in comparing different images within one Screen. For example, you may wish to see if somebody prefers to look at one picture more than

5,204,703 49 50

another or if one image cause more pupillary activity than does the other image on that same screen. For this reason, this option also provides a summary of each images in the Screen. Pressing any key, you will see the screen change to reveal the same statistical summary information as on the previous screen. If the value of n on the previous screen was 250, meaning that a total of 250 data points were taken, you will see that these 250 points have been divided up among the different images which were on the screen. For instance, there may be 125 lookpoints for the first picture, 100 for the second picture, and 25 lookpoints outside both of the pictures. This would indicate that the subject looked at the picture on the left more often than the picture on the right and looked overall at the pictures more often than not. Similar comparisons could be made with the average pupil diameter measures and with the standard deviations. If you press a key again, you will be returned to the Analysis menu SCree.

Analyze results using box definitions Using the same line of reasoning as with the information summary, the analysis may

be concerned with even more specific parts of a stimulus screen than the images as a whole. For example, you may wish to see how long the subject looks at the caption on a Far Side cartoon or how often he looks at the eyes in the picture of a face. This is where the box files come in useful. Selecting the option "Analyze results using box definitions' you will be presented with a small box definition menu.

On the bottom right corner of the screen, you will see the name of the box file which you chose to analyze the stimulus screen with. If there is no file name in that corner, you must have pressed the Escape key when asked to choose a box file for the screen (This is when you were creating the batch test to run on subjects). If there is no box file associated with your results file, you can always use the option "Load new box file" to load a file to use in the analysis.

There are two advantages to the "Load new box file" option. First, as mentioned above, suppose you did not choose a box file to associate your results with. Suppose that now you do want a more detailed analysis of your results. You wish to look into specific parts of the images used in the stimulus, you can load in an existing results file So that you can now perform such analyses. Secondly, suppose you have already associated a box file with the stimulus screen and that the box file contains the maximum of fifteen box definitions. If during your analysis, you find that there is still another part of the stimulus Screen which you would like to analyze, there is no more room in the box file for another definition. You can place this new definition (and up to 14 more) in a new box file. You could then use the "Load new box file" option to load in the new box file and continue your analysis of the results.

In summary, the "Load new box file" option lets you analyze any rectangular section of the stimulus screen used for your results. If the appropriate Sections are not already defined in any of the existing files, you can go back to the option "Define important areas of image" in the Test Menu. There you can add the definition to an existing file or enter it in a

5,204,703 51 52

new file, return to the "Analyze results using box definitions" option in the Analysis menu, and load up the appropriate results file.

Other than loading in new box files, this menu allows you to view summaries of the results in either text or graphics format. These summaries are the same as those in the "Information summary of results" option in that they describe the number of lookpoints, average pupil diameter, etc. They are different in that each summary describes one of the 15 boxes defined in the box file. Thus, if you choose the text summary option, you will see data summary information for each box defined in the box file. At the bottom of the screen, you should also see a summary for data points "Outside all boxes." These are data points for which the x-y lookpoint was not within the boundaries of any of the boxes defined in the current box file.

Although the advantage of the text summary is that the summaries of all boxes can be seen simultaneously, the problem is that the summary does not show you the box you have defined. This, in fact, is the advantage of the graphics summary. By choosing this option, the stimulus screen will appear as it did during the actual test; however, the upper third of the screen is reserved for the box summary information. A white strip broken into 15 numbered segments appears about one third of the way down from the top of the screen. By clicking the mouse on one of the segments or typing the number of the segment (and pressing return), you can view the summary of the box associated with that number. If the box number you choose is undefined, you will hear a beep and see a message which indicates as such. If the box is defined, the summary information will appear on the top part of the screen and the box will be displayed on the stimulus screen as it was defined.

Analysis using sanalyze.exe Now that we have seen the various analyses allowed by saccade.exe, it is easier to

recognize the need for sanalyze.exe. The program sanalyze.exe is useful for two reasons: 1) Obtaining a printout of the results 2) Performing your own analyses of the results The essence of the sanalyze program is that it converts results from their compressed

form in the results file either to a printer readout, to a DOS file, or to a LOTUS spreadsheet file. A file in DOS format can be viewed using a DOS text editor, using the "type <filename>" command at the DOS prompt, or using the "list <filename>" command (if provided) at the DOS prompt. More importantly, the DOS file itself can be printed directly to a printer, called into a word processor for editing, or read in by programs to do additional analyses which the user has in mind. Similarly, the LOTUS spreadsheet file can be imported into LOTUS 1-2-3 or compatible spreadsheet programs in order to gain access to the powerful data manipulation and analysis capabilities of the spreadsheet.

To run the sanalyze.exe program, type "sanalyze' at the DOS prompt. You should then be presented with the following menu: Set File Specification Set the pathname for selecting results files

5,204,703 53 54

Select File(s) Load up a single results file or set of results files Export File(s) for LOTUS Write loaded results to a file for use in Lotus 1-2-3 Generate Report Information summary of results -- sent to printer or DOS file Generate Report by Stimuli Summarize results by picture Set Report Destination Destination can be the printer or a DOS file Configure System Set margins for printer Quit Leave the program

To the right of this menu will be information summarizing the results of the file. This will help you see what activity is going on in the program: which file is currently being loaded or processed, how much memory the program has remaining, etc. When the program generates a report, the report will list all of the lookpoint data, summarize the data for each picture, and summarize the data for the box file associated with the results.

Some of the important facts to remember about this program are 1) You are not limited to loading in one results file at a time

When selecting files, you can press "t' beside all the files you are interested in loading up. This will tag the files you want. Once you press return, only the files which have been tagged (as can be seen by the marker beside the file name) will be loaded up. To untag a file, press "t" again when the selection bar is highlighting the filename. The use of tagged files is important in that over a set of results files, some pictures may be used more than once. The report, if generated using "Generate report," will generate a separate summary for each picture in each of the loaded results files. But the "Generate report by stimuli' option will combine such summaries when the pictures from any of the results file are the same. The results file will appear in the directory from which you ran the sanalyze.exe program.

2) When entering the name of the output file, you may enter the name of an existing file If the file specified as the output file already exists, menu will appear indicating as

such. You can go back and reenter the filename, write over the existing file, or append the new summary report to the existing file.

5,204,703 55

saccade2.prj c: Vuaryland\fastgaze c: Vddhv fast Vfast find c: \cgazeVaat funct c: Vddhv fastvnew fg c: Varicalib\tools c: Varicalib\grtools c: VericalibWmenu c: Verical bVaous tool c: Vericalib\timex c; Vaarylandvinagebox c: Waaryland\sacboxes C: Vaaryland V8acinpro c: Vuaryland\sacio c: Vnaryland Vacre Sul. c: VturbocVEGAVGA. obi c: Vaarylandvbox de f\032

sacimpro.c include Catdo.h> include Cs tdlib.h> include galloc.h> include co, hid include gunath.h> include CConio.h> include Chahd include "c: Vc.gazev fgroutins.h" include "c: Wericalib\tools.h"

typedef unsigned char rowbuf (sl2:

typedef double rhi starray (256):

AUTHOR: Kevin S. Spetz t dAT: 9/

a routine to anooth the image in the current video lage of the ran grabber. The laaga is snoothed with a sple averaging filter

af void soothiauge (void)

int. lx, ly unsigned char ave: window type but unsigned char bufi, buf2;

to on (unsigned char ) alloc (sl2 sizeof (unsigned char ) buf2 as (unsigned char aalloc (sl2 sizeof (unsigned char); if (bufl as NULL) buf2 as NULL)) {

fprintf tatterr, "Error Memory Allocation -- soothinage") ; exit (l):

buf - open window (15, 10, 65, ), WHITE, CYAN, 1,"Status: "): xycprint ft 6, 2, HITE, "Saoothing Image...); for y - 0; ly < 40; ly++)

percent bar (s, S. 40, ly, 40; get rowavw (ly, O, Si2, bufl: get rowavy (lytl, 0,512, buf2): or (li as O. lx c. 5; lx++)

ava - bul - lix) + a bufl + lix + 1) + (buf2 t x + buf2

Af

+ lix t lid: buf lb a bufl x + 1) m a buf2 + x) as bu2 t lix + 1) an ave;

putrowav (ly, 0.52, bufl: putrowavy lyti, 0,512, buf2):

parcent bar (5, 5,40,1,1); delay (300); close window (buf): free bul): free buf2);

/ AUTHORS: Kevin Spatz, Greg Schubert a da

Routine to reduce inage to l? its original size ef

void reducaillage (vold) int Ex, ry, lx, ly, Ki float ave: unsigned char buf, buf2, abuf: window type buf:

buf an open window ( 15, 10, 65, 10, WHITE, CYAN, 1,"Status: "): xycprintf(l7, 2, YELLOW, Reducing Inage...} : bufi - (unsigned char *) malloc (sl2 sizeof (unsigned char) }; buf2 (unsigned char nalloc (512 sizeof (unsigned char) ; a buf - (unsigned char ) malloc (256 sizeof (unsigned char}: new set abu, O 256);

56

5,204,703 57 58

sacimprC.C. rx as ry - 0; percent bar (5, 5, 40, 0, 1) : for tly - 0: y C (40 - 2); ly t 2) {

get rowav Wily. O, Sl2, but ) : get rowavw tytl, 0.52, buf2):

percentbar (S,S 40, ry, 240); for (ix 0; lx C (52 - 2); lx tar 2)

aw a 0. for x O; x & 2; x++)

ave + bul + x + x): ave a buf2 + x + x);

ave awe f 4.0; (abuft rx) - (unsigned char) ave;

rx++:

putrowavw (ry, 0,255, abu f : rytt rx - 0:

percentbar (5,540, l, l); delay (300); free buf); free buf2): free (a buf); close window (buf);

/ AUTHOR: Kevin s. spatz a dart f

a Routine to generate the cumulative distribution function of an image. The inage histogra is passed in his tarry. The CDF is returned in rhi starray. t holds the number of pixels in the image

Af static void CDF this tarray i, rhi starray c, double t

int index; double cus as O.O; for (index as 0; index c 256; indext +

cun - cindex) s (double) index)/t + cun;

/ AUTHOR: Kevin S. Spetz DAT; 7/BB

Routine to generate a flat power density function from a cumulative distribution function.

af static void flatpdf (rhi starray c, rhi starray a)

int index; for index n 0; index C 2S6; index++)

index or floor cindex 255.0);

/ AUTHOR; Kevin S. Spetz a Te AB

Routine to equalize the inage in the current frame-grabber video window. 2qualization basically results in contrast enhancement.

a/ void equalize image (void)

histarray inage in: rhi starray image cdf, image map: rowbuf buffer; int x,y; window type buf:

buf is open window (15, 10, 65, I, WHITE, CYAN, ), "status:"); xycprintf 13, 2, HITE, "Calculating histogram..."); nakehistograa (in age in, l, ); acycprintf 13, 2, YELLO, "Equalizing histogram. . . "): percontbar (5, 5, 40, 0): CDF (in again, inagacdf, $12.0480.0); flatPDF (image cd f, image. Jaap); for y - 0: y C 480; y ++

percent bar (5, 5, 40, y, 480); getrowavw ty, O, 512, buffer): for x in O: x C 52: x++)

buffer (x} - (unsigned char) image map? (unsigned) buffer (x): putrowavy (y, O, 512, buffer) :

percenthar (5, 5, 40, ll) ; delay (300); close window buf) :

wn Author: Kevin S. Spetz dAT.: /S/89

Routine to reverae the intensities in an image. Basically take an image and turns it into a negative

tf

5,204,703 59

sacimpro.c void reverse image (void)

rowbu buffer; int x, y; Window type bug:

bu open window (15, 10, 65, l, WHITE, CYAN, 1, status; n : KYCprintf(6, 2, WHITE, "Revering Image...) for y - 0. y C 4 BO: y +

percentbar (5, 5, 40, y, 480): get rowavw ty, 0,512, buffer); for (x - 0: x & Sl2; x++)

buffarx a 255 - buffers PutroWavw ty, O, 512, buffer):

Percentbar (5, 5 (0,1, delay (300): close window buf);

/* AUTHOR: Kevin s. spatz DATE: Als/9

" outline to take an image and make it binary. All points below threshold :r turned to 0 and all points above are turned to 255.

void binary image (int threshold rowbuf buffer; int x, y; Window type buf;

bu open Window (i.5, lo, 65, la, HIT, CYAN, 1, status; xycprint ft 4, 2, WHITZ, "Making image binary...n) ; for y - 0: y < 4 BO: y4) {

percentbar (5.5, 40 y, 4 DO); getroWavw ty, O, 512, buffer) for x 0; x C S2 x++)

buffer r (buffer x C threshold 7 O ; 255; PutroWavw ty, O, 52, buffer):

percentbars, 5, 40, , ; delay (300); close window (buf);

WO32

Sacboxes.c include Cstco.h> include Cdos.h> include Calloc.h> include Cstdlib.h> includin Cconio.h> include Cqraphics.h> include (ctype.h> include Co.h> include Ctime.h> include c: Vericalib\tools.h" include c. Wricalib\nous tool.h" include c: VercabVtlinex.h include c : \cgazev gaze.h" include "c: \maryland \saccade. h" include "c: Vaarylandvaacboxes.h" g

static int req_lp 15: a Required number of lookpoints for a selection a? static int boxnark l; f sox routines will place a yellow icon if in box of static int noboxnark l; a box routina will place a blue icon if not in box of

fe AUTHOR: Kevin S. Spatz a DAT WAB 9 Routine that sets the variable for the raquired number of lookpoints needed to select a box. The lower the number the faster the sytan operates

inf void Det gaze speed (int sp)

relp ap;

void sat box mark (int on) if on boark on 1; eise boxnark a 0;

void set nobox mark (int on) f or nobonark

else noboxark or 0.

60

5,204,703 63

f a house arrow); f f a house on (); /

if tab as NULL) useboxes - 0: while get gazeposition ( x, y, shd, wid)

if kibhit atch (ch to upper getch ()

case SPACE : return 0; can R. : narknext as break case ESC return to : default : narknext is ch; break;

lf kibhit switch (ch a toupper getch () r

case SPAce : return 0; case REF : aarknext break cas 2SC return O); default : narknext ch: break

O

/ INSERT TIME state ING HERE, I THINK of data . . . ; data y y; data.hd as his datavid a vid; data time- tiex () - start time;

(arknext data park arknext arket in

else data.nark as 0; write than, data, sizeof (datatype);

- Texr data . x . ; data.y no 8.

frD nod bind six, O, S9); a f bind (x, 0, 639; bind ty, 0,389);

else binds, 09): bind (ty, 0.24):

if useboxes if prevbox lookupbox tabl, x, y) {

lf (boxaark) place p(x, y, YELLOW);

delay (20: ovel p(x,y, sh;

else if noboxark place lipt, Y, YELLO):

delay (20): Enovel p(x,y, si

elp if noboxnark) a place p(x,y, YELLOW); delay (20; renoval p(x, y, s) :

for index is 0; index C raq p - li index++) while gatgazeposition (6 x, y, Ghd, Svd)

lf kibhit) switch (ch toupper getch

case SPAce : return (C); case RE arket so break cape ESC return to: default : aarknext ch; break;

lf bhit switch ch toupper getch ()

case SPACE. : return to case Ref : aarknext as break case S : return to : default : darknext s ch: braic:

data . x ; data. Y so y; data.hd a hid; data.wd a vid; data. t. ine- timex () - start time:

Parknext data. Rark aarknext; arknext - d.

else data.aark C: write than, data, sizeof (datatype));

if (t totallip > maxlp - 1) return 0;

it screen sode () a text) { x a ; y is 14:

find nod blind six, 0, 649) : A bind six, O, 639: bind (sy, 0,349;

else bindis, 0,79); bind sy, G, 24);

5,204,703 67

matfunct.c include (stdio.h> include stdlib.h> include Caat.h> include c \cgazevatrixdie.h"

A AUTHOR: Kevin S. Spetz (Adapted from B J Haberkorn's code or Routine locates space for a matrix of size (rows, cols, and return a pointer to that atrix. The element of the atrix can be accessed through the acro ele

Of atrix newlatrix (int rows, lint cols)

atrix tap; double temp; unsigned lint ris, cle;

rts unsigned int) row: cis unsigned int) cois; if tap a new Matrix) a NULL)

fprintf(stderrheory Allocation error in newlatrix: retirl

nunrot (tap) a rows; tap->rowdimen a ris; nuco tap cols; tap-scoldlaan a cle: tamp double calloc (rwe (unsigned int) sizeof double a lf tep ULL)

fprint (stderr, Masory Allocation Error in nawatrix return ...)

for as O: C w; + teep - double calloc clis (unsigned int sizeof double): if tapi - NULL) {

fprint fatderranery Allocation Error in newatra ot

tap-daat a tamp; return tap):

f AUTHOR: Kevin S. Spetz (adapted from a J Haberkorn a code da; fs Routine to destroy, and free the nanory held by a natrix created by newetatrix. to use pass the pointer to the condeaned atrix

void freelatriatrix fa nt : assert (unsigned int) nuanows () ) ->rowdiner) ; assert (unsigned int nucolis (a) opedcolden; for e O; Carowdien; ++

fran asdaat); free ediat); free n:

fe Author: Kevin S. Spetz a de A Routine to print out the elements of a satrix pointed to by a

void printaatrix Matrix in int E, ci printf iatrix Vn); for r g unto a r++ printf f : for c e : c cols ra: ct printf 3.3 elen (ra, r, c):

printf(wn);

f AUTHOR: Kevin s. spetz s daTE: As

routine to clear all of the elements in a matrix, n. af

void clienaatrix (latrix a nt r, c. for r a 0: r g nunarows); r++

for c as 0; c g nucos (; c. ele, rc as 0.0;

int dupatrix (latrix are, Matrix a dist)

lint r, c. if dst see NULL) freeatrix dist); if (dat newetatrix nunros (src, nucols (rc) is NULL) return to : else

for c at O; c g nacos (src; c.4 + for is 0; r < nuarows (orc) ; r +)

elendst r, c e learc, r, c.

68

5,204,703 69

return (i);

as AUTHOR: Kevin S. Spetz dAT: 7/8 Routine which calls newhiatrix to create a matrix, then sets all the element to Eero

f atrix nawaeroatrix (int rows int cols)

int r, c. Matrix temp;

if ( (taap - newlatrix (rows, cols) - NULL) { for c 0; c < nuncols teap); c ++)

for r an 0; r c nurows (tamp, rt + elant teap, r, c) 0.0:

return tap):

/ AUTHOR: Kevin S. Spetz ?h DATE 7/

Routine that suns all of the points in vector n. A vector is considered to be a matrix of piza row, li), and is not a separate type from the atrix type. Zach element is raised to the power p before being added to the sun in this way you can create sun (elanp), etc

f double sunvect Matrix n, double p)

nt index; double temp; taup 0.0; for (index or 0: index c nunrows (a); index t t )

tamp n tempt pow (elen (n, index, 0),p): return tenp):

as AUTHOR: Kevin S. Spetz a dATE 7/ Routine that multiplies two vectors together, after raising each element of the vectors to pl and p2 respectivy. What is aeant by vector is described by suvect. These routines are used for solving a linear system of equations defined in natrices

tf double autvects (Matrix all double plMatrix x2, double p2)

int index: doubla temp: temp or 0.0; for index a 0; index g nuarows (pal); indext t)

teap - tamp + (pow talen (al, index, 0), pl.) pow (elera (m2, index, 0), P2) : return (tamp);

f AUTHOR; Kevin s. Spetz a DATE: 7/

Routine that switches rows t and b in the matrix a. A/

void switchrows (Matrix nint t int b) int col: double tep; if it - b)

for col 0; col (c nuncois (a): col++) teup - alien (n, t, col) ; elena, t, col) or eleran, b, col); alienta, b, col) a taup;

A AUTHOR: Kevin S. Spetz DAT; As routine to find a valid pivot row for a gaussian elimination solution to a system of linear equations.

unsigned findvalrow (Matrix fan, int x, Matrix fel) nt row;

for row n x; row C nuarows (a): row++) lf talenta, row, x as O. O.

switch row (n, row,x): switch rows (), row,x); return );

return (0):

f AUTHOR: Kevin S. Spetz

Routine which solves a systan of linear equations for a parameter wactor. The coefficient matrix la c, the parameter natrix (solution) and the solution vector is s. c. p .

of unsigned matrixaoln (Matrix sc, Katrix p, Matrix s)

int col, row, pivocol; double temp;

70

5,204,703 71 72

for pvcol in O: plvcol C. nucols (c) pivooltt if (findiva row (c., pivool, s)) {

for row - pivocol; row g nuanows (c) row t+)

if (row a pivocol) lan (, row, O) relan (, row, 0) fellen (c., row, plvcoli:

elas if (elanc, row, pivool)) alians, row, O - alents, pivool, O - (oleats, row, a felaa (c. row, pivotoli) ) i.

for tcol nuco (c) - l; col 2 pivocol - l; col--) if (row a pivool)

elenic, row, col) alien (c., row, col) /elen crow, pivool) ; else it (elanc, row, pivool))

telen (c., row, col) akanic, pivool, col) - (olenc row, Col deleta (C. rott, pivool)):

for trow a nunrows (p) - : row > -t; row-- teap 0.0: for col a numCo) (c) - l; col. X row - ; cols temp step t alun (p, colo) elec, row, col):

lf (elec, row, row)) elen (p, row, O - telen (, row, O - teap) alanc, row, row):

elp printf "Calibration Matrix Unolvable...wn"); return ( :

return (l):

W 032

newfg.c include Cstdio.h>

define use DOTHAV / O will make the systers use on AVs f fe will make the systen use both AVs f

define PI 3. 14592654

include Cdos.h> include Cen.h> include cath.h> include Calloc.h> include c : \cqazav fgroutins.h"

int avw.bank - 0: int avy vold)

{ return (av wbank);

int ww.void) if (USE BOTHAV)

return (1 - avw bank); else return (avy bank);

void initialize void) int :

/ set up the input LUT a f

outportb (FGCONTRol. PORT + 1, 0x00); for o 0; C 255; ++)

outport bt FG control. PORT + 2, 1) : o outport.b (FG control Port t 3, 1) :

out port.b (FG cold TRol. PORT t 1, 0x0). for 0; c. 255; + +

outport b (FG control Port + 2, i ; outportb{FG CONTROLPORT t 3, 1) :

outport bt FG corrol Port + 1, 0x02) : for a 0; i ca 255; ++)

outportbt FG coTRol. PORT + 2, ...); outportbt FG control Port + 3, 1) :

outportb(FG couTRoi Port + 1, 0x03 ): for as O. Co 255; it

outportb FG cormol Port + 2, 1) : outportbt FG CONTRol Port + 3, 1};

outportb (FG control Port + 2, 0x00); outportb (FG control. PORT + 3, 0x00);

fr Set offset voltage 1

outportb FG CONTROL PORT + 4, 0xF3):

outportbt FG control. PORT, 0x04); for an O: Cam O.

outportb (FG CONTROLPORT + 4, 0x0):

5,204,703 73

outportb FG CONTROL PORT, Ox): for 0; C- 90; itt outportb{FG control Port t 4, 0x0) :

/ enable the framegrabbar frame remory onto the PC bus /

outportb FG CONTROLPORT, Ox16);

/ choose the syster clock source to be the phase lock loop /

outport.b (FG control PORT t 5, 0x02); a set the horizontal and vertical offsets in frame ruernory to 0 if

outportb{FG control. Port 6, 0x00); outportb (FG control. PORT + 7, 0x00):

/ Enable the pixel buffer /

outportb FG control. PORT + 8, 0x40);

wn do not protect any bit planes during host access or a frarse 1 / memory clear operation

outportbt FG CONTROL PORT + 9, 0x00);

/ Do not protect any bit planes during image acquistion f

outportb (FG control. PORT t 10, 0x00): continuous ():

void continuous (void) short temp; teap - inportb (FGCONTROL PORT + 5):

outportb FG CONTROL PORT + 5. (temp 0xCO3) :

void grabfrane (void) short taup;

/* Turn acquisition off and at until done

Af tamp - inportbt FG CONTROPORT 5); outportbirc control. PORT + S, tamp & 0x3F) }; while ( (inportb FG CONTROL PORT t 5) & Oxc0) :- 0):

A start a frare grab af

tamp - inportb FG CONTROLPORT + 5): outportb (FG control. PORT t 5, ( temp 0x80) ); while ( (inportb (FG CONTRol. PORT + 5) Oxc0) - 0):

void and acquire (void)

short temp;

taap - inportb{FG CONTROLPORT t 5); outportb FG control Port t 5, (tomp & 0x3F) };

void waitinop (void)

while ( (inportb (FG control. PORT t 51 & 0xc0) - 0):

void find odd (void)

int odd short tanp;

/ wait until odd field occur "f

odd as O: While (odd a 0)

tamp - inportb FG CONTROLPORT + 5); if ( (tamp is 0x04) - 0) {

delay (2): temp - inportb (FG control. PORT t 5); if (tep 0x04) - 0x04) odd - l;

74

5,204,703 75

void pans witch (void)

short port: int find odd;

find odd - 0:

if use both Av) t lf (avy bank me 0)

aw bank : port 64:

else

aw bank - 0: port o;

fe aoth of the methods below work for the aquisition of frames in 33 J. f.

fe. This algoritha looks for two vertical blank periods. Because f f this is the casa, only le as may be used for the eye tracking / A algoritha. af

f while (inportb (FG control. PORT + 5 & 0x10) - 0x10): day (2: while ( (inportbt FG CONTROLPORT t 5 & Ox 0) - 0x10): outportbro Cotrol. PORT t 6, port): delay (2):

delay (2 : 1 aka sure that the previous frame's odd field is not recognized while (find odd or O)

if (inportb FG control. PORT t 5) 0x04) - 0x04)

find odd 1:

delay (2) while ( (inportbt FG CONTROL PORT 5) & Ox C) as 0x10): outportb_rG CONTROLPORT + 6, port):

a e s an in a a sea as a on is a a e s to a a as a a a a a it a a a t t e is a s a s in a a t t a ?t f a th/ f's This routine will wait for the present frangrabber operation f fR to couplete, switch the active video repory, and then grab a "f f a new frame of data. If USE both AV is 0, the active video af fe will not be switched. af A a af A as a as a a on e s a e is a a a n e s is a s a s a to a a na n is a a n is a a a b d e o O a a e s a a a?

void grabniswitch void) short terp: int find odd; short out val:

fe short port: A fa while tinportb FG control. PORT * S) { 0x04) - 0x04); "f

find odd - 0:

while (find odd -- 0)

temp - inportbt FG CONTROL PORT t 5) :

if teap 0x04) - 0) {

delay (2): tamp - inportb FG CONTROL PORT t 5) ; if teap Ox04) as 0x04) find odd - 1 :

out val - teap I 0x00: fe set acquid bit to a out val out Val & Ox OBF : f a sat acqado bit to a 0 if

outportb FG CONTROLPORT * 5, out-val) :

if (usz both Avi) { if avy bank -- 0) {

avw bank a l; fe port - 64; /

ale avwbank - 0: fa port - 0: 1

Af teep - inportb FG CONTROLPORT t 5): ' f

5,204,703 77

f start a grab operation f

outportib (FG coTROL PORT + 5, terp 0x80) ); af

int fgstatus (void { return ( (int) tinport b (FGCONTROLPORT + S. & Ox 10 ;

void tinewblank (void

while ( (inport (FG control Port + 5 & DxO - O

void switchavv (void) { if use of HAW)

if (avy bank - 0) { aww.bank a l; outportb FG CONTROLPORT 6, 64): outportbt FG CONTROLPORT + 7, 0):

ele ( avbank - 0: outport bt FG control. PORT + 6, oo): outportb (FG control PoRT + 7, 00);

alse outportbt FG control PoRT + 6, oxoo: outport.b (FGCONTROLPORT + 7, 0x00):

void setav Wint avv) awf was

if (avibank -- 0) { aww.bank - : outportb FG CONTROLPORT 6, 64: outport.b (FGCONTROLPORT 7, 0,

else avw bank - Oi outportb FG CONTROLPORT + 6, 00); outportb FG CONTROL PORT + 7, 00);

else outport.b (FGCONTROL PORT t 6, 0x00); outportb FG CONTROL PORT t 7, 0x00);

unsigned char inpic (unsigned it, unsigned y lint d) f ( x > Sl2) y > 40)

return to : outportb FG CONTROL PORT, 0x16 + ty/2) + 4) 32) ); return peakib (FG IMAGE DASE, y 51.2 t x):

void outplc (int x, inty, unsigned char vint w { outportb FG control. PORT, Ox6 ty/l2O) + w 4) 32);

pokab (FG IMAGE AASE, y 51.2 t x, v) ;

void get rowvw (unsigned row, unsigned start, sizet count unsigned char buf outportb FG control Port, Ox 6 + ( ( (row/20) + ivy () 4) 32}) : movedata (FG IAGE DASE, row 5.2 t start, DS, (unsigned) but, count):

void putrowivy (unaigned row, unsigned start, sizet count, unsigned char buf) outportbt FG control. PORT, Ox16 + (rowfla + iv ( t ) 32); movedata (DS, (unsigned) buf, FGIMAGE ASE, row 512 t start, count):

void get rowav (unsigned row, unsigned start, size it count, unligned char bu) out port bfc control PORT, Ox 16 ( (row/128) avw () 4) 32}) : aoved at a FG iMAGE ASE, row 5.2 t start, DS, (unsigned) buf, count);

void put rowavy unsigned row, unsigned start, size t count, unligned char buf) outportb (FG control. Port, Ox6 + ( ( row/12) t avy () : 4) 32)); moved a tatDs, unsigned) buf, FGIMAGERASE, row 512 t start, count):

void nakehistogranthi starray hint inc, inty inc) int x,y unsigned char bufi buf is unsigned char maloc (5.2 sizeof (unsigned char) : for x as O: c. 256; it

hit of for y - 0: y C 4.0; y + y inc) {

outportb FG CONTROLPORT, Oxl G t ( (y A 12 + aww A 32); noved at a FG IAGE BASE, y 512, DS, unsigned) buf, 512); for x as O: ; c. 52; x + x inc)

hint buft x) ++:

free bus :

78

79

void initializa again ()

int find odd-0; short temp;

initialize ();

5,204,703

While (find odd or 0) f wait until odd field occurs of

tarap - inport b (FG control PORT + 5): if teap & Ox 04) - 0

delay (2): temp - inportbt FG control PoRT 5);

ift (tenp is 0x04) - 0x04) find odd - 1:

continuous (); a perform a grab /

vold newgrabniswitch

delay (l); pan switch (); W032

fa stfind.c a a no so a on as a na n as a as on a on a as a on to a at a so a na no on as on an an at a a set on earn a?

This file contains the routine that will identify the center of the W pupil and the glint by THRESHHOLDING. The threshhold for the glint /

fe fe f a fa f a A f a fe fe f a f a

de de d de de de de de de de de

n in in in ir n n in

r r

is established by coaputing a histogra over the entire image and R/ lacting the highest values. The threshhold for the pupil is p/

established by first locating the glint, and then computing the why histogram over a 50x50 box centered at the glint. The histogram f should then appear a two peaks and a valley, with the valley" a/ representing the threshhold value. a/

e/ Authors; David Handel snan and Kevin Spatz A/ dat /22/9 Af

fina PI 3.141592653 in Fores 20

file corole 26 fine NcNTS22 fine Adredfailures 20 fine NGLINTS2 fire NUISS 2 fine x_inage start 0 fine x-inage stop 470 fine fine

ciude gae.h> clude Cstdio.h> clude Cotype.h> clude Cocono.h> clude (cht.h> clude Catring.h> clude Clo.h> clude catat.h> clude Cdo.h> clude cath. X

include cloc.h>

th

clude c : \cgazev fgroutins.hi

cloal. STATIC VARIAbles A typodef struct

double x: double y;

yooords;

float p; float py; float gxi

o int int int int int unsigned char frame buffer (512);

/s an an as as a on as a a on an as a on a a e global debugging variable

int int

at 9-y: g thresh: p thra ah-255; wpupildlan; hpupildian: previous eye-O:

noglint global: no pupil global:

start

/h Wh f / a fo WA f a fa We wn

e an an as a r a a to r n e a n e as a n e o as a na n an an as a na nate a na ea an an an a nea a?

Resolution used to scan image for pupil Minimum eye width f Hinilaua glint size /

of failures before recalibrating f hiniuun glint size / Allowed of skipped pixels in chord / swallet usable x frame coord f largest usable x frane coord / analle at usable y frana coord A largest usable y frana coord /

pupil center x a/ a pupil conter y A/ / glint center x af a glint canter y a/ a glint edge threshhold tf a pupil threshhold Af A vertical pupil dianatar f horizontal pupil diameter / 7 lapervious info available ef 7 row of frau buffer a/

80

5,204,703 81

int global failed; int global found: int no glintlocal : int no pupil local: int local fail ad; int local found: Ant no start: int bad a spect : f a as a an in a a s a n n n n e s a p is a in a 1 a global debugging variables end

a a n is a s a s p n n a s an a n is a a a a as a n n f

a routine returns ratio of two numbers passed in parameters. 0 - 1 / double aspect tint x, inty)

if (x > y) return double) y fix);

else return ( (double) x/y):

f in Sets a threshold for glint detection. Returns success or failure.

n/ unsigned set thresholds void

int gp: hist array localh;

nakehistogram (localh, 2, 2):

g_thresh - 256; while ( (gp4-local hi-g threshl) < 5); g thresh-r:

if g_thresh >100) return (1) : else raturn (O);

wn Following four routines: Look in a given direction until NUMMliss point " in a row are below the threshold t, used by to find the edges the pupil. a/

int lookbackx (int x, inty, int t)

int misses;

for tuisses 0; aise C NUMMISS; nails sett) while inpic --x, y, ww () ) > t.)

misaa as O: return x + NUMSS);

int look for WX (int x, inty, int t)

int as Jes:

for tases in C; asses c NUMMISS; ninett) While (inpic (+t x, y, livw () > t.)

aises as 0; return x - NUMSS):

int lookbacky (int x, inty, int t

int asses;

for nisses as O: nissas C. NUMISS; nisses++) while (inpic (x, -y, iv w () > t.

Risses or C: return y + NUMIss);

int look forwy (int x, inty, int t)

int asses;

for tisses or 0; aissas C. NUMMISS; asses++) whila tinplc (x,tty, ww { } ) > t )

passes as 0; return y - NUMMIss);

f a

deterraines the location of the new bright spot glint by searching a inited area around the old gl int.

Returns success or failure. aw

int local quick findglint (int old x, int oldly, int x, int ty) int ; int ; int tenp: int counter;

int int

ol whil

5,204,703 83

direction2: widths2s;

tra; dy;

counter g 30

get rowivw (, old-60, width, frame buffer): or a 0: C20); litt

teap-' (frame bufferti) : if t teapeg thresh)

it old-60; ye; return);

counterts; tecounter direction:

directionedirection -l) ;

raturn to

Uses previous information to find the glint quickly. Searchan out trom the previous glint to find the new glint. Returns success or failure.

af unsigned local findglint (void

int int rt int int int

tex ty:

y v chordond: v chordbeg:

x - (int): y - int) gy:

if (local quick findglint ( (int) ge, tint gy, six, y) }

return 0;

tx is look forw (x, y, g_thresh) + lookbackx (x,y, thresh) ) / 2.0; v chordbeg r lookbacky (tx, y, g_thresh); v chordend a look forwyt, y, g thresh) : ty a gy as tv chordand v chordbeg f 2.0; if abov chordend - Y chordbeg ) > MINGLINTSI2E)

g (look forwa (tx, ty, g_thresh) + lookbackx (tx, ty, g_thresh) ) f 2.0; return l;

else

return 0;

f Global routine to find the glint. Returns success or failure.

e unsigned global findglint void

int int int t

int int int

is

y - whil

tex; ty;

y: v chordend; v chordbeg:

kinage start: f initial x position f ynage start: A initial y position / e tyg y_nage stop)

f Finding Glint f get rowivity, Xiage start, x_iage stop-x image start-2, frate buffer: statisage start: while (x : x_image stop)

ir-x.inage start: if ( (frame bufferti) > g. thresh)

t look forwk (ty, g. thresh) t look backx (x, y, g_thresh ) / 2.0; v chordbeg - lookbacky tx, y, g. thresh): v chordond - look forwy tx, y, g thresh); ty a gy - vehordend t w chordbeg / 2.0; if (abs (v_chordand -

g-look for tx, turn

v chordbeg ) > HINGLINTSIZE)

ty, g_thresh) look backx (tx, ty, g-thresh)) /2.0;

84

5,204,703 85

+3;

y+-2:

return 0):

G.N. SARC ROUTINES EN f a at a n n n n as a n n s in a na a a a A in a na n n n n n an as an a a a a on e s is a as a saf

fa Identifies the pupil threshhold by first computing a local histograra centered around the glint, and then identifying the valley between two peaks as the pupil threshhold. Returns success or failure.

h/ int get pupil thresh (void) int nt : int hist (256; Snt has 6; int min-2s6; int Jax-0; int teap: int status; int x; inty:

x- (int) gx; y - tint gy; meabot (hist 0,512);

for (3-y-50; Cey+50; +-4) f make histogram /

introv wt), -son, trans-butter), for isdicardo; it and

ir { {xas (y-lo)) is C- y + 0 ) . . . . K. 60) ) is kidr (40) ) }

. skip over glint / else

temp- frame buffer ); hist templt t; if (tamp Cain) nin-tamp; if tamp >nax) max-teap:

for tiarain; ignax; it t) /f median filtar

hi-hist (i-1)+hilst it hist (it ) /3

statusso; /s identify threshhold "l for imax; i>nin: i--

switch status) case 0; if (hi)sh-ll) /s look for peak

statu or ;

break; case : if (hil Chi- ) ft look for valley /

p thresh-i ; status2;

break case 2; if thishi-)) we look for peak /

return (l):

break;

return (0);

p search the area near the glint for three consecutive pixel that are a above the pupil threshhold. These three pixels are used as an arbitrary a starting point INSIDE the pupil. Returns success of failure. th

int find pupil start (tnt x start, int Ystart int : int : rt doned; int width: int x: inty; int offset;

86

5,204,703 87

- (int): yr int) gy:

width: offsetawidth 2: cyto:

if (1->y image stop) myinage stop-5;

while t > y-offset) is done

introv wt), -ottist, vlach, trans butter): 3-0. while ic (OO) ) is done

ir offet-i () Coffsettlob is > y - ) is g(yi)

. skip over glint a 1.

if (frame buffer ti-l) >p thresh) is ( frare bufferti) PP thresh) 6 frare buffer titl) >p thresh) )

start+x-offset; y tarter; return i;

i++

--2:

return

fa Identifies the center of the pupil when no previous information is available. This routine calls the pupil-threshholding routine. Return a success or failure.

ef unsigned global pupilsearch (void)

nt t; int ty; int x; nt y;

int v chord end: int V chord beg: int. h.chord-beg; int. h.chordend: int x start: int. ystart;

if (!get pupil thresh () )

return

f ( ; find pupil start (&x start, systart))

eturn to

a start: y ystart:

while (l

if (inpic (x, y, ilvw () ) > p thresh)

tx (look forwk (x, y, p thresh) t look backx (x, y, p thresh ) / 2.0: v chordbeg - look backy (tx, y, p thresh); Yehordend - look forwy (tx, y, p thresh); ty py a tv chordand + v chordbeg ) / 2.0; if (vpupildian s (w chordend - v chordbeg ) ) > MINCORDLEN)

h chordbeg - lookbackx (tx - 20, ty, p thresh); hchord and - look forwa (tix t 20, ty, p thresh); hpupildia s h chordend - he hord beg:

pitchchordbeg thchordend) /2: if (a spect thrupildian, woupildia ra) ( 0.75)

retro

return);

else

f No action, try another point af

w

if ( xit-FINDREs) > x_inage stop) an unp to a new pixel

88

5,204,703 89

if y--FINDRes} < gy-loo)

return 0;

fa Identifies the center of the pupil if the threshhod is known. Return success or failure.

fif unsigned local pupils earch (void

int ty; int x: int y; int hehordend; int h.chord-beg; int v chordand; int v chordbag; lint xstart: int, ystart: int tempx;

if (! find pupil start (6 xstart, ystart))

no start++; return 0;

x is start: y yet art;

V. chordbeg - look backy tx, y - 10, p thresh; Y chor dend - look forwy (x, y + 10, p thrash); ty ply - w chord and t w chordbeg f 2.0;

h chordbeg - look backx (x - 10, ty, p thresh); h-chord and - look forwk (x + 10, ty, p thresh); hpupil dian h chordend - h chordbeg: Pix-tap-th chordbeg thchordend) /2: V-chordbeg - look backy (tempt, ty - 0, p thresh): v-chordend look forwy (top ,ty + 10, p thresh): ty - Py - (vichordend t w chordbeg f 2.0; VPupildian-v chordend-v chordbeg: lf (a spect (hpupidiara, wipupilldian) ( 0.75

bad a spect++: raturn to :

return i; w

fe as a At a t an a a on a a in a a as a a a an in a a a a a a a a a e s a as a f f Goa. ANd locA search corroLLING Routes a if wd a a it a a a a a a a a at a s is a a n is a n n is a a is a as a a as a a af

f a Identifies the glint and pupil centers fron within the entre inage. turns acco or ille.

tf int global search (xycoord "p, xycoordy g, int hdlater, int vdiameter)

static int glint is ses; if : global findglint () )

noglint global + +: global failedt t; g intrii best t; if (glint raises> 0)

g intesses ano;

return 0;

outpic (int) g, int) gy, (unsigned char) 0, iv () };

if (global pupil search () }

global found t +: p->xpx; p-dy-py; g->x-gx; g->yrgy; hdianetershpupildara; wdiameter-vpupildlaa: outpic (int) p. x, (int)py, (unsigned char 200 v (); return l;

els i.

global failed th: no pupil globalt t;

90

5,204,703 91

return 0;

f a Identifies the glint and pupil centers from within a portion of the inage, as determined by the previous glint and pupil centers. turns success or failure.

tf int local search xycoord p, xycoords g, int. hdlaeter int widiarieter

if local findglint ( )

local failed++: noglintlocat t; return

outpic tint gx tint gy, unsigned charo, iww) :

if (! (local pupilsearch () ) )

local failedt t; no pupillocalitt: return 0;

so

local found t +:

erashpupildlan; vdilantervpupildlara: outpic (int) pix, tint ply, unsigned char 200, lww () }; return );

unsigned get delta double dx, double dy, int hpupdia, int vpupdia, double rur

int count: nt : int noeye so: rt don a);

int found 0; int thpd -0; int twpd ano; int hpd 0; int vpd so: double x s); double ty 0: xycoords pupi: tycoords gint:

or count as count g in countt f get ruler lookpoints a

forcio if (! previous eye) A to previous Foro af

new grabna witch(); frcontinuous (; f.

lf (globalisearch (spupil, glint, 6thpd, twpd

previous eye-l; founcis

als At previous romaTo ef f TRY is local searches f

lso: core; while g5 is done

new grabnatch ( ; ?continuous (); /

if local search (pupil, glint, 6thpd, stypa)

co founds previous ayal:

els

+

f search coupleted - success or failure f

found fe aura f

previous eyeso: noeyett:

est

93 count--

count conte);

5,204,703

if (nonya X ALLOWEDFAILUREs)

return to

tx to glint. x - pupil. x: ty t- glint. y - pupil.y;

/ND nodhpd to hpdtthpd;

a/

dy -

vpd to wipdtvpd. hpd hpdtthpd; wpd as wipdit twpd;

hpupdia is hpdatint) nur: vpupdia a vpdf (int) nuar: if (abs (dx) > 75.0)

previous eye-O; return 0} :

return (l) ;

void print at at ) {} print f print f printf printf printf printf printf printf printf

ft success /

/s average raw lookpoint information

(abs (dy) > 75. O))

("wnDebugging informationwn"); "Global glint failed t"Global pupil failed Global failed

("Global found town); ("Local glint failed ("Local pupil failed "Local failed

: A4 d\n", naglint global) : : A4 d\n", no pupil global): 44 d\n", global failed;

: A4 d\n", global found);

: 14 d\n", no glintlocal): : 44 d\n", no pupil local) ; : A4 d\n", local failed);

94

printf printf

Local found own:

print f ("Bad a spect printif ("No start gotch();

V032

: 14 d\n", local found);

: 44 d\n", bad a spect): : 14 d\n", no start);

fe an an a on as a a a he et at n n is an a a s is a a s h m a ean to a na n e s a n n n is a a a he et at at a a Modul A.C. Author Kevin Spetz

last Revision l-1-9

coming soon : Edge Detection by Dave Handel saan

These routina is are responsible for generating and returning a lookpoint to the calling program. To use; include the file gaze.h and link this file with the program of interest

Interface: void initigate xryr, nunave, nuncial);

xir : Defines x resolution for returned lookpoints : define Y resolution for returned lookpoints

: Number of lookports kept in the running filter nuncial : Nuber of calibration points used

unsigned get gazeposition tint x int ty, int pupdia); x : Return x lookpoint

ty : Return Y lookpoint pupdia :

A.

th

A

th

Returns : returns pupil diameter

success or 0 is failure as a a to a a s at at a t t an in a At s m at A t e n n at a a s h an in a sat a a net e a that the a na n e s et a at as a?

define FG IMAGE has 2. OxD000 define FG control PORT 0x0300 define PI 3.4592653 define Ndo dine NDR2S 40 define coron 26 define KINGLINTS 22 define HS 45

define Vs SS define NSS 2 are Alloof AVRS 20 cene NOPUP 0 define biocht define SMALLUPL 2 define S.A.G.I. 3 define ARGEDXDY define aadPUP. S define Today ALS 6 define AdcAl 7 dine N2RRMSGS )

define DEFAULTTEXT COLOR CYAN

Base addres of Frane Grabber 1 / Control Port for franagrabber /

f window used in setting threshold A Resolution used to scan image for pupil A f Minimun, eye width f a tiniau glint size / wn half of the horizontal window size placed around pupil

Half of the vertical window placed around pupil / a Allowed of skipped pixels in eye cord wn of failures before recalibrating / As error code a

f Current number of tracked errors if

5,204,703 95

include Cdos.h> include Cs to.h> include (stdlib.h> includa (conio.h> include (graphics.h> include gaath.h> include Cetype.h> include c : \cgazevnatrix.de.h" include c : vcgazev fgroutins.h" include "c: Vc.gazeVgaze.h" include c : \cgarev find eye.h" include "c verical b\tools.h" include "c: vericalibvgr tools.h

include "c: \cgaze \l towyd. c. A Include file with desired calibration / / curve filt a?

AP PA 8 a a P sea a A at a h of a e s as a to a a no e s a as a as an a a ea as a a a na as e g a a Local static Wariables

in is 8 d a a e s is a A a a as a as a a as a as a e e s an a a a as s as a as a a n n n n a a

/ Coefficient Matrices and running filter static Matrix "cal coeff six - Null, cal coeff sy - NULL; static Matrix "oldcal coeffix - NULL, roldcal coeffisy - Null: Static filter type filter pos - Null:

f General Gaze routine running variables f static xycoordia pupil, glint; unaigned do recalibrate: static int x_res, yres, nun callpoints; static double x scale fact, Yacale fact: static int concolor, icon size;

static struct palette type regpalette, curpalette; static showcal points l; f when true routines Will display calibration

f information, pause, then continue 1

static int calinaxx, calaxy; / Max x and Y coordinates when calibration / / was performed /

a A A a A a h f fift it ?hift he is a in A A a a a a as a a 8 to a a a R a Global declarations

as a a sea as s a an a na as a so as a n e s as a a e e s a sa e o a as a sat it a a na n e s a n e s to a a n n e o as e a saf A currently aust be set externally a

extern int Graphidriver, Graphode;

/a variables for error Tracking f unsigned eye errarray (10); char eye errags ( ) - {

No Pupil Found", "No Glint Found", "Pupil Too Shall", "Glint Too Small", dx or dy Too Large, Pupil blot Round enough", "Too Many Failures", sad calibration"

FILZ output file :

a n e o as a a s as a as a so a 1 m a n a in a a na has a as a on a na e a as a s a s a a on a e s is a a on a n e a saf

static void in tgraph nonds (Matrix ix, Matrix iy) int count; cal maxx getnaxx); Calaxy - gatnaxy ( ; x scale fact - (double) x-res / calmaxx; y scale fact - (double)Y_res / calaxy: a witch nucal points) t

cale 2 : alanx, 0, 0) O. calNaxx; elen (ly, 0, 0) or 0.2 calaxy: elenix, O - 0.2 calinaxx; elen (iy, l, O - 0. calaxy: break;

case 4 : Olanix, 0, 0) on O.2 calinaxx; alea iy, 0, 0) - 0.2 calaxy; elea ix, 0) as O. calaaxx; elen (ily, G) - 0.2 canaxy; allen (2,0) a C.2 calaaxx; elently, 20 - 0.8 cataaxy: elen (x, 30) - 0. calaxx; elenkiy, 30 m O. callnaxy: break;

case 5 : elanix, 0, 0) or 0.1 calinaxx; elenty, 0, 0) - 0. calinaxy: elenix, 1, 0) or 0.9 calaaxx; elen (1y, l, 0) - 0.1 calaxy: allenix, 20) as 0.5 caluaxx; elen (iy, 2, 0) - 0.5 canaxy: elen (ix, 3,0) - D. l calaxx; elay, 3, 0 - 0.9 calaxy; elen (x, 0 - 0.9 calaxx; elan (ly, 4, 0) - 0.9 calinaxy: break;

cane B ; Olenix, 0, 0) - 0. calaxx; elen (ty, 0, 0) or 0.1 calaxy: elenix, 0) on 0.9 callnaxx; elepatiy, O - 0.1 cainaxy: elen (ix20) - 0.5 calaxx: elently, 2, 0) - 0.3 calaaxy: eleax, 3, 0) or 0.3 cal Naxx; elentiy, 3, 0) - 0.5 calnaxy:

on X, iO) 0.7 canaxx; elen (ly, 4, 0) - 0.5 calmaxy: elenix, 50 m 0.5 calinaxx; elen (iy, 5, 0) - 0.7 calaxy: elenix, 6,0) - 0. calinaxx; elen (ly, 6, 0) - 0.9 calaxy: elen (ix, 7,0) 0.9 calmaxx; elen (iy, 7, 0) - 0.9 calnaxy: break;

default ; fprintf stderr, incorrect number of calibration points \n"); fprint f (stderr, "specified (Vd) : Possible (2,4, 5, 8) \n", nun capoints): exit (0); break:

if (GraphDriver - EGA)

96

5,204,703 97

consize - 36: concolor as CYAN;

alse consize l; concolor l;

for count is 0; count C lis; count++) f save the config for a Regular Palette a regpalette. colors count - count:

regpalette. size a l6;

as AUTHOR: Kevin S. Spetz VN a daTE: AS/B routine generates a circular linked list with of nodes nunodes and returns a pointer into that circle. Will provide for real-time averaging of the generated lookpoints, and will halp alianate time-delays

ef ?tatic fill tertype init filter (int nuanodas)

int count: filter type near firstnode, near currentinode, near teapnode: lf currentnode a filter type near 8) allocsle of filter type)) a NULL)

return (NULL):

first node m currentinode; firstnoda->xlp 0.0; first node->ylp - 0.0; firstnode->next a NULL; lf nuanodes > 1) {

for count 0; count C nuanode - lit count + tanpnoda is currentnode: if ( (current node a filter type near alloc (slaoffilter type) 5 m NULL)

return NULL);

teapnode->next currentnode; currentinode->xlp in O. O. currentnode->ylp us 0.0: Currentrode dest NULL;

currantinoda->next first node: return (first node);

f AUTHOR; Kevin S. Spetz dAT; l/s/ destroys the running filter created by init filter

e/

static filter type kill filter (filtertype knode) filtar type nextnode: filter type start: filter type teap; int c s O;

start a teap - knode; c4+; next noda is start->next; free taap: while (nextnode a start) wn work through the loop t/

c4+:

temp nextnode: nitroci rhetnodes>naxt W. Go to next nod af free (taap):

return (NULL);

f AUTHOR; Kevin S. Spetz dAT2: WA Incranents the filterpos and pats x and y with the filtered values.

af static filter type lp filter filter type cnode, double x, double y

filter type next node: /a Pointer marking pos in circle a filter type start: ?t Pointer marking Start in circle 1 double tx as 0.0, ty a 0.0; in t c as 0;

start a code a set start to passed pointer pos / txt a start->xp: tyts start->ylp; c + +: rexerhode a start scorext; While next node - start) /* Work through the loop af

tx to next node->xlp; ty t- next node->ylp; c++ next node as next node->next; / Go to next node

nextode in nextrodeo Xnext; next node->xlp n x; next node->ylp a ty:

a twc; y - ty/c: return (nextnode); /* Return the new pointer pos a?

f AUTHOR: Kevin S. Spetz

98

5,204,703 99 100

a DATE: 7/88 , a Routine to place and icon centered around the point (x,y) a/ static void place icon (int x, int y)

pet color (icon color); circle (x, y, consize/2): circle (x, y, iconsize/4): putpixel (x,y, iconcolor) ; delay 500 :

a AUTHOR: Kevin S. Spetz dAT: WB An anti-place icon routine

of static void renovelicon tint x, int y)

satcolor (0); circle x, y, icon size/2): circle x, y, consize/4); putpixel (x,y,0);

wr AUTHOR: Kevin S. Spetz a dATE: AR The following routine manage and display the errors encountered by the gaze routina S. Problans are shown on the calibration acreen

af void clearerrs (void)

int index; for index or 0 index C NUMERRMSGS; index++

eye arrarray index 1 - 0:

A AUTHOR: Kevin S. Spetz a DATE: / th/

static void showerr (void) int index, x loc,yloc: gotoxy 2, 23 - NUMZRRMSGs); textcolor (CYAN): cprintf("Last Error Msgs : "); loc is wherext):

yloc 0; textcolor (R2D); for index 0; index C NUMERRMSGS: index++) gotoxy ( xoc, 23 - NUMERRMSGS + y loc); if (eye or rarrayfindex) {

gotoxy (xioc, 23 - NUMERRMSGS + y loc++): cprintf("YS (Ad)", eye erras gst index, eye errarray index):

a AUTHOR: Kevin S. Speta dATE: As

Of static void showcal as g (void)

gotoxys, 12); textcolor (LGHTCYAN); cprintf("Performing calibration --) ; gotoxyl 7, 3) : textcolor CYA); cprintf("Stare at each icon as it appears on the screen"); how) ;

delay (2500); Clearers ():

/s AUTHOR: Kevin S. Spetz data. As

a/ static void show egal call asg (void)

int index,yloc; change textstyle (SANS SERIF FolT, HoRIZ DIR,3); settextuatify (LEFT TEXT, CENTER_TEXT) ; etcolor (LGHCYAN :

out textxy (getnaxx () flo, getnaxy 0.4, Performing calibration sa): changetextstyle (SMALL FONT, hoR12 DIR, 5); stcolor (CYAN); out textky (getnaxxts, getnaxy ( ) 0.5,

"Stare at nach icon as it appears on the screen); out texty (getnaxx () as getnaxy ( - (textheight (x) Ver.RMSGS,

Last Error Msgs : ); satcolor R2D); yloc - 0: for index 0; index c UMI2RRSGS; index++) {

if eye errarray index) out texty (getnaxx () f2, getnaxy () - (textheight ("X") (rUMERRMSGS - yloctt ),

eye erraigs index)); dalay t2500);

5,204,703 101 102

fe AUTHOR: Kevin S. Spetz a dar: Of

routine to to the current calibration and returns success if the calibration is good, failure if the current calibration is not close enough

af static unsignad goodcal (Matrix ax, Katrix ny, Matrix nix, Matrix iy, Matrix dx, Matrix "dy)

int indet, ipx, iPy cpt., cpy: int good l; for index - 0: index < nun call points; index++) {

X- l" p? r icon polition, cp a calculated point / px int) elonix, index, 0):

cpK - tint) calclp (ux, elen dx, index, 0), allen (dy, indax, 0)); if tabs tipx - cpx) > 40

eye errarray (BADCAL)+t: good is 0;

py - tint lently, index, 0): cpy - (int) calclp (uy, elen (dy, index, 0), elan (dx, index, 0) ) : if tabs tipy - cpy > 40)) {

aya errarray (BADCAL) +t; good - 0:

if showcal point) set color (concolor); circle px, py, icon size/2): circle tipx, i.py, consize /4); put pixel (ipx, py, iconcolor): st color (RED); rectangle (cpx - 2, cpy - 2, cpx + 2, cpy t 2):

if a howcal points) delay 3000);

clear device (): return (good;

/ AUTHOR: Kevin s. spatz f DAT2: / Routine perforas a calibration and tests it for validity. Managed by calibrate.

af static unigned call (Matrix Pax, Matrix, y)

Int index, thpd, tvpd; hiatrix conx, icony, dxa, dyna: double dix, dy: static int prograa start-l;

f (program start)

f initialize again (); / program start-0;

iconx - new zeronatrix (nun calPoints, l) ; cony - new zeronatrix (nuncal points, ) ; dxn - new zeroatrix (nunical points, l: dy - new zeronatrix (nuncial points, l: if ( (conx as NULL) i (icony as NULL) dx a bull) (dya a NULL)

fprint f (stderr, Manory Allocation Failure -- Not Enough Roon for Matrices\n"); exit (0); P

in it graph needs (iconx, icony): place icon (O.S getnaxx (), 0.5 getnaxy ( ):

continuous (); newgrabnowitch(); continuous ); newgrabn switch(); fa grabfrane (); /

at thresholdo); renova icon ( 0.5 gatnaxx (), 0.5 getnaxy () }; for (index or 0; index c nuncal points; indextt) { place contalan conx, index, 0), allen (lcony, index, 0)):

f a grab frane (); / / Make Sure the IW will have a good image a/ fe newgrabniswitch(); /

ift get deltas (6dix, & dy, 6thpd, twpd, 5.0) ) { fraahiatrix (da): free Matrix (dyn) : freeMatrix (iconx); free Matrix (icony): return

ele elen (dan, index, 0) a di elen (dya, index, 0) dy;

ranovelicon telen ( iconix, index, 0), elan icony, index, 0):

if (!gen coeffs (ax, y, iconx, icony, dxn, dya) ) free Matrix (dxn): free Matrix (dy); free hiatrix (icon X) : free hatrix (icony: return (0):

if t! goodcal tax, ay, iconx, icony, dixon, dya) franMatrix {dxn); free hatrix (dya); free hatrix (icon X) : free Matrix (Acony); return (0);

5,204,703 103 104

freshiatrix da): freeMatrix (dyn) ; free Matrix (conx); frematrix (lcony); return ( );

in AUTHOR: Kevin S. Spet 2 dAT: 7/88

a Routine takes care of the screen and then call cal. Will quit to DoS a if q pressed. Returns success if a succesful calibration in completed.

a/ unsigned calibrate (Matrix ax, Matrix ray)

char scrbuf 0.252; void buffer; unsigned size; int sn; unsigned succe; int cur:

f continuous (); / clearaatrix tax); clearaatrix (ny):

f want to find out what type of screen We have before paving the screen and performing a recalibration

s/ switch (screen node () ) { A calibration called from text screen

tust save text and eatablish graphics screen and calibrate. Text restored

a/ case Ext : gettext (0, 0, 9, 24, scrbuf);

cur getcursor (); textbackground Black); circr: showcal ng (): setupgraphics (EGAEGAHI): success - call tax,y); close graph (): puttaxt (0, 0,79, 24, scrbuf); if kibhit () )

if (toupper (gatch () ) - 'O') exit (0); set cupor (cur; break:

f Calibrate called from Graphics screen with on a page of nanory. Graphics creen saved and calibration performed.

Graphics restored. a/

case ONZPAGE. : a lae - imageSize (0, 0, getnaxx ( ), getnaxy ()); if (buffar alloc (see ) a NULL)

fprint f { otderr, "Manory Allocation Error -- Not Enough heory for Screenvn"); exit (0);

getage (0, 0, getaxx (), getnaxy), buffer); get graphnode ();

restorecrthode (); textbackground Black): circr: showcalug (); setgraphnode (un); success on call tax, ny); putinage (0, 0, buffer, COPYPUT): fraa bufer); if (kbhit)

if (toupper (gatch () ) - "Q") axit (0); break

A calibration called from Graphics screen With two pages of nenory. Assuing second graphics screen free. Switch pages and calibrate Pages withched back

w case Two PAGES : aatactivepage (l) ;

setwsualpage (1) : cleardevice (): getpalette (curbalette): set all palette (regpalette); show egal calmsg); cleardevice (): buccess r call tax, ny); at all palette (curpalette : et activepage (0): set visual page (0): lf kibhit))

if (to upper getch ( ) - "o") exit (0); break;

continuous ); return success);

f AUTHOR: Kevin S. Spetz a dATE: 7/

routine called externally to setup the function of the gaze routines.

5,204,703 105 106

Manage the calibration natrix Remory, the running filter, etc. Af

void initgaze (int xr, intyr, int nuraave, int nunca) if (cal coeffix - NULL)

free Katrix (cal coeffs x); if (cal coeffigy - NULL)

freeMatrix (call coeff sy): if ( (call coeff six new zeronatrix (CAL COSIZE, l) ) mr NULL) {

fprintf(stderr, "Memory Allocation error -- Initgaze : Making coaf Matrixvn): exit (O)

if (cal coeffsy - new zeronatrix (CAL COSIZE, 1)) a NULL) printf(stderr, "Menory Allocation error -- Initgaze : Making coaf Matrix Vin"); exit (0);

pupil. x - 256: pupil.y 256;

res Kr; f resolution x lookpo in is return in of yres r yr; f resolution Y e h a nun callpoints nuncial: / specify the Number of Calibration points / if (filter pos is NULL)

filter pos - kill filter (filter pos); if t (flater pos - init filter (nunave) ) - NULL) {

fprint f (stderr, *Menory Allocation Failure -- init filter*): exit (O);

do recalibrate - l; A Want to start with a calibrate af clearers ();

/a AUTHOR: Kevin S. Spetz a DATE: 2/B Routine that forces a recalibration on the next call to get gazeposition

f void set recalibrate (void)

do recalibrate - l;

void clear recalibrate (void) { do recalibrate 0:

7 AUTHOR: Kevin S. Spetz a DATE: A3/9 Routine that allows the user to set the x and Y resolution in which the screen lookpoints are returned in,

a/ void set gazeras (int xr, int yr)

caluaxx or getnaxx (); calaaxy n getnaxy ();

scale fact (double) xr 1 calaaxx: y scalefact (double) yr / calaaxy: - x_res a Kr: yres syr;

AUTHOR: Kevin S. Spatz dATE: A WB9 Routine pet the showcal points variable to on or off. When on the calibration routines will display the calibration points and the corresponding calculated lookpoints on the screen, pause for 3 seconds and continue. The default is off. If on an 0, calibration results are not shown, they are shown otherwise.

af void showcal (int on)

showcal points on:

f AUTHOR: Kevin S. Spetz DAre: 7/ o

routine called axternally to read an x and y lookpoint and the pupil diameter

af unsigned gotgazeposition tint x, int y int hpupdia, int Vpupdia)

double dx, dy: unsigned done; int tamp : int tampy;

do if tdorecalibrate) {

while (! calibrate (cal coeffox, Cal coeffsy)); do recalibrate - 0:

done any if (get delta ( 6d, 6dy, hpupdavpupdia, l) {

filtarpo - lip filter (filter pos, id dy); a (int) (x scalefact calclpical coeffox, dx, dy) : ty - (int) (yacale fact calclpical coeffy, dy, dx)); done on

5,204,703 107 108

le return to while done;

A a output file-fopen ("crash, data"): fprint f (output file, x: dy: d\n", x, y): fellose toutput fle); af

teap-a: tempty-y; if t t teap_x > 0) 66 (tap_c < x_res) is teap y > 0) 66 (tamp y Cyres +100

return

also

as sy l; return to :

void save call (void)

dupatrix (cal coeffox, Goldcal coeffix): duPatrix (cal coeffy, Goldcalcoeff sy):

void restore call (void)

dupatrix (old calcoeffox, Scal coeffs): dupatrix (oldcal coeffay, scal coffey); claar recalibrate (): freakatrix toldcacoeffix); freelatrix (oldcal coeffsy): olcalicoes as L.; old calicoeffy ULL;

unsigned evalcalibrate (void)

return (calibrate (ca coeffs x, ca icoeffigy) : V032

tools.c define corer air corner

define corter define corer define Hrz LINE define VRT LINE

include conio.d include (stdlib.h> include ce.hd include Cdir.h> include stdio.hd include gio.h> include Cfcnt.h> include cdos.h> include (stdar.g.h> include callochd include Cotype.h> include Cstring.h> s include Craphics.h> include c. Varicalib tools.hi

static char bone26 a 28, 1992, 27, 1969), 20, 17, 200, 8, 205, 16

static int line type Double;

static int allowtagging a 0:

1 AUTHOR: Kevin S. Spetz dA: 29

Routine that extract the fillerhaue and extenson fron a path and return a pointer to that extracted string. Uses frnsplit to do this.

af char extract fin char text fin, char path)

char drive ADRIVE, dirAEIR file MAFIL2 extAre: insplit path, drive, dir, file, ext; sprintf(ext fin, ss, fille, axt): return (ext fin);

A Routine which inquires to see in what ode the screen currently rats

ef screenodes screen node (void)

union REGs res:

f it

af

screenods

109

regs...hah or of: into ( Olo, Gregs, regs); switch (regs.h.. a

3. s

c

s

ce ce c

e case ce

c

ce

e.

0.92 3 :

OS 00s

009 : x0A : COs : xOc : OG : 0x0E :

: sm - on EPAGE: OF

Ox :

returns:

in a recT; break;

break sin - Two PAGEs: break

AUTHOR: Kevin s. spet DATE: fleft

routine to define the shape of the cursor

void set cursor (unsigned int shape

f

of

union REGS reg:

reg. h. ah. . ; reg. x. cx shape: inte (Oxl O, reg, Greg);

w

AUTHOR: Kevin S. spetz DATE: Iflist

Routine to read the shape of the cursor unsigned int get cursor void

union REGs reg:

af

reghah as 3: ints (0x0, reg, reg); return (reg. c. cx):

AUTHOR: Kevin S. spatz ote: A

Routine to read in a string fron the keyboard.

5,204,703

the routine will use the current forground and background color when showing what is typed.

char get string (char estr) char C. char int done 0: int x,y:

a tr; s wherex :

y - wherey (): DetCursor (SHORT cursor): to

c getch; if t c > 3.

ast as c so ot

gotoxy (x,y); cprints str;

is (c C. la

if c -e Ret - c.

done a

if ( (c. - BCKSP) is s > str -- as or

gotoxy (x,y); cao (); cprintf is str;

while done); batcursor (No cursor): return tr;

110

5,204,703 111 112

AUTHOR: Kevin spatz (adapted from the TurboC Microcalc program) dATE: BA/9

routine accepts the string input s, and allows a user to edit that st ring. The string legal defines which characters are legal input for the string s. If legal is an empty string, then all character are legal. The string s can be no longer than naxlength. If the first character entered is an editing coaand, than the old value of is maintained, and the user may edit it. If a character is the first string, then the old string is erased, and the user starts fresh. Insert and typeover nodes are suported

unsigned edit string (char , char legal int axlength) Ant c, en a strien (s), pos len, insert l; int oldcursor - gatcursor (), firstchar l; int x - wherex (), y - wherey (), fgc - getfgc (), bgc - gotbgc () ; set cursor (TAL CURSOR); fastwrite (x, y, baci 0x08, fgc & 0x07, s): / Inverse video before first char / do

gototypo f x, y); switch (c getch ()) {

cage O : switch (c getch () } {

case R.Arrow : if pop glen)

post t; break

case farrow : if (pos X 0.

pos--: break:

case de : if (pos C lien)

nowner & spos t , ss pos), lien - pos); er--;

break; Case inst :

insert an insert: if (in Jurt) Detcursor (TALL CURSOR): else set cursor (SHORT CURSOR): break;

break; case CKSP :

if (pos > 0) { novents is pos), is pos - l, ien - pos + 1) :

break; case ZSC

en a 0: break;

default if (firstchar)

gotoxy (x,y): cleol (): len a 0. pos - 0:

O) O:

if legal 0 - 0 strchir (legal, c) - Null) ) is (c >e ' ' ) 66 (c Cee - ) Si ( (len C maxlength) (pos C len) 6 insert) ))) {

f (insert aeranovets pos t , is pos, len pos t ); lent :

else if pos de len ent t;

Spott a c;

break:

allen a 0; fast Write tx, y, fgc, bgc, S: fastwrite x t len, y, fgc, bgc," "): firstchar O: while t c s RET) 66 (c or ESC) };

set cursor oldcursor; return c or ESC:

fe AUTHOR: Kevin S. Spetz a dAT2: liff

Routine returns the current background color at the cursor position af V

5,204,703 113 114 unsigned xybgcolor (void)

union REGs regs: unsigned byc; regs. h.ah as 0x0s: rags.h...bh 0x00: ints (0x10, Gregs, Gregs); bgco (regs. h.ah 0x70) >> 4: return bgc);

unsigned get fgc (vold) struct text info t: get textinfo (it); returnt. Attribute 0x0F.);

unsigned getbgc (void struct taxt info t; gettest info (it); return (t. attribute 0x7O >> 4);

1 AUTHOR: Kevin S. Spetz DAT /2/

Routine prints a string at the specified xy location is the specified color af

vold xycprintf tint x, inty, int color, char at . . . )

struct text info ti: char st2; valist argptr; gettextinfo (it); gotoxy, y); textcolor color): textbackground (xybgcolor ()); vastart (argptr. st); vsprint flist, St. argptr}: valend targptr); cprintf("As", list): textattr (t. attribute);

as AUTHOR: Kevin S. Spetz h Art

Routine which prompts a user with the string it and expects the user to respond by entering an interger. af

int getnun tint x, lint y, int color, char st) char tempt 01 - ": int h; gotoxy (x,y); textcolor color) : cprint f "Ag", st; edit string tenap, "012345679, 0): scan f (temp,"d", Gh; return th;

An AUTHOR: Kevin S. Spetz oA:

Routine which writes a string in the colors fgc, and bic. The strials written at the point x, y and is directly written to screen neory.

a FAST, FAST: ; : Does not obey TC text window confines. af

void fast Write tint x, inty, int fgc, int bgc, unsigned char st unsigned screen seg, offset; unsigned char attr; struct text info ti; int far screenoci

if peakb(0.000, 1040) 48) - 4 B) screen seg - 0x800;

else screenbeg - 0x000; attrar (bgc (c 4) fgc. gettextin fosti) : offset - ( (ti. winleft - 1 + x - i) 2) + ( ti. wintop - 1 + y - il 160); screenloc - (int far ) MK FP (pcreenseg, offset); while at a .9)

a screenloc+ an attr (c. 8) st++

An AUTHOR: Kevin S. Spetz 8 Art / 6/

routine puts count number of chars at the location xy in the colors fgc and bgc h/

void cputchartint x , inty, unsigned char fgc, unsigned char bgc, unsigned char count, unaigned char c) long screenseg, offset; int : short at tr; struct text info ti:

5,204,703 115

if peekb ( 0000,040) & 8) - B) screen seg is 0x800;

elan screengag - OxB000; attr - bgc CC 4) fgc: get textino (it); for 3 - 0: , C count; ++) {

offset - ( (3 + t. winleft - 1 + x - l) 2) + (ti. wintop - 1 + y - 160): pokeb (acraenog, offset:++, c): pokab (Scrann sag, offset, attr);

/ AUTHOR: Kevin S. Spetz A dar: Wils/

Routine to automatically put the attribute created by fgc and byc in the screen hemory locations ?tarting at x, y and extending to xt count,y. The routine does not affect the characters on the screen.

Af

void cput attr (int x, inty, unligned char fgc, unsigned char bgc, int count long screenag offset; int : short attr; Struct text info ti:

if ( peekb ( 0000. 1040 & 4) a 48) screen seg - 0x800;

else screen seg - 0xB000; attr - bgc CC 4) fgc: get text info (st); for a 0: 3 C count: 3++) of set - ( ( t t i. Wineft t x - l) 2 + 1 + i t ... wintop + y - 160); pokeb (screenseg, offset, attr);

/ AUTHOR: Kevin S. Spetz DATE: W6/8

Routine set the attribute of the current text Window to the attribute defined by fgc and bgc

Of void set attr (unsigned char fgc, unsigned char bgc)

long screenpeg, offs at: int i ; unsigned char attr struct text info ti: unsigned char far screenloc; f' char so We can address just the attribute /

if peekb (OOOO, l040) & 4) : 4) /* deterrane the correct screen f screeneg - OxBB 00: W address /

else screense g r 0x8000; gettaxt into titl): attr - bgc gC 4) fgc; for ti - ti. wintop; i < t . Winbotton t l; it t) {

offset ti. winleft - 1) 2) t l t ( i - 1) 160); screenloc - (unsigned char far ) MK FP (screenseg, offset); for ( - ti. winleft; C ti. Winright + i ; it t) {

scrandch + is attr; /* Write attribute to screen memory of creenoct t; A skip over the character location of

A AUTHOR: Kevin S. Spetz a DAT 6/29/89 Sets what type of line is used by the draw box routine, single or double aw

void set box line (int lit) line type or lt;

fe AUTHOR: Kevin S. Spetz a date: W 6/8

e Routine drawn a box with corners defined by ulx. ...lry in the colors fgc and bic. Setting shadow to will float the box above the screen

af void draw a box (int ulx, int uly, int lira, int liry, int fgc, int bgc, int. shdw)

int : struct text info ti: get textinfo still: if (shdw) / Set the shadow attributes A/

window tulx t 2, uly l, lrx t 2, liry t ); getattr (LGHTGRAY, BLACK);

Window (ulx, uly, lirk, lry: textbackground (bgc): crocr): window, 10, 25); cputchar ulx, uly, fgc, bgc, lrx-ulx, boxneratinetype HR2LINEl) : cput char (ux, ry, fgc, bgc, lir-ulx, boxneral inte type) HR2LINE): for uly: C ly; it ) {

116

5,204,703 117 118

CPutchartulx, i, fgc, bgc, l, boxsen fline type) (VRTLINE)): cput char (lrx, i, fgc, bgc, l, boxneral line type (VRTLINE) :

cput chartulx, uly, fgc, bgc, l, box men line type ulcoRNER); cput char (lrx, uly, fgc, bgc, l, boxneral netype URCORNER): cput char tuix, iry, fgc, bgc, l, boxneraline type LLCORNER): cput char (lrx, liry, fgc, bgc, ), boxner line type (LRcoRNER}}: text attr (ti. attribute):

/ AUTHOR: Kevin S. Spetz DATE: W 6As

a routine draws a box defined by ulx. ..lry in the colors fgc and bgc and a returns a pointer to a structure defining the characteristic of the Window the active text window D set inside the box such that l, l is ust inside

s the line. The structure window type holds the xy coords of the corners, a structure with the previous screen state information and a pointer to a buffer with the screen that was under the window tra e/

window type open window (int ulx, int uly, int lrx, int iry, int fgc, int byc, int shadow, char format, . . . ) valist argptr; char pburl2: int dunay: windowtype win: if t win - Window type ) maloc ( sizeof (window type)) or NULL)

fprintf(stderr, "Error: l Memory allocation failure -- open window"); exit (0);

if twin->prevscrn - (void maloc (lrx - ulx + 3 s (lry - uly + 2) 2) ) - NULL) fprintf(stderr, "Error! 2 Menory allocation failure -- open window"): exit (O);

wn->ulx ulx; win->uly as uly: lf shadow

win-slrx is ir t 2: win->lry an iry + l;

alse

win->ry as ry;

get textinfots (wnroti)}: textcolor fgc): text background (bgc): if shadow)

duy - get text tulx, uly, l r t 2, lry t , win->provocrn); ele

dunny - get text{ux, uly, lrx, ly, win->prew scrn); drawa box tul, uly, lx, lay fgc bgc shadow): vastart (argptr., format): vsprint f(pbuf, foraat, argptr); if strian pbuf) > 0)

xycprint fulx t , uly, fgc," a "pbuf): valend (argPtri) : Window tul x t , uly t l, rx - , liry r ); return (win);

/a Author: Kevin S. Spetz a lot; 1/6/8

Routine closes a window opened by open Window. Simply pass the name of the opened window and close window will replace the information under the window and free all neaory associated with the window h A

void close window (window type win) put text twin->ux, Win->uly, win->lrx, win->lry, win->prevscrn); window (win->ti. Winleft, win->ti. Wintop, win->ti. Winright, win->ti. winbotton); gotoxy (win->ti. curk, Wiln->t. cury): textattr Windt. attribute): free (win->preva crn); free win);

a muTHOR: Kevin S. Spetz b ATE:

a Routine which displays a lightly shades bar at the location x,y. The bar is filled up to the percentage point indicated by curatotal. in f

void percent bar tint x, inty, int len, long cur, long int total) int b: static int last perctg. - 0: if total X 0)

b - int) float) in a float) cur? (float) total)); if b as last perctg) {

cput char (x, y, YELLO, BLUE, b, \333') : cput char (x + b, y, WHITE, BLUE, len - b, ' ' ); yeprintf(x + len/2 - 4, y + 1, YELLO, "4. Of", float) ( 100.0 float) curl (float) total)); xycprint? (wherext), y + 1, WHITE, "I done") ; lastparctg. b:

5,204,703 119

/ AUTHOR: Kevin S. Spetz AT: Routine to check and see if a fle exists

af

int file-exists char rst) int han; lf than - open (st, o RDONLY) ) - -l) {

close than : return ( ;

return 0;

a AUTHOR; Kevin S. Spetz dAT: Sts/9

Routine which takes the parameter n and nakes sure that it is not greater than t or less than b. If it is, n is clipped to these boundaries.

void bind (int in, int b, int t) if an C. b)

in a b: else if n x t)

in a t

t t t A f it A R a t d at it A A A a a a

Routines for showing a directory on the screen and selecting a file from that directory. These routines rely on the routines above that open windows etc.

f AUTHOR: Kevin S. Spetz dAT: A 6/88

Routine that scans the directory defined by fillespec and returns a pointer to a linked list of these file names. Also returns the number of flies in the directory

th/

static dirtype gatdir list char file spec, int nunafies) int done; struct ffblk tapff blk: dirtype firstentry, curentry, tapentry; char far curdita; curdita e getdta ():

done - find first file spec, stapffbik, 0): if done) return (NULL); nufle a l;

if (curantry - fair type ) malloc sized f { dirtype) ) -- NULL) fprintf stderr, Zirror! Memory allocation -- getdir list"): ext l;

firstentry - curentry; first entry->naxtentry - NULL; firstantry-> tagged - 0; stricpy first entry-> filename, trapff blk, f finane); while ( : done)

trapentry - curentry; if done on findnext timpf folk sa O

nunfliest t; if (curentry dirtype ) alloc (sizeof (dirtype)) ) - NULL)

fprintf stderr, "Error! Menory allocation -- getdirii at"); exit ();

tapentry->naxtentry - curentry: strcpy (curentry->filename, tapff blk.ff name); curentry->naxtentry a NULL; curentry-stagged - 0:

setta (curdita); return (firstantry):

/ AUTHOR; Kevin S. Spetz a dAT: WSW

Routine that destroys the linked list containing the directory entries. The linked list is created by the routine getdirilist a/

void kilidrilst dirtype dirlist dirtype curantry, nextentry: nextentry - dirlist->nextantry; curantry a dirlist: while (curantry is NULL) {

free curantry):

120

5,204,703 121 curentry as nextantry; nextantry curantry->nextantry;

Au Thom; Kevin S. Spetz dA.; lil 678

Routine that nove the pointer to the directory list forward from the pointer position startentry to the position startentry t nun. If the new location As past the end of the list the routine returns a pointer to the last flie in the list

f

static dirtype novaplualdir (dirtype startentry, int nun) int count - 0: dirtype curantry, preventry; preventry or curentry a startentry:

while ( (curentry->nextentry as NULL) is count C. nuva)) { prevantry is curentry: curentry - curentry->nextentry; counttt

if (preventry->nextantry as NULL) return prevantry): I don't lose list if run into and 1 else return (curentry);

/ AVrhoR: Kevin S. Spetz s AT; fs/

e Routline that displays the directory dir starting at startnun and displaying the files is a rectangular matrix defined by rows and cols af

define color BLUE. atatic void showdir (dirtype dir, int startnura, int cols, int row

int col, row, done; dirtype curentry: char tis;

cos s : row is 0; done or ;

? ) c. sc. curentry - noveplus dir (dir, startnun? : while curentry - NULL & E ( : done)) {

sprint? (t,n-12s", strawr (cur entry-> filename)); fastwrite col 1.5 + 2, row t l, WHITE coloR, t) : if (curantry->tagged)

fastwrite coln list, row + 1, WHITE, COLOR,"f"); + +row X rows - i) row 0; lf +col + 1 > cols)

done l;

curantry a curentry->nextEntry;

f in AuTHOR: Kevin 5. Spetz to dTE:

a routine which extracts all the files in "dir list which have been a tagged". The tagged list is returned in nake taglist. af

static dirtype naketag list (dirtype "dirt) nt done as 0;

dirtype firstentry, "curentry, "tapentry: dirtype taplist:

taplist - dirlist; while tapist -> tagged) & 6 (taplist->nextentry NULL } { timplist taplist->nextantry; if trapli st->nextentry - NULL)

if tapli st-> tagged) if first entry - dirtype nailoc ( sizeof (dirtype)) ) -- NULL) {

fprintf stderr, "Error! Memory allocation rr nake tag list"); exit (l) ;

return firstantry);

else return (NULL);

if it firstantry - (dirtype ) malloc ( sizedf (dirtype}}} - NULL) fprint f (stderr, Error: Menory allocation - naketag list") : exit l;

firstentry->next entry - NULL; stricpy firstantry-> filenane, trapli st->filename) : curantry first entry: while done)

122

5,204,703 123 124 taplist traplit->nextantry;

if (tapist-> tagged) is traplist - NULL)) { tapentry - curantry; if ( (curantry (dirtype ) realloc (size of dirtype}}} - NULL)

fprintf stderr, Error! Memory allocation -- uaketaglists: exit();

tupentry->nextantry - curentry: strcpy (curantry-> fillenare, tripl is t-d filename); curentry->nextantry - NULL;

if tuplist - NULL) done - 1:

return firstentry):

a AUTHOR: Kevin s. spetz dAT 2/29/9

routine which sets the allow tagging flag for the select file routine. f tagging is allowed the routines will allow the user to press T when t

highlighting a file and tag it. When the select file routine is exited all of the tagged files are returned in a list

n/ void set file tagging (int stat)

allow tagging - stat;

fe AUTHOR: Kevin S. Spetz fe daT2: l/SW

e routine which displays a directory defined by file spac, and allows the user to elect a fille with the arrow keys, etc. The filenane is returned

e by the pointer filanane. The Window used is defined by xstart, ystart, a rows and cols. If all of the directory entries won't fit in the box, the user can page up and down.

s/ define color Ghoya dene color2. CYAN

unsigned select file (char file spec, char fillename, int taglist int xstart, inty start, int cols. An t rows) window type buf; dirtype dirlist, curentry; int nuadrfiles: int startnun, row, col done - 0; char efn MAXFIL2+MAX2XT), boxlab6);

if (cols X 43 cols in 4; if (rows X 21 rows 21; if (start + 6 cols) + 3 > 0 xstart O; if ystart t rows + 3 > 25) ytart - 0: if (cols X 2)

strcpy boxlab, fillespec): alse strepy (box lab, extract frtefn, file spec)); buf open Window (xstarty tart x at art t (cols i5) t , ystart t rows t 1, coloRi, color2, l, box.la b);

if dirlist a gatdirlist file spec, inundir files)) a NULL) fatwritel, , HITE, RED, No entries : getch () : close window (bu?); tricpy (filianane," "): return (O);

startnun a 0. row up 0: cot a 0; showdir (dirlist, startnun, cols, rows); cputattr (col 5 + 1, row, HIT2, RED, 13): while done)

switch (toupper (getch () )) cajo O : cputattr (col S + l, row, WHITE, COLOR2, 13):

switch (getch ()) { case uparrow ; if (--row < 0) {

row a row a if (orco C. O)

col a ; ow 0; if -startnun X -

showdir (dirlist, startnun, cols, rows: else startu o 0.

break; case druarrow ; if (startnu t col rows t t 4 row > nuadir files - 1)

rota; else if row X rows - 1)

lf ++co X cols - ) row.-- coss; show dirt dirlist, t t startnun, cols, rows :

else row a 0:

5,204,703 125 - 126

break; case Rt Arrow : if t t col X cols - )

col- -; if (startnun a startnura t rows) > nuradir files - startnut a start run - rows;

else if (startnut t col rows row > nundrflies - ) if t-r-col & 0

col+ +; Strut - Statnur - rows;

criscr); show dir (drlist, startnun, cola, rows :

else if t startnun col rows it row > nund irfiles - 1) co-- break;

cage farrow : if --col c 0) { col in O:

i if t (Startnut - startnue r rows) g ) startnut a 0;

showdir (dirist, startnun, cols, rows);

break; case PgUp : startnun startnun - rows cols;

if startnun C 0) startinua - 0;

row 0; co) - 0; criscr) ; show dir (dirist, startnut, cols, rows); break: startnut a startnue t rows coln; if (startnun C nuadir files)

row a ; coli o 0; circr); showdir drist, start run, cols, rotif):

else startnun an startnue r rows cols: break; col in O: row an 0; break; col or cols - ; row a rows - l; if startnun + col row row X nuadrflies - 1) {

col a (nuradir files - startinua - l f rows; row a nudir files - startinua - col rows) - l;

. . break:

case PgDn

case hone

case 2nd

cputattr (col 15 + , row, WHITE, RED, 13); break;

case 'T' : if allowtagging) { cputattr (col 15 t , row, WHITE, color 2, 13): curentry - aove plus dir (dir list, startnun col rows row): curentry-> tagged - ) - curentry-> tagged: if (cur entry-> tagged)

fastwrite cols +1, row + 1. WHITE, COLOR2, "?h"); else fast write (coll still row HITE, COOR2," "): if startnun + col rows t t t row X nundrfiles - l

rota -: eise if trow X rows - l

if (++ cold cols - 1) { row - -; coll-on; showdir (dirist, t t startnun, cols, rows;

else row - 0:

cputattr (col 5 + ... row, white, RED, 13) : break:

cane RET : done ;

curentry - rovepius dir (dir list, startnun t col rows t row) ; stricpy filename, curer try-> filename); break;

case Esc : strcpy filename," "): A cancel file request ef close window (buf); killidirist (dirlist); taglist in O: f Set tag list to null of return (O);

close window (buf) : staglist n (int) make tag list (dir list); killdirilist (dirist); return ():

ChoosebirFile char fillespec, char filanana, char heading) int colos;

5,204,703 127 int rowso: int doneso: int cols: int ros-s: int nun files: int startnunro; window type fin: dirtype dirist, curantry: fin into 5,025, YELLO, eluz, O. heading if tdirist - Stirlist file opec, snuarie see NULL)

fatwrite ( , ), WHITE, RED, a No Entries gotch close window fin tricpy (filenana, a return to

holdir (dir list, startnun, cols, row, CPut attr (1, row, WHITE, RED, 20 While done

witch toupper getch () case 0 : GPut attr ( ), row, WHITE, BLUE, 20

Witch getch case Uparrow : if --ro O)

of r row - if (--col ( 0.

col o: row - 0 if (r-startnun > -

her (dir list startnun, co, row else startnun - 0

break; a DnArrow : if startnun rows t t t row > nuarles -

ro-s: else if (row X row -

if (**col. X cols - 1 ro--; coll thodir (dirlist, t t startnun, cols, ro

else row a 0

break: case PgUp thun at artinua - row a col

lf (startnun ( 0. startnut - 0

row 0; col - 0 cracr);

dir (dir list, startnun, cols, ro break; case PgDn tartinue startnun + ro a cos;

case hole

case Er

if t startnun k nunfilles row a 0. co e 0. Clr scr); hoir (dir list, startnum, cols, ro

else

startinuin a startnun - r cols: break; : col 0.

row as 0. break

: col cols - row re rows - if (startnum + col a rol row > nun files -

nurn files - startnun- l) a rows: nurt files - startnu - col rows) -

CPut attr (l, row, WHITE, RED, 20. break;

case RET : done - Ctirantry - vPlus-dir (dir list, startnun ' co rota row Pyfilename, curentry-> f in break;

Esc ; strepy filena, a A cancel file request a f kill dir list (dirlist. close window fuin: return to

close window (fwin kill dir list (diriat return

A main ()

int x: window type obur: char ft 66 as a int fge or ), bgc: clocr; Print (stdout, started with but Win", core left

128

5,204,703 129 130 textbackground (BLACK);

while (select file ("c: V\accade\ \pictures\V. pic", f, fgc, ss, 2, 14 fprint f (stderr svn, f:

fprint f (stdout, "Ended with Wuwn, core left();

grtools.c include castdio.hd include cconio.h> include Cstdlib.h> include Cuath.h> include c.graphics.h>

int fred:

struct linestruct int xl, y, x2, y2: unsigned buf:

; typodef struct line struct linetype:

int Graphidriwar, GraphMode:

a Routine to set up a graphics screen. Set up to require an EGA card, can be designed for autodetection.

af void setupgraphics tint GD, int GM)

nt arrcode: Graphidriver in Gd: GraphMode or GM; initigraph (Graphdriver, SGraphMode, c: V\turbocVV): errocode a graphrenult (); if (arrcode a grok)

closingraph(); textcolor LIGHTRED); clrscr); cprintf("ERROR!, Encountered Graphics Error : Ys, grapherrorasgarrcode); ext ( ):

wold new set upgraphics tint GD, int GM) Snt or rcotia: Graphdriver a Gd; GraphMode as GM:

if (registerbgidriver (EGAVGA driver) C 0)

fprintf stderr, Error with EGAvGA drivervin"; exit (l) ;

intgraph (Graphidriver, Graphiodac: V\turbocVV"); errocode - graph result (): if terrcode - grok)

close graph (); textcolor LIGHTRED); Crscr); cprintf("ERROR, 2ncountered Graphics error : Ys", grapherrornsg (errocode): exit ():

f AUTHOR: Kevin S. Sputz e AT:

Routine to change the text style used by the graphics libraries. Checks for eror. Af

void change textstyle (int font, int direction, int char size) lint Errorcode:

graphresult () : A clear error code af set text style font, direction, charize; Errorcode - graph result (); A check result th/ A f ( Errorcode : grox i wn if error occured Af close graph () ; print f * Graphics systera Error: svn, grapherror as g( Error Code ) }; exit (l) ;

lina type place line (int xl, inty, int x2, int y2, unsigned color) { int x, y, xend, y end, p, consti, const2, Points; unsigned dx, dy;

line type topbuf; unsigned inabu f:

if tupbuf maloc (sized fline type}} ) -- NULL

135 void setuouse cursor (int hotx, int hoty, int bitmap) union REGs reg:

reqs. x.ax - 9: rega . x.bx - hotx; regs. x. cx - hoty; regs. x .dx - bit Rap: Es - DS: rt 6 (0x33, regs, regs) ;

void set mouse shape (int sh) switch (sh case ARROW :

set nou secursor (-l, -l, tint) arrow): break;

case hoRGASS :

flet house cursor (7,7, tint) hourglass); dreak; case PONTING AND :

set_nous a cursor (5, -l, (int) pointing hand); break;

void get nouse position (int rub, int inx, int any) int al; all - 3; pnoue (N), Nab, x, y):

void bet_nouse_position (int nux, int. Ray) mouse 4, 0, x, ny);

void aouse button presses (int button, int “bs, int bc, int. "bx, int by) int nl: a S; pnouse (dial, button, b, by); bc in button: bs as a1;

void nou be button releases tint button, int "bs, int. bc, int "bx, int by) nt al.: all on S; paoue (Sn), button, b, by : bc. button; be an a

vold at nou Behninuax (int hain, int haax} house (7,0, hain, hanx);

void set Rouse vainnax (int vain, int winax) ouse, Owain, winx);

int get ouse click tint ax, int ray) int be as G, duy - 0: nouse button releases (LEFTBUTTON, 6 dunny, Gbc, ax, ny); if (screen node ( ) - TEXT) {

ax o ax7 t l; ny as ay/8 + l;

return (bc);

int aou Derb down (void) nt all a 3:

int n2, a3, n.4: pnoue (in Graz, Guin); if (n2 & 0x2)

return ( ): tle return 0;

int nous elb down (void) int all as 3; int n2, a3, ra: paouse inl, a2, a3, 6 m) : if n2 is Oxl

return ( ; else return 0;

W032

timex.c include Cdos.h> include Cstdio.h> include Catdlib.h> include "c: vericalibvtinex.h" define TIMERO Ox40

5,204,703 136

5,204,703 137 138 (cene cirr ori

define READ RACK Oxe2 define BossG 0. define Tics to ox6c define Tics. HI oxse

dane COVERT TH is. 6.36s

nigned long timer (void unsigned tapl. tP2 tap3, tap4, hi, o, int status: unsigned far ?addl tangned far add2. long ticks, ticks2;

add unsigned far in Iosses 16 t Tics lo); add2 (unsigned far a BOSSG a TICs HI); disable): outp (CNTRL, READ ack): status-inpri Mero lo (unsigned) inprizRoy hi" (unsigned inprero tripl (unsigned add tap2 unsigned add2. be

h in c < 2 lo

sat : tap (un gned 8 add

P4 (una igned add2 teable :

lf (lo & 63 ) hi-s: hi-hi is 023) > if status 12 hi. los2;

tickel (long) tap2<<ls tapl; ticks 2 (long tap cCl6 tap3;

if (ticks2 X ticks 1 + 1 return (hi sl2 return (tickal Celo hi;

else return ticks2cco hi) :

float ticex void ( unsigned long tanpck:

float tieck; teep clk timert time clk a (at) tap_clk/convert Th: return time cle:

A an

unsigned long a tine, etine; foat elapsed; int randore nurt;

Print f "program begins...wn

while

Print? ("press ENTER to start timing \n"); get chari andonun-randon 30 to

ztime-timex delay trandon nun etnetiaex

if (ztine CO etine & 0 break; printf "delay: d\n", randon nu, etiae-natine

Ped-float) etiae/.36s Print? ("time was \n", elapsed);

Printif ("tining error Wnt fW2

5,204,703 139

imagebox.c fa a e is a s as a n is an a as a is a a in a a a in a c e s a h is a a a to a a is a is in a n e s h m in a a a a

ox to dyes for dc orTA Ecos

oWar. A cardo Age scaleN.

f

t h

8

Auror: Nirav R. desai AT: A29 aff

in a s h a a s h m an in a in a t e in as a in a a a a a a a s an an of a a a of

include c tdo.h> include Cdoshid include glochd include (stdlib.h> include (conio.h> include Cstring.h> include Co.h> include (cnt.h> include Cdr. hd include guys WStat.h> ?include Catdar.g.h> include (ctype.h> include cath.h> / include (graphic.h> include c : \cgazeVf groutins.h include "c: \cgazeVgaze.h include c. Varicawtools. include c. Varicalibvrtools.ht include "c: Varcalibvaenu.ht include c vericalbynous tool.ht include 'c: Werical bytine.h. include "c: \maryland Vangobox.h include "c: Vaaryland v sacimpreh include civaaryland\acresul.h include "c: Waaryland Vaccade.h include c : \ryland vacboxes.h f as saccadev sacboxes.ht/

typed char aerulabel. 2;

menu label box defenus - clear a box, save box File, redit Box Fille, Define a so, show a box out;

activebox type boxdaf boxa - NULL; int colors 6 as EGAli GHT GREEN, EGA LIGHTR2D, EGACYAN,

EGA YELlo, EGA LIGHTBLUE, EGA anon:

box deftype image boxes (Max boxes);

void bat boxide boxes (void int i , Y: for 0; C 6: ++

a 2s y - i.6) C3 O; 35. if (box defboxes as NULL)

boxder boxes - start activebox titl, x, y, x+20, y + 32); el be add activebox (boxide boxes, it , , y, x+20, y +32);

void initiaage boxes (void) it is for ti - l; iCrMAx Boxes; ++)

inage boxes t-l.ul. x s - ; in Age boxes -ll.ul.y is el; age boxe-l.br. x s - ;

image boxes i-l.br. y -l:

f AUTOR: Nirav R. Desa ATE 7/27/9

Draws a box using the predefined graphics function rectangle and labels the box with the specified menu abel.

f void Drawa Pox (int x, int y), int box color, char tenulabel

setcolor (box color); rectangle (xl, y, x1 +20, y +32); satcolor (EGA WHITE) : out textxy (alt los y +6, aonullabel);

AUTHOR: Nirav Desai da f27/so Routine which present a menu choices to user for defining boxes on the screen age. Uses ouse input.

f

140

5,204,703 141 void show box defchoices (void)

int x, y, i. set fill style (SOLID FILL, BLACK); bar (0, 0, 640, 100) bettextiustify (CENTER Text, CENTER text): bettextstyle (SMALL FONT, HORIZ DIR,6}; for tiano; icos; ++

(3 25 ye (i.6) C3 2 (0 : 35: Drawa Box (x, y, color still box definenu (1));

static int boxchoice (activebox type abi, int x, inty) t activebox type tapb; tapb as abl; while tapb a NULL)

if (x > tapb->ulx) & 6 x C tapb->lrx) & y > tapb->uly) i y C tapb->lry) is (tinpb-Xenable

return (tapb->nua). tapb - tapb->next;

return (0);

/ AUTHOR: Nirav Desai AT: 7/27/90

Routine which uses mouse input to deterraine selection from box defenu. int get boxide choice (activebox type abl)

int x, y, curbox;

set fill style (SOLIDFILL, ZGABLACK) : settextstyle (SMALL FONT, HoRIZDIR, 4): bettext justify (LEFT TEXT, CENTER_TEXT): bar ( 0.70, 639,95); out textxy (0, 75, "Select a choice from the menu above."); set_nouse shape (POINTINGHAND} : aou belon (): x as as:

while (curbox - boxchoice tabl, x,y)} { while (get house click (6 x, y)) { }

mouse off () raturn curbox);

A AUTHOR: Nirav Desai f DATE: 7/27/90

a routine allows user to define corners of box areas on a graphics af screen using a nouse interface.

void gathouse pos (unsigned int xpos, unsigned int. "ypoa) int xpos y poa: set mouse shape (POINTINGHAND); mouse on (): xpos as - : Ypos - -l: while (! get Nouse clicktix pos, typos) ) { } f printf("xpos is d, Ypos - 1 d\n",xpos, yipos); / house off (): xpos - xpos :

typo sypos :

AUTHOR: Nirav R. Desai AT: s/2/90

Routine displays Ax Box.ES boxes, each containing a number between aust choose to or l) and MAx Box25, inclusive. User uses the noue to select which box number he wishes to clear, define or show, as defined by nig.

int choose box nun (char asg, int aust choose) int i, box: divt. ni. char ch3;

unsigned int boxx boxyro; settextstyle (SMALL FONT, HORI 2 DIR,4); sat color (EGA WHITZ): set fill style (SOLIDFI Ll, EGA BLACK); bar 0, 0, 639,95; bett at justify (LEFT TEXT, CENTER_TEXT): out to try 0, s, agh; set tact justify (CENTER_TEXT, CENTER_TEXT): for 1-gust choose: ic-Axl Roxzs: it +)

ractange i40, 80, title40-194): outtaxty (40t 20, 7, itoa (ich, lo));

142

5,204,703 143

while ( boxx) ( (Axeoxes + 1 a.40) box cut choosato) boxy CBO boxy>9)

get house pos (iboxx, boxy):

in div boxx, 40; box n. quot; bar to 70,639,95); w settext justify (LEFT Text, cENTER Tzxt): rete box:

void open file char pathnane) char drive Ascowe, dirAxDIR box file MAXFILE), ext{MAXExT): int ; char ch: char teappath APATH: Aprintf("Input the name of the box file (xxxx.ox : ; gets (box file); / bar to 70, 639, 95): settext justify (LEFT text, CENTER Text: outtetxy (0, 75, "Input the name of the box fille: );

as to

teappathi - ch (char) getch (); tanppathitl V0; bar 200, 70, 639, 9s); out textsty (200, 75, char teappath);

: while (ch Vr is ch - win? ch - s iCMAxPATH

teappath t i- a vo; fn split teappath, drive, dir, box file, axt): if strlen (drive) rO stricpy drive, "c: ); if strlen dire0) stricpy (dir, VVSACCADEvveloxv\); lf strlen extes0) stricpy text, ". Box"); finnerge (pathnane, drive, dir, box fille, ext):

void save box file (void) char pathnane AcPATH);

oute; int l;

open file pathnane); outflo fopen pathnanew"; if outfile { } {

perror systern error encountered); exit {i};

el. printf out fille, Vn BoxMARK): for is: A C-MAx Box.ES: it +) {

fprintf(outfile,"d hd ad Advn, image boxes (i-1).ul.x, age boxes i-1 ul.y, image boxes -l, br., age boxes -l .br.y):

foclose outfile;

void edit box file (void) char pathnane AxpaTH): nt : are

marker type marker;

open file pathnane); in file is open pathnae, tr: finfil & O

perror system error encountered; ext ;

els fiscarf infle, havnarker): for all: CaAx boxes; it ) {

fiscanna, did d\n", age boxes (i-1).ul.x, Gimage boxes - uy, Ginage boxes - i.br.x,

age boxes i-l.br.y);

follo infla:

void show corner (int x, inty, int color) pet cold coor; binds, 0, 635); bind (ly 5,345); a alloc (imageSize tax 0, x - 4}, y - 4, x + 4, y + 4 ;

144

5,204,703 147 148

delay looo); no sound): bar 0, 70, 639,95); return;

else

boxptr r show box (image boxest box-l. ul. x, iage boxes (box-l.ul.y, - image boxes ( box-1), br.x, image boxes (box-l. bry, w EGA WHITE) : delay (500);

renova box (image boxes (box-1). ul. x, image boxes box-l.ul.y, image boxes (box-ll.br. x, image boxes box-1).br.y, boxptr, 0):

free boxptr; inage boxes (box-l.ul. x - -l: image boxes (box-1)..ul. Y - -l: inaga boxes box-1).br. K - -l: image boxes (box-l.br. Y - -l:

void define a box (void) int box: unsigned int xl.-1, yll, x2-0, y20; wold cl: char a pagall; unsigned int boxptr; if box - choose box nun ("click on the number of the box you wish to define O to quit: , Op. 1-0 while (Xanx2 y >-y2} {

bar 0, 70, 639,95): sprint message, "Une the nous a pointer to define the top left corner of box Ad", box); outtetxy(0, 75, essage): get ouse posixl, Yli inage boxes (box-1). ul. x -xl: image boxes box-ll.ul.y Pyl; cl. s show corner (x,y), EGA WHITE): bar 0, 70, 639,95); outtetxy (0, 83, "Now define the botton right corner); got house pob (6 x2, y2): image boxes box-l l.br. K. x2; inage boxe? box-l.br. y -y2: remove corner (xl,y1, Cl); bar to 70,639,95);

boxptr or show box (x1, y), x2, y2, EGA WHITE): renovebox alyl, X2, Y2 boxptrl) ; free boxptr);

void show a box (void) int box: char essagell; unsigned int boxptr; box as choose box nun ("Click on the number of the box you wish to view: ", l; if (image boxes (box-1)..ul. x - - )

bar (0, 70,639,95); sound (400); sprint finessage, "Box Ad is not definedVin", box): out text xy (0, 75, and sage); delay {000); cond ()

bar 0.70, 639,95); return;

as a

boxptr - show box (image boxes box-l).ul.x, image boxes (box-1).ul.y, inage boxes (box-l.br.x, image boxes (box-l.br.y, EGA WHITE):

renovel box (image boxes box-l).ul.x, inage boxes box-l.ul.y, image boxes (box-l.br.x, inage boxes (box-l).br.y, boxptrl);

free (boxptr;

18 AUTHOR brav R. Dasa a daTz: 7/30/90 This is the main routine which allows manipulation of boxes which define the areas of interest on the image screen Af

void define image boxes (void) nt choice, end; initiange boxes ( ); show box defchoices (): - for cond end;

switch (choice - gat boxdaf choice (boxder boxes)) { case : clearbox () :

break;

5,204,703 149 150 case 2: Dave box file () ;

break; case 3: edit box file ();

break; case 4 ; define a box () :

break; cape 3: show a box ( ) ;

break; cane 6: end l;

break;

sound choice50); delay (200): no sound ();

closegraph; return

V032

Saccade.h defire HAFS2e 2 define FULSE.2 f Marks a Full size image 1 define Reduced O W Marks a Reduced laage 1 define WAIT a loadinage will wait for a key / define NoAT / Loadinage won't wait... used hen performing test f define ATCh 0 / File type flag, used to decidr where to look for file 'f define RESULTS 1 fe a --- a? deine cure 2 f a n w --- h/ define S Text 3 fth at --- A define box 4. f a n --- at

1 Then the file format for the four different files that saccade deals with are changed, the fille version marker should be updated

define RSARK RS3 Wh host current Result file version / define charx IG" ft lost current Picture fila varion 1 defin TxTMARK TxT fe biost current Text file version / define TCARK rel f Host current satch file verion / define boxark ax fND mod/ / Most current so ille version / define ENUFG Te 7 standard menu colors used f define NG LU, a FG as Foreground, so a Background / define MZNUSC Yello / sc up select Char color / define ZNUBR Red ff R an ear color /

define image start 10 f onallast usable x frame coord 7 define Xinage stop 490 a largest usable x frame coord / define yaage start 24 a smallet usable y frame coord f define y inaga stop 470 a large at usable y frame coord / den NUPICS 2

include car.h> include citat.h>

f Type def for marker at beginning of file to specify type f typodef char narker type 4):

/ Structure for information header for picture files / struct pict rect char aarbor 4 ; int size; char than 2:

;

typedef struct pict rect picture:

typedef char inagenane MAXPATH

struct in age struct naganane name:

char active:

typeda struct Rage struct image type;

f structure holding a list of images to be displayed on the same screen 1 fed mode/

struct imageliststruct nt size; image type image:NUMPICS: char results MAxPATh); f The results file for the screen f char boxdefs (MAXPATh); The box file for the screen f ;

typedef struct inagelist struct imagelist type:

/ data structure for information gathered during a test / struct data struct int x;

5,204,703 151

int y; int hd: int w; int aark; float time:

typedef struct data struct data type;

struct subjects truct char name (41); A 40, 10, and so characters pius char number lll: A the NULL terminator / char infoil);

; typed af struct subject struct subject type:

void close outfile (void); void status charst); void boundlines (void); void reove status (void) : void initcolors (void); void icon (int x loc, inty loc): unsigned valid file (char st, char w); char, make file name char Pafn, int typefile); char getnage descr (char name); unsigned char create text out file (FILE *h, char st, char attr): unaigned get new filenan (char "f, char asg, int typefile): unsigned getbox filenane (char f, char *ng): unsigned get results file (char results name, char file spec): unalgned getinagelist laagelist type i, int test): void drawinage (int size, int. Wait, int boxx, int boxy); wold loadinage inagelist type i, int wait); void gave aa go vold); void show aain (char n); V032

imagebox.h define KAxboxES 15

typedef struct corner int x: inty:

}; typedef struct corner cornertype;

typedef struct boxdef corner type ul; corner type br;

;

typedef struct boxdief box deftype:

void pet box defboxes (void ): void get house pos(unsigned int xpos, unsigned int. "yPos): int choose box nun (char asg, int aust choose: ; unsigned int show box (int x1, int yl, int x2, inty2, int Color) : vold define image boxes (void) : void ranovebox (int x1, int yi, int x2, inty2, unsigned intract, int wait) : \032

sacboxes.h define BUFSIZ 275 struct activebox struct

struct activebox truct rhext: int num; int ux, uly, lirk, liry; int enable;

; typedef struct activebox struct activeboxtype:

void satga za speed tint sp): void set box mark (int on); void set noboxnark (int on): void place-lptint x, inty, int color): void renovel p (int x, int y, void s ); void add activebox tactivebox type abl, int nun, int ulx, int uly, int lirk, int liry): activebox type start activebox (int nun, int ult, intuly, int lira, int iry): int lookupbox (activalbox type able int x, inty:

ef

152

int getbox (activebox typerabl, int box, int han, unsigned maxlp, int useboxes, float start-tin): Yold disposal activeboxlist (activeboxtype abl) : VG32

5,204,703 153 154

gaZeumd.h struct filterstruct

struct filterstruct near next double xlp; double ylp;

:

typedef struct filter struct filter type:

void intgaze (int xr, intyr, into nunave, int nuncal): void set gazeres (int xr, int yr): void set recalibrate (void) : void rebetical pts (int nuncial) : void clear recalibrate (void) : unsigned get deltas (double dx, double dy, int hpupdia, int vpupdia, double tdx, double toy, doubl 8 hur;

unsigned gotgazeposition (int x, inty, int hpupdia, int vpupdia): void fill Grarrays (int nunstart, int nunstop, int anove, intynova, int A, int B, int c, int cal): void get animation configuration (void); void a signania buffers (void anim buffer (3), void sailey); void init graph needs (Matrix *ix, Matrix iy, void ania buffer (13), void aailey); void animate (void ania buffer (13), void sailey, int counter) : void clear graphics (void} : static unsigned callbrate (Matrix ax, Matrix day) V032

sacimpreh typede f double rhistarray (256):

void snooth image (void); void reduce image (void); void equalize image (void ): void rever beliuage (void); void binary image (int threshold); Wo2

sacresul.h void ouanarize results (char res file) ; void list results char res file) ; void graph results (char resfie): void supiap results (char res file, int. Wait forkey): void analysis options (void) ; \ 032

boxdefic for a nons as as . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . an an enia a a rare as a a an an

AUTHOR; Kevin s. Spetz AST REVISION: S/SA9 As odification 3/29/90

B

a Progra which allows a user to generate text and picture stimuli, then a nonitor a subject's pupil diamater and look-point as the stimuli are presented on the screen.

a A a has a a a a a A set t t a a at he h a a

include (stdio.h> include CConio.h> include (string.h> include Cio.h> include Cdos.h> include Cotype.h> include cotd.lib.h> include calloc.h> include Cstdar.h> include Cfcntl.h> include castat.h> include Canth.hd include Cyraphics.h> include "c: Vcgazav fgroutins.h" include "c: Vcgazevgan.h" include c : varyland \sacboxes.h a was saccade\pacboxes.h? include c: Varicalib\tools.h" include c \ericalibvgrtools.h" include "c Wericalib\menu.h include "c: Vericalib\nous too.h" include "c: Vercalibvtnex.h" include "c: Vaaryland viaagebox.h" include "c: \maryland\sacimpro.h" include c \maryland v sacresul.h" include "c: Vaaryland\saccade.h"

extern active box type boxide fooxes; char outname l;

te static

static

static

static

paletta type palette; aubect type subjectinfo;

int filterpts as :

int calibpts 8:

int box.lp on as O:

5,204,703

f unber of points used in the running look-point filter. Used Whan intgaze is called

af

fe Number of icon positions used during calibration. Sent to ratgage

t

to Determines whether or not a look point hown within active boxes.

A.

f a Determines whether or not a look-point shown outside of active boxes. If

boxes are not active, this deteranes whether a look-point is shown at all.

f Total number of look-points allowed on an individual screen.

f Number of look-points required to

static int nobox.lp on - 0:

f

static int totallip 0;

af

st at ic int. nurn box.lp - 0: select a box.

af

static int useboxes as O: A Flag to determine whether or not the screen boxes are active or not.

af

f Standard menu color definitions used by paenu software in menu.c

af static nanucolor type blue menu WHITE, BLUE, YELLO, RED, L1 GHTGRAY}: static enucolor type grayaenu - BLACK, LIGHTGRAY, WHITE, RED, DARKGRAY :

static enuiteratype main menu ( ) or

1 enu definitions used in the saccade et settle

Af

digitize and Store an age", l, 20, Generate and save a Text screen", 1, 0, "Perforasaccadic aotion test", 1, 0, "Analyze results,

(, "Register a Sub?ect", l, o, out 1, 0, to

static int a stan option - 0:

static enuitentype picture menu ( ) - 7, Grab a Frane,

, o, continuous, l, "Make inage Binary",

0, "Draw the laage on a graphics screen", lo, "Equalize the image",

25, reverse the intensities (legative), l, ld, Perfor image Reduction, l, Average inage intensites, snoothing, l, 2 box on inage, l, "Load inage from disk, O'Sava nage on disk,

l, Orcut l, Co.,

static int last poption 0;

static menulteatype pics Zeenu ( ) - ti, O, Full sland, l, O, reduced to to

static menultantype textiomenu ( ) - l, Ooad text screen", lo, "Save current screen l, 0, out, l, Oed

static nanui teletype test menu a lo, "Perform test with a set of pictures or text,

156

5,204,703 157 158

l, O, "create a Picture test batch file", 1, 0, "Edit Batch file,

23, "Execute a picture test Batch file", tl, "Configure test environment", il 7, "define Important areas of image", l, O, "Quit" , 0, ow

static int last troption - 0:

static Renuitantype configurhenut - , 10, "Number of Calibration Points",

( ), 1), "Look-Point Filter Window size". l, 7, "Active Box lookpoint feednack", , 0, "No Box feedback"),

ti, O, "Total lookpoints allowed", l, O, "Look-points required to select a box", l, 0, "Active boxes status"), l, O, "Quit, l, O, Eoi", ;

static Renuite atype box menu ) - { tl, O, "l (V334 )"), l, 0.2 t \334 )"),

{ 1, 0, 3 ( \334)"), l, O, Out", l, Oreo ;

static int lasta option - 0:

?tatic Renu iteratype calibpts aenu () in O, ),

l, O, 5, own,

l, 0, oil, :

static nenuite atype on off aenu ) - { , o, on"),

l, loff), O, on

/ Mod for two images instead of four ; 29 MAR 90 Greg Schubert (and the program is writtan for the top LEFT corner) f

f" screen locations for placing age on a graphics acreen two at a tina. Point p represent the (x,y) locations of the upper right hand corner of aach maga. WILL WANT TO CHANGE. This then UMBER OF boxes ILL at WAt ABLE. PROBALY READ IT N or solo FINE IT DYNAICALLY.

int in age loc{4) - O, O, 376, Ol

:

int textbox.loc. 3) {4} - { 9, 7, 24, 23, 33, 7, 4, 23, is 7, 7, 72,23, l;

A ND nod to add time stanping a? float start time: static void showmen (void)

( xycprint? (60,2 WHITE, "core - u, coraleft());

f Routine that displays a message to the user s/

void status (char st lint xp: struct text info ti: get textinfo (ti) : xp - 79 - strlen (st; if {xp > 5)

xycprint f(xp, 24, LIGHTCYAN, is", st: text attr it. attribute):

void remove status (void) cput char ( ), 24, Blue, Blue, 66, );

void boundlines (void

yeprintf(, , cyan, a - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n: - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - ycprintf(1,23, cyan, a - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - n:

5,204,703 159

fe e is color Blues palette Used as a "grey scale

for image presentation. af

vold intcolor (void { get palette (spalette);

palette.colipriso) or 0: palette. coldrill r ; palette. colors 2 - l; palette. colors (3) - 33: palette. colors (4) 7; palette. colors (5) - 49: palette. colors 6 - 9: palette. colors 7 - 4 l; palette. colors) - 25; palette. colors 9 a 57; palette. colors 10 ll: palette. colors l l r 43; w palette.colors (2) - 27; palette. colors 3 - 59: paletta. colors 4 - 3 l; palette. colors (5) or 63; palette. size - 5: setall palette (spalette):

/ Routine to place a small icon at the screen location xloc, yl oc. af

vold icon (int x loc, inty loc) set color to: circle {x loc,ylocs): circle (xlocyoc, 4) ; circle {x loc,yoc,3); circle xocyloc, 2) ; etcolor (S);

circle (xlocyloc, l;

/ AUTHOR: Kevin S. Spetz a DATE:

routine to draw an image held in the frame buffer on the graphic screen. The paranetar size indicate a reduced or full size inage, and the parameter wait tells drawinage whether or not to wait for a key to be pressed before returning

af

/ Modified 27 AR 90 Greg Schubert /

void drawi naga (int size, int wait, int boxx, int boxy) /ND nod int x offset, maxx; /

int y offet, x tart, ystart, x, stop, ystop, x, y, maxy: unsigned char near buf: int countereo; int max counter-) 000: int y counter-0;

lf (but - (unsigned char near ) realloc (512 sizeof (unsigned char) as NULL crecr () ; fprint f (stderr, "Not enough memory for image buffer -- Aborting." : exit (0);

new set upgraphics EGA, EGAHE): in it colors ();

Wild pod naxx a getnaxx (); / naxy or getnaxy () : if (size a FULSI2E

WND iod xoffset s (naxx - 52/2; A yoffset - axy - 5272; xstart - ystart - 0: x stop - siz: y stop - yuage stop: y counter-ystart: nax counter-4;

else if size ALFSI2E)

WNo add xoffset-taxx-S12 /2: / yoffsets (aaxy-52)/2: x start-box_x: Ystart-boxy: xotoprxstartt 210; ystopy start+400; anx counter; y countary start:

else ND aod offset s 235; / yoffset - 95: x start a ystart as 0;

We nod a xstop a 256: add a ystop - 239:

y counterey start:

if yoffset C 0) yoffset - 0:

160

5,204,703 161

for y y start: y Cystop; y ++)

countert t;

if (counter i-nax countar) get rowawl ty, 0, 52, buf); for (x or x start: K C x stop;" x++)

Putpixel (x-x start, y counter-ystart, int) ( (buft x) (palette, size t lb A256. O));

y countertti

else

countern

lf wat) getch () : close graph ti

free buf);

/ AUTHOR: Kevin S. spatz AT:

Routine to load an image from disk to the EGA screen s/

f aodified 27 MAR 90 Greg Schubert / void loadiraage tiaagelist type A, int wait)

wod buffer; char buffer 294; unsigned size; it han; picture descr; int count; marker type Rarker; rt k; wn, ; signed offset, offs at 2; int change; long int tapint; unsigned tapnun, nurabytes; f

if (valid file (1->image O. name, TxTMARK) ) set cursor (NOCURSOR); textbackground (BACK); circr (); if (han r open (i-Xinage (O. name, o RDONLY) ) :- -1 ) (

read than, Smarker, sizeof (marker type)); if (buffer as allocOt2 P2) a NULL) {

fprint f (stderr, "Mardry Allocation Error -- idad image \n"; exit (0):

read than, buffer, 0.2) 2): puttax t ( , 3,80,23, buffer); ree buffer) ;

close (han): if wait)

getch (): ciscr) : Otn;

eise return:

P

also fprint f stderr, "error File won't open -- loadinage"): exit (1):

new setupgraphics (EGA, EGAhI); astvisualpage ()); intcolors : for (count a 0; count C -> size; count + i) {

if (han - open (i->inage count). nano, o RDONLY) } : - -1) read than, & descr, sizeof (picture));

if descr. size a FULLSIEE) size image size (0, 0, 639, l l 7) : if (buffer - malloc ( size) are NULL)

closegraph (); fprintfestderr, Error! Memory Allocation -- Load image large)

u", corelleft (), le; Oxit (l) :

read (han, buffer, siza); put image (0, 0, buffer, COPY_PUT): read (han, buffer, size):

putinage (0,118, buffer, coPY PuT): free buffer): size a nage size (0,236,639,349; if ( (buffer as alloc size) as NULL) {

closegraph ();

162

ses cornft in hu needed

fprintf(stderr, "Error! Meraory Allocation -- load image large) -- core art - Yu needed

5,204,703 163 164

u", core left (), size): exit (l);

read than, buffer, size): putinage (0,236, buffer, coPY Put); close than ); free buffer);

A nod elsa size inage size 0, 0,256,239); if buffer naloc (size) a NULL)

close graph (); fprint f (stderr, "Error! Memory A location -- load image snali) -- core left - u", cut eft : exit l;

read than, buffer, size): put inage (image loci count 2, in age loccount 2 + 1), buffer, coPY. PUT: close than); free buffer;

a/

a nod of else f printf(" (left) upper left x : *): fiscan f (stdin, "d", ulx); printf(" (top) upper left y: "): facanf (stdin,"d Guy): print f { botton botton right x : "): fiscan? (stdin,"d", brx): print? (right) botton right y: "): fiscan f (stdin,"d, bry): size imageSize ( ulx uy, brx, bry):

printf("Enter off sat from beg. of buffer2: "): can f stdin, "Wid, soffset);

print f { Enter int to add to buf2: "); scan f (stdin, "Wid", change);

print? ( enter bytes to read: ); scan f (stdin, "Wu, Gnuabytes) :

D/

read (han, buffer 2, 1984); / tapchri a vo: print? (buffer2 loc p", buffer2) : for so: Cl2; ++) {

macpy (tapchir, (char ) buffer2+3), l) : tapint (int) tipchir to: if (in tapint, 0) go tapint - 256ttapint: printf("vintapchir - as, buf2, byte td contains Ald - also: d", tapchr, 3, tapint, peakbt.

DS, (buffer2t)));

getch (); / nenset buffer2+2, 14, l; put image (Anagaloc count 2), inage loc count 2 t 1), buffer2, coPYPUT): for k a s kC240; as

read than, buffer2t 4, 1980): if a if k <76

printf("k - d", i.); for (3-0: Cl; ++)

nancpy (tapchir, (char buffer2*), l: tapint or (int) tapchiro): if tain (tapint, O.) CO) tapint 256+tapint: printf("\ntmpchir - As, buf2, byte d contains Ald - also: hd", tapchr, , tapint, p

eakb (DS, buffer2t1) )) :

ef putinage (inagaloc count 2), inage loc (count 2 t itk, buffer2, COPYPUT):

close (han);

alse out textxy (inagaloc count 2 t 5, image loc count 2 + 1 + 5, "Error!"); outtextxy (inage loccount 2) t 5, image loccount 2 t l! t 5, "No Picture fill found");

setwisual page 0}; if wait)

getch(); close graph ();

f AUTHOR: Kevin S. Spetz tr dAz:

Routine to save an image from the screen to disk. s/

f modified 27 MAR 90 Greg Schubert wod saverage (void)

wod buffer; window type tbuf; rt han:

unsigned size: char ch: picture descr;

5,204,703 165

touf an open window 10, 5,73, 19, WHITE, CYAN, )," "): textcolor (WHITE): xycprint f {2, 3, LIGHTCYAN, "Please enter filename to save the image under: *): get string toutnane) : strupr (outname); sake filename outname, PICTURE): xycprint f (4, 6, LIGHTCYAN, enter a description of the picture : "); xycprint f (4, 7, YELLO, -> "): get string (descr. name); if (han open (outname, o WRONLY O CREAT, SI READ i s WRITE) ) :- - )

sat cursor (SHORT CURSOR): xycprint f {4, 10, LIGHTCYAN. "sava the Entire image or the Reduced one Ear 7 : while ( t (ch - toupper getch () )) is 'E' ) is ich in "R") ; set cursor (No cursor close window (tbuf) : if (ch - E' )

strcpy (descr.uarker, PICMARK): descr, size a FULLSI2; write than, descr, size of (picture) :

drawinage FULLSI2E, NOHAIT, 0, 0): size n inage size (O, O, 639, 117): if (buffer a malloc (size) as NULL)

closegraph (): fprint f (stderr, "Error! Neory Allocation -- Save inage Vn"); exit (l) ;

getinage (0, 0, 639, 7, buffer);, Write (han, buffer, size): getiuage (0, ), 639,235, buffer); Writa (han, buffer, size );

free (buffer); size inage size (0,236,639,349); if (buffer as alloc (size) as NULL)

close graph (); fprint? (stderr, Error: Menory Allocation -- Save inagevin'"); exit 1);

get age (0.236,639,349, buffer): Write than, buffer, aize); free buffer: close (han): closegraph ():

else descrize REDUCED; stricpy descr. aarker, PICMARK): Write than, 6 descr, sizeof (picture)); drawinage (REDUCED, NOHAIT, 0, 0):

nod a size or image size (0, 0,256,239): if ( (buffer ... aalloc (size) NULL)

close graph (); fprintf(stderr, Error: Memory Allocation -- save inage\n"); exit (l);

a nod / getinage (0, 0,256,239, buffer); Write (han, buffer, size): frae buffer): close than); costagraph();

166

als a close window (tbuf} : xycprint f(, 25, RED, "Can't open file s -- terminating program with error dwin, outnana, arr no

exit (l) ;

char text screenio (char filenarie window type A touf int lo an 0; lint tag, han void buffer; unsigned done; char finane MAXPATH: char temple): markertype marker:

to a open window (5, 5,30, 11, LIGHTCYAN, BLUE, 0, "Fila I/O); switch (menu select (42, taxitionenu, lo)) {

case "l.' don a 0. while (dohl)

select file c: V\saccade\\textVV.txt" finane, tag, 55, 51.4): if (valid file (aka filenane (fmann, STEXT), TxTMAK) )

extract fr (tamp, finane); streat (teap," is not a valid"): error teap, "text file.");

else donel a l;

close window (thufl) : if than an open (fname, o RDONLY)) or -l)

read (han, Smarker, sizeof (markertype));

5,204,703 167

if buffer (void ) alloc (80212) a NULL fprint? (stderr, "Error hemory Allocation -- save inage. Vn') : exit (-):

read than, buffer, 80212); put text (, 3, 0, 23, buffer): frea buffer): close (han):

else fprint f (stderr, "Error couldn't open file A. Vr", frname: exit (- );

break; case 'S'

close window (tbufl); if buffer on naioc BO 22) or NULL)

get text (), 30, 23, buffer; tbufi - open window (12, 9, 6, 15, LIGHTCYAN, cyAN, ),"); taxtcolor (WHITE); gotoxy 2, 3} : if (getnaw filenane (fnane, "Enter the file to save the Text in: , STEXT ) {

han - opent frane, o RONLY to CREAT, SIREAD SIWRITE) : atricpy (parker, TxTMARK): Write (han, Guarker, sizeof (marker type)); Write than, buffer, 802.2): cios (har): fra (buffer) ;

else free (buffer; close window (tbuf1) : nes age ("Successfully save thatext screen");

alse xycprintf(l, 25, RED, Not Enough Memory to get screen"); exit (l);

break: case 'O'

close window ( t bufl.); strcpy (finana, " : break;

extract.fm file name, finane); return filename:

a AuTHOR: Kevin S. Spet 2 ATE.

Rout he which allows a user to enter text on to the screen, then allows the screen to be saved to disk.

a / void savetext (void

int x, y, color - CYAN; unsigned done; char terpil 6; char ch:

textbackground (BLACK): set curb or (SHORT_CURSOR): criscr (); textcolor cAN); cputchar (l, 2, CYAN BLACK, 80,' " : cputchari, 24, CYAN, Black, 80,' ' ): xycprint (, l, LIGHTRED, *Text screen"); xycprintf(30, 1, RED, "File: "): xycprint f(1, 25, GREEN, F : File I/O F2: clear/Restart"); xycprint f 36,25, CYAN, "F3: change color"): xycprintf(5S, 25, GREEN, F4 : Add Boxes"): xycprint? (7), 25, WHITE, "Esc: Quit"); x - : y at 3: done - 6: while (done) {

xycprintf(67, l, WHITE, "Row: 2d Col: Y2d", y - 2, x); gotoxy (y); switch (ch a getch () )

case O : switch (ch as gatch ()) { case F. : xycprintf(36, , YELLOW, "bl2s", texts creenio (terp} );

break; case F2 : window (l, 3, 0,23);

criscr (); window, l, 0,25); x - 1: y - 3: break:

cage F3 if (++ color > 5) color - ;

textcolor (color); xycprintf(36,25, color, "F3: break:

cale F4 : set box line (SINGLE) : switch choose ("Draw which box?", "1, 2, or 3", boxenu)) {

case draw a box textbox-loc ( 0 (0), textbox locf O))),

text box.loci O (2), textbox.loc.0) (3,

change Color"):

color, BLACK, 0; break;

case 2" draw a box (textbox.locf. ) (0), textbox.locil) 1,

168

5,204,703 169 170

textbox-loc{ll 21, textbox.loci) 3, color, BLACK, 0);

break; case '3'

draw a box textbox.loc. 2 (0), text box.loc. 2) ( ) 1, textbox-loc (2) (2), textbox local {3}, color CKO);

break;

set box line (Double): break;

case Uparrow : y --: bind t&y, 3,23): break:

case on Arrow : y : t: bind sy, 3,23; break;

case farrow : x- -: bind (&x, 1, 0); break;

case Rt Arrow : x + i ; bind (x, 0): break:

break: case BCKSP : x- -

bind ( 4 x, , 80): got oxy (x, y): putch ( ' ' ); break:

case RET : x s ); y++; bind (sy, 3,23); break:

case ESC done - l; break;

default : putch (ch): xt +; bindix, 0); break;

set cursor (NOCURSOR);

7 AUTHOR; Kevin S. Spetz DATE: 7/7/9 Routine which looks in the imagelist and build a box lit for use by the

a sacboxes routines. Uses the Active flag to decide how to build the list. for image files active is l or 0; dupending on the value the image position

added to or not added to the box list. For Text screen the valua is 0 - 7, where the first bit is the first allows box on the text screen, the second bit is the second, and the third the third. In the future this aethod ahould be inproved.

s/ activeboxtype build abil (image littype 'i)

activebox type b NULL; int count; if (valid file (i-Xinage to.naas, TxTMARK)

lf ->inaged) active 0x0l.) b is start activebox (l, textbox.loci O (0), text box.loc.0) (l),

textbox.loc.0) (2) textbox loco (3)); if i-Xinage O. active & Ox02)

if b am NULL) b or start activebox (2, text box loctl ) (O), text box local ill,

textbox octl (2), text box loc{l 3); oise add activebox (b. 2, textbox.loc. 1) tol, textbox.loc. 1) (l),

textbox.loc. 1) 2, textbox.loc ill (3}); if i-Xiange O. active & 0x04)

if b m, NULL) b a start activebox (3, text box.loc. (2) (0), textbox loc{2} ),

textbox.loc (2) 21, textbox loc{2)(3): alse add activebox (b, 2, textbox loc{2 (0), textbox loc{2) ( ),

textbox loc{2) {2), textbox.loc. 2)(3));

for count a 0; count C i-> size + 1 countt t) if i-diaage count). active) {

if b ap NULL) fad nod / b - start activebox (counttl, image-loc (count2), image-loc count2t image-loc (count 2+256, image loccount"2t 1 +239); W and nod A else add activebox b, counttl, inagaloc count 2), image loccount"2+ll, inagaloccount 2 +

256, image occount2til 239);

return to);

AUTHOR: Kevin S. Spetz OAT: Routine which displays the images (picture or text) held in and then nonitors the subject's pupil diameter and look-point as the stimuli is displayed. The test will cease after a number of look-points, or when the space bar is pressed. If any other key is pressed that key

5,204,703 171 172

will be stored in the data file as a label. a /

void perform test image ist type I) nt han:

char marker 4: it box;

act we box type arol - Null: unsigned tip;

if total lp -- 0) tp - 32000: At trying to grab 0 lookpoints means

just Walt for a key, ie rake tip big else tp - total pi stricpy marker, RESHARK); if than - open ( -> results, o WRONLY o TRUNC O CREAT, SI READ ISI WRITE) : : - - ) {

Write (han, & marker, sizeof (marker) ) : Write (han, 6 subject info, sizeof subject type} ): Write (han, l, sizeof (imagelist type)); abl build-abl (i); getbox tabl, box, han, tp, useboxes, start time : dispose activeboxlist (abl); close (han);

1 routine to show the main screen. af t

void show main (char ra) patcursor (No CURSOR); draw a box (), l, DO, 25, CYAN, BLUE, 0): boundlines; fastwrite (3, 24, LIGHT GREEN, Blue, "Messages:"); fastwrite 3, 2, LIGHTRED, BLUE, a); fastwrite 10, 7, YELlo, BLUE, "Please Make a selection"); pho When

/ AUTHOR: Kevin S. Spetz DATE: Routine which asks the user to select a batch file and then executes St.

a/ void batch test (void)

int han, done, x, y, duanyint: Window type buf; char outname 12; imagelist type image list; arker type aarker;

lint tag:

/* its May have to do something hare for grapics error / setracalibrate ( ); continuous ); textcolor (ITE); done m. O; while (idone

if (select file "c: V\saccade\Vbatch v\.bt.c", outnaae, Stag, 8, 7, 4, 14)) { if (valid file (nakefile nane (outnana, BATCH), BTCHARK) )

textbackground (BLACK); cirs cr); buf a open window 25, 10, 55, 4, CYAN, BLACK, 1, ""); xycprintf(32. HITE, "Press any Key to begin..."); getch(); close window buf): get gazeposition (x, y, duanyint, dunny int): han - open outnase, ORDONLY): read than, imarker, size of (marker type));

/ Nd aoda/ start time - timex ( ); while (read (han, Áinagelist, sizeof (imagells ttype) ) -a sizeof (image list type)

load image Giaagalist, 0): perforatest (disage list); if (screen node () - TEXT

close graph ();

dore a l; close (han):

else buf - open window { } 0, 12, 70, 14, WHITE, RED, , "): xycprint f(2, , WHTE," is is not a valid batch fle", outname): delay (2000); close window (buf);

se done a l;

/ AUTHOR: Kevin s. Spetz DA: Routine which allows the user to select a batch file and then edit the result a file list it holds. Allows the batch file to be updated to have new, unique names to store data in during a test.

5,204,703 173

void edit batch file (void) int hanhan2 done, count; window type buf, but 2: char outname 6d); char d4) "; char tapnaaet 32: char tampfile (64: inagali at type inagelist; int tag: marker type aarker:

textcolor h; done one of but - open window (15, 4, 65, 12, Black, LIGHTGRAY.), "Batch file edit) : xycprintf 5,3, LACK, "Filename: s xycprintf (5,4, BLACK," Seed xycprintf(5, 5, BLACK," Ranga: ; who done

if (selectfile ("c: VV saccade\\batchVV.btc., outnane, Stag, S5, 31, 19)) { if valid fille (nakefile name outnaas, AATCHTCHARK)

xycprintf 153, HITE, n) xycprintf(15, 3, WHITE, s, outname): xycprintf(ls, 4 BLACK, n: buf2 as open window (28, 15, 53, 9, HITE, cyan, 1.): xycprintf(2, 2, LIGHTCYA, Enter Seed: ); textcolor thre: textbackground CYAN); if (edits tring seed, "", 5)

close window (buf2): xycprint fils, 4, whITE, is , goad); tapnaya (tenpfile) ; han as open (outnane, ORDONLY); han2 is open (tampfille, o WRONLY to CREAT, s_i READ is IWRITE): read than, Guarker, sizeof (marker type) }; Write (han 2, narker, sizeof (aarkar type) }; count at (); xycprint fils, 5, WHITE, Vs OOO. rea, seed); xycprintf(2, 5, Alack, to):

174

while tread than, imagelist, sizeof (image list type) ) - sizeof imagelist type)) { printf(tupnana,03d", saad, count); xycprintf(31.5, HIT2, "Ys... res", tapname): stricpy (inagelist. results, aakefilenane (tapname, RESULTS)); Write than 2, image list, size of image list type)): court ht

close than); close than2); renowe outnane); ranane (tempfille, outnana); remove (tenpfille); xycprint fl3, 7, Black t BLINK, "Press any key to continue"): getch(); done s l;

else close window buf2); done m l;

else buf2 - open window 10, 12, 70, h4, WHITE, RED, 1, ""); xycprint f(2, , white," is not a valid batch file", outname); delay 2000); close window (buf2):

else done as ;

close window (bu?);

AUTHOR: Kevin S. Spetz f dATE: a Routine which allows the user to select a stimuli then aonitors the a subject as the timuli is displayed. The test length is lated to on screen.

a/ void single test (void)

window type buf: int x, y, dunynt; imageliottype imagelist:

f a a a to one an is a Kay have to do something about calibration here" / patrocalibrata t) : continuous; if (gatinagalist tainagelist,

textbackground (BLACK); circr); buf - open window (25, 10, 55, 4, CYAN, BLACK, o,"; xycprintf 3, 2, WHITE, "Press any Key to begin..."); getch(); close window (buf); gatgazeposition (6 x, y, dunyint, 6dunny int):

find nod start time timex () : loadinage & inagelist, 0); perforatest (Sinagelist):

if (screen node () :- TEXT)

5,204,703 175 176

close graph ();

void test configure (void) window type buf, int done or 0. char ch:

buf2.

nonucolor type trap;

gataenucolors (stap): setanucolors (Sgray enu) : bur - open window (3,3,54, 14, Black, LIGHT GRAY, l, "Configure: "): while (done)

xycprint? (45, 2, WHITE, ( Add ) ", calibpts); xycprintf 45, 3, WHITE, 4d)", filterpts); if (box) pion) xycprint f (45, 4, WHITE," ( On)"): else xycprint f (45, 4, WHITE, "t off)"): if nobox pon) xycprintf(45.5, HITE, ( On)"); alse xycprint f {45, 5, WHITE," ( off"); xycprintf(45, 6, WHITE, (Vdd ) ", total lp): xycprintf(45, 7, WHITE, " (Y4d), nun box lp): if usaboxes xycprintf 45 s, WHITE, on)"); as a xycprint f (45, O, WHITE, "t off"); ch as aanu bact (s.2, configmenu. Slast caloption): switch (ch)

case 'C'

case 'F'

case 8 menu)) {

CA

*, on off menu) )

ca

Ca

Ca

fif

f

fA

of

: switch (choose ("How Many calibration Points", "to use", calibpt a renu) case "4" : calibpts 4; break; case "s" : calibpts a 5: break; case "B" : calibpts - ; break:

nitaze (BO, 25, filterpts, calibpts : or eak

; but 2 - open window () 5, 14, 75, I, Li GHTCYAN, CYAN, 1, "input"): do

filterpts - get num 3, 2, WHITE, "How many points to use in running Filater " : if filterpts > 8) (f) terpts c 0) } {

error ("Acceptable numbers of Points to Filter", "over Range from 1 to ". circr) ;

while ( filterpts X is filterpts < 0): close window (buf2): initgaze (0, 25, filterpts, calibpts): break; switch choose ("Should Look-point feedback in active", "boxes be on or off?", on off

case 'o' : box lp on l; break: case 'F' ; box.lp on - 0; break:

set box mark (box.lp on); break: switch (choose ("should look-point feedback outside", "of active boxes be on or off?

capa 'o' : nobox.lp on l; break; case 'F' : nobox.lp on O: break;

set noboxnark (nobox.lp on); break;

: buf2 open window (12,10, 6, 15, LIGHTCYAN, CYAN, l, "Input"); xycprintf(4, 4, LIGHTCYA, Enter O to operate until a key is pressed."): total lp a getnu (2,2, HITE, how anny look-points to use during tests? ): close window (buf2): break; buf2 open Window (12, 10, 6, 15, LIGHTCYAN, CYAN, l, "Input); xycprintf(4, 4, LIGHTCYAN, Turn Active boxes off to disable this option.} : nun boxlip - gatnun (2,2, HITE, "How any look-points activate an active box? ); set gaze speed (nunbox.lp): closa window (bu2); braak:

switch (choose ("Should the active boxes be", "on or off?, on off enu)) { case 'O' useboxes to break; case " : taboxes a break;

break; done - l; break;

close window (buf); setaenu colors (Stap):

void draw half box (char right, char left, char top, char botton,

int :

int x, inty, int width, int height)

unsigned char tanps OOl:

for iO: Chaight; it t)

left in plc (x, ity, aw ()); rightiinplc (xt width, ify, avw ()); outpic (x, ity, (unsigned char) 200, avy () };

5,204,703 177 178

outpic (xt width, ty, unsigned char) 200, avid () };

getroWavw (y,x, width, top): getroWav W (y theight x, width, bottoe): neas at teap, 200, width); putrowavw ty, x, width, tenp} : putrowawwytheight, x, Width, temp):

void clear half box (char right, char left, char top, char botton, int x, inty, int width, int height)

int i:

put rowavu? (y,x, width, top): put roWav Wythaght, x, Width, botton); for t i-O: Cheight; it

outp c (x, ty, unsigned char) lefti, avul () }; outpic txt Width, ty, unsigned char) righti, avi ());

void position half box tint x coord, int “y coord) define uparrow 72 dafin DnArrow 0 define farrow 75 define Rt Arrow 77 define RTURN 3

int x: inty: char C. char right buffer (500): char left buffer 500); char top buffei (300); char bot buffar (300); int dones-0; Window type thuf; int height-400; int width 240;

thu? - open window (24, 15, 57, 18, WHITE, cyAN, 1," "): textcolor (HIT): xycprintf(2, l, EGA YELLON,"Use arrow key to position box"): xycprintf(2, 2, EGA YELLow, "Press return when complete..."); x a y - 40: draw half-box (left buffer, right buffer, top buffer, bot buffer, x, y, width, height): while (done)

if ( (c. - toupper (getch () ) -- RETURN) dones: eise if c-0)

c-toupper getch () }; if cr-Up Arrow} cm-DnArrow) is can R tarrow (calf Arrow )

clear half box (left buffer, right buffer, top buffer, bot buffer, x, y, width, height); if (coupArrow y-es: else if (cm-Dnarrow) ytes; else if (camatarrow x+10: else if cer. Arrow) x-0;

if t ( xt width) > Xinage stop) x-x_iuage stop-width; if (ty the light) > y image stop) y-Ylanages top-height; lf ( x < xixaga start) x-x_image start: if (y C. yuage start) Y-Yinage start:

draw half-box (left buffer, right buffer, top buffer, bot-buffer, x, y, width, height):

clear half box (left buffer, right buffer, top buffer, bot buffer, x, y, width, height); x_coord - x: tycoord - y:

close window (thuf):

f ND nod void savehalf image int x, inty) i? void savahal finage (vold)

void buffer; find aod window type tbuf: char ch; A int han: unsigned size: picture descr:

textcolor WHTE : xycprintf(2, 6, LIGHTCYAN, "Enter filename for image: "): get string outname); if (stren outname) as-0)

5,204,703 179 180

t ND nod close window ( t buf); "l

return

strupr (outnane): make fanaue outname, PICTURE) xycprintf 2, 17, LIGHTCYAN." Enter description: "): get string (descr.naaa) :

if than - open (outnana, o WRONLY I O CREAT, SIREAD 1 SIRITE}) : ) etcursor (SHORT CURSOR):

stricpy descr. marker, PICMARK : descrite a halfSI2; write than, descr, sizeof (picture) };

fe drawinage (HAFSIZE, NOAIT, x, y); / size as image size (0, 0,240, l00): if (buffer a maloc (size) m UL)

clobe graph (); fprint? (stderr, Error! Memory Allocation -- Save age\n"); getch (); fflush (stdin); exit (i);

get image (0, 0,240, 00, buffer) ; write than, buffer, size): getiaage (0, 0,240,200, buffer) ; write than, buffer, size): getiuage to 20,240,300, buffer): write than, buffer, size): get inage (0, 30, 240, 400, buffer) : write than, buffer, size): a buffer;

close that close graph th:

oise

AD add close window itbur); /

void box and save (void)

int boxx; int boxy:

position half box (& box.x, 6 boxy); drawinage (HALFSI2E, NOWAIT, boxx, boxy):

/ND nod save halfi.uage (boxx, boxy) : 1 save half age (): close graph ():

void pic options (void) char ch: inagelist type inagelist: int done: int tag: char teap. 64):

show main ("Picture Menu"): f entire aenu only active When

frame grabbed. af

menuchoiced Asable (picture menu," D') : menuchoice disable (picture anuri menuchoice disable picture menu, 'R' ): menuchoice disable (picture Menu, 'B' menuchoice disabla (picture menu, 'N' ); menuchoice disable (pictural menu, 'm': menuchoice disable (picturemenu, x' );

ND aod raenuchoice disable (picture menu, "l' ); / continuous (): while ( (ch - menu select 25, 10, picture menu, Siast pin-option) : 'O' switch (ch)

case 'F' : grabfrane (): menuchoice enable (picture menu, 'D' ) : menuchoice enable (picturauenu, 'E' ) : nanuchoice enable (picture menu, 'R' ); Renuchoice enable (picture.nenu, 'B' ): aenuchoice enable (picturemenu, 'N' ); menuchoice enable (picture menu, 'A') : menuchoice enable (picture Renu, 'x' }; break;

case 'C' : continuous ti aenuchoice disable (picture menu, "D"); menuchoice disabla (picture menu, 'E' ); menuchoice disable (pictur enu, "R"): menuchoice disable (picture menu, 'B' ); aenuchoice disable (picture menu, 'N') : menuchoice disabla (pictural menu, 'A') : nanuchoice disable (picture menu,"X") : break:

case "D' : switch (choose ("Draw the Full sized picture", "or the Reduced one?", pics in nonu)) { case "F" : drawinage (FULLSIZE, , 0, 0);

break; W case "R" : drawinage (REDUCED, 1, 0, 0);

break:

show main ("Picture Menu"): break;

5,204,703 181 182

case "E" : equalize image () ; break;

case L. : do done l; if (balect file ("c: V\saccade\\plctures\\ .pic", inagel St. inage (O. name, 6taq, 4S, 4,

27) f (valid file (nakafilenane (image list. in age (O. name, PICTURE), PICMARK) ) axtract fin (temp, inage list. image? Ol. name); stricat (temp, is not a valid"): error (temp, "picture file): done as 0;

alae image list. size - l; loadinaga (iinagelist, l) ;

while done); show main ("Picture Manu"); break:

case 'S' : Savenage (): show main ("Picture Menu"); break;

case 'R' : reduce image (); break;

case 'B' : binary image (70); break;

case 'N' : reverse image (): break

case 'A' ; anooth image (): break: W

casa 'x' ; box and save (); show main ("Picture Menu"); break;

void subject registration (void) window type win;

win - open window (5, 10, 75, 20, WHITE, CYAN, ), "Registration"); xycprintf 2, 2, WHITE, "Subject Name: *); editstring (subject info.name, "", 40); xycprint f {2, 3, HiTE, "Subject Number: "): edit string (subject info. number, "", 0 }; xycprintf(2, 4, WHITE, "Subject info; *; xycprint f {4, 5, WHITE." -> "): edit string (subject info. info, "". 65); close window (wi n);

man () char ch: int han; aarker type marker; inagelist type inagelist : window type buf:

satcursor (No cuRSOR) : if than - open ("saconfig. sc", o RDONLY) } :- - ) {

raad (han, 6 filterpts, senof (int) read (han, scalibpts, sizeof (int) }; read than, 6 box-lp on, sizeof (int) ; read (han, Snoboxlip on, sizeof (int)); read (han, 6 totalp, sizeof (int)); read (han, 6 nun box.lp, sizeof (int) ) : read than, Euseboxes, sizeof (int));

close than);

set box mark ( box-lp on); set noboxnark (nobox-lp on) ; set gaze speed (nus box-lp) :

/ND nod / set boxide f boxes () : initialize (); continuous () : initgaze to, 25, filterpts, calibpts): show main ("Main Menu"); setaenucolors & blue menu) : sat file tagging (0); while ( (ch menuoelect (25, ll, Juain menu, last an option) ) - 'O' } {

switch (ch) case "D" ; pic options ();

break; case "r

case "P" ; show unin ("Tast Menu"): while (tch - nonuselect (25, ll, test menu, slast tin option) ) - 'Q' ) {

switch (ch) case "P" ; single tast () :

show main ("Test menu" : break;

case 'c' ; but - open window (10,1), 70, 15, LIGHTCYAN, CYAN, , "Input: "): gotoxy (3,2); textcolor (HITE); if tetnew fillename outnane,"Znter the filianana to use for the Ba

183 tch file; rearch

case :

case :

cas O

fad acdf case

dreak

5,204,703 184

close window (buf); han r open (outnana, o RONLY o CREAT, SIREAD is WRITE) : stricpy marker, BTCARK); write than, Sarker, size of narker type)); while getinagelist (Sinagalist, 1)) {

Write (han, Sinagalist, sizeof (image list type));

close than);

else close window (buf); break: edit batchfille () ; break; batch test (); show maint"rest menu); break; test configure (); break: getinagelist silagelist, c): fselect pages for working with the

screen, not to use them for a test

load rage (Gina geist, 0} : fo tell a load image not to erase the image screen if

define image boxes t) : show main "Picture Menu": or eak:

cast. 'A' : analysis options (); eak

case "R" : subject registration (); break:

show main ("Main Menu"):

set cursor (SHORT CURSOR): textbackground BLACK); textcolor CYAN : cl cr;

fND nod / dispose active boxlist ( box de f boxes): if than a opensaconfig. sc", o WRONLY O CREAT, SREAD is WRITE) :- - )

write than, 6 filterpts, sizeof (int) write (han, calibpts, sizeof (int) ) write (han, box.lpon, size of int) ) write than, nobox.lp on, sizeof (int)); Write (han, 6 totallip, sizeof (int) ); Write than, fun box-lp, sizeof (int) write than, Gueboxes, sizeof tint) ); close than);

W2

sacresul.c include (stdio.h> include Catdlib.h> include gotonio.d include gent.h> include (string.h> include Cath.h> include (dos.h> include gctype.h> include Cohd include gallochd include (graphics.h> include "c: Wercalib.wtools.h include c. vericalibvgrtools.h include "c: Varicalibvaenu.h include Vaaryland\sacboxes.h" include "c: Vaaryland v saccade.h include "c: varyland Viagebox.ht

static enuitotype analysis menu () - O, Information Summary of results

lo, List raults on acrean, l, set file specification,

file,

l, O'Superpose lookpoints over ages, l, C. Graph results l, 4, "Get New results file for analy 2, Show result based on a box

l, o, out, , o, ed.,

static int last an option as O:

static menuite atype supenu () a

sis, ille,

tl, O, "tait for key between look points,

5,204,703 185

1, 0, continuous look point display, 1, 0, "Abort",

0, oil :

static menu itentype box menu a l, Oload in a new box file"), l, O, Text sunary of box results", l, O, "Graph sunuary of box results, l, Oouit"), 1, 0,204), ;

lagelist type images: int nunrecs; int boxtlp: unsigned box hit (MAx_boxES+) ; double box Jupd(MAx Box2s +1), box tpd: double box sunpd 2 (MAxeoxEst1), box tpd 2:

textern int inage loc); static subject type subjectinfo:

boxdeftype ing boxes (Ax Boxes); /* Used for user dafined box locations f

/* decides which quadrant a lookpoint is in. Returns quadrant 1-4 or O if not in quadrant. Note: The quadrants are numbers froa O - 3 in the structure Ragelist type.

tf w" static unsigned which quad tint x, inty

x > x 640/0; y - y 350/25; | f { {x > image loc{0} - 10) & ( ( x C inage locz) - 10

if Y P image-loci ) - 10) & ( y < image loc(s) - 10) ) return t):

if t t y > image-loc ( S) - 10 (y C image...loc. 5) + 159 - 10 ) return (3) ;

lf ( x > image loczi - 10) 66 ( x < image loc{2} t li t 10) } { if y > isuage hoc - 10) 66 (y C image locs) - 10

return (2) ; if y > image loc(5) - 10) 66 (y C image loc (5) 159 0).)

return (4):

return (O); / / Not in any box f : of

7 Decides which sector a lookpoint is in. Returns quadrant 0, or 2. Note: The halves are numbers from 0 - NUMPICS in the structure imagelist type. This is the testing ground program.

a/ static unsigned whichhalf (int x, inty)

x - x int) 640/0); y a y int).350/25);

/ND nod printf(x - W3d Yg - 43d * x, y); / if t y > 100) & y C 341)) {

if ( x C 257) is x > -l) return (1) :

1 in left box a/ if x > 374) (x c 6) )

return (2); f" in right box. 7

else return (0);

a pe return (O); a not in any box /

1 Decide if a lookpoint is in a user defined box. Returns if it is and 0 if it is not.

a/ static unsigned inhaif (int x, inty, box deftype box)

x as x int) 64.0/10): y - y int) 350/25); if (x >e box.ul. x 6 x C box.br. x 6

y >e box.uly 6 y C. box.br.y) return );

else return); Wf not in the box B/

/ AUTHOR: Kevin S. Spatz DAT; Routine which reads the results file res file and breaks the data down into a unary. Calculates the mean pupil diameter for the whole data file, and also breaks the look-points down as to being on one of the NUHPIcs quadrants. Calculatas the respective nean pupil diameters for each quadrant too.

186