USF Libraries
USF Digital Collections

A P300-based brain-computer interface

MISSING IMAGE

Material Information

Title:
A P300-based brain-computer interface testing an alternative method of communication
Physical Description:
Book
Language:
English
Creator:
Sellers, Eric William
Publisher:
University of South Florida
Place of Publication:
Tampa, Fla.
Publication Date:

Subjects

Subjects / Keywords:
human-machine interface
Locked-In Syndrome
event related potentials
assistive devices
ALS
Dissertations, Academic -- Psychology -- Doctoral -- USF
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )
theses   ( marcgt )
non-fiction   ( marcgt )

Notes

Summary:
ABSTRACT: The current study evaluates the effectiveness of a Brain-Computer Interface (BCI) system that operates by detecting a P300 elicited by one of four randomly presented stimuli (i.e., YES, NO, PASS, END). Two groups of participants were tested. The first group included three ALS patients that varied in degree of disability, but all retained the ability to communicate; the second group included three Non-ALS controls. Each participant participated in ten experimental sessions during a period of approximately 6 weeks. Sessions were conducted either at the participant's home or in the lab. During each run the participant's task was to attend to one stimulus and disregard the other three. Stimuli were presented auditorily, visually, or in both modes. Additionally, on each run, the experimenter would either tell the participant which stimulus to focus on, or ask the participant a question and the participant would focus on the correct "YES/NO" answer to the question.Overall, for each participant, the ERPs elicited by the target stimuli could be discriminated from the non-target stimuli; however, less variability was observed in the Non-ALS group. Comparing across sessions, the within session variability was lower than across session variability. In addition, waveform morphology varied as a function of the presentation mode, but not in a similar pattern for each participant. Offline and simulated online classification algorithms conducted using step-wise discriminant analysis produced results suggesting the potential for online classification performance at levels acceptable for communication. Future investigations will begin to focus on testing online classification performance with real-time feedback, and continuing to examine stimulus properties to determine how to maximize P300 amplitude for individual users.
Thesis:
Thesis (Ph.D.)--University of South Florida, 2004.
Bibliography:
Includes bibliographical references.
System Details:
System requirements: World Wide Web browser and PDF reader.
System Details:
Mode of access: World Wide Web.
Statement of Responsibility:
by Eric W. Sellers.
General Note:
Includes vita.
General Note:
Title from PDF of title page.
General Note:
Document formatted into pages; contains 102 pages.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 001498266
oclc - 57717297
notis - AJU6871
usfldc doi - E14-SFE0000536
usfldc handle - e14.536
System ID:
SFS0025227:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader nam Ka
controlfield tag 001 001498266
003 fts
006 m||||e|||d||||||||
007 cr mnu|||uuuuu
008 041209s2004 flua sbm s000|0 eng d
datafield ind1 8 ind2 024
subfield code a E14-SFE0000536
035
(OCoLC)57717297
9
AJU6871
b SE
SFE0000536
040
FHM
c FHM
090
BF121 (ONLINE)
1 100
Sellers, Eric William.
2 245
A P300-based brain-computer interface
h [electronic resource] :
testing an alternative method of communication /
by Eric W. Sellers.
260
[Tampa, Fla.] :
University of South Florida,
2004.
502
Thesis (Ph.D.)--University of South Florida, 2004.
504
Includes bibliographical references.
500
Includes vita.
516
Text (Electronic thesis) in PDF format.
538
System requirements: World Wide Web browser and PDF reader.
Mode of access: World Wide Web.
Title from PDF of title page.
Document formatted into pages; contains 102 pages.
520
ABSTRACT: The current study evaluates the effectiveness of a Brain-Computer Interface (BCI) system that operates by detecting a P300 elicited by one of four randomly presented stimuli (i.e., YES, NO, PASS, END). Two groups of participants were tested. The first group included three ALS patients that varied in degree of disability, but all retained the ability to communicate; the second group included three Non-ALS controls. Each participant participated in ten experimental sessions during a period of approximately 6 weeks. Sessions were conducted either at the participant's home or in the lab. During each run the participant's task was to attend to one stimulus and disregard the other three. Stimuli were presented auditorily, visually, or in both modes. Additionally, on each run, the experimenter would either tell the participant which stimulus to focus on, or ask the participant a question and the participant would focus on the correct "YES/NO" answer to the question.Overall, for each participant, the ERPs elicited by the target stimuli could be discriminated from the non-target stimuli; however, less variability was observed in the Non-ALS group. Comparing across sessions, the within session variability was lower than across session variability. In addition, waveform morphology varied as a function of the presentation mode, but not in a similar pattern for each participant. Offline and simulated online classification algorithms conducted using step-wise discriminant analysis produced results suggesting the potential for online classification performance at levels acceptable for communication. Future investigations will begin to focus on testing online classification performance with real-time feedback, and continuing to examine stimulus properties to determine how to maximize P300 amplitude for individual users.
590
Adviser: Donchin, Emanuel.
653
human-machine interface.
Locked-In Syndrome.
event related potentials.
assistive devices.
ALS.
690
Dissertations, Academic
z USF
x Psychology
Doctoral.
773
t USF Electronic Theses and Dissertations.
4 0 856
u http://digital.lib.usf.edu/?e14.536



PAGE 1

A P300-Based Brain-Computer Interface: Testing an Alternative Method of Communication By Eric W. Sellers A dissertation submitted in partial fulfillment of the requirements for the degree of Doctor of Philosophy Department of Psychology College of Arts and Sciences University of South Florida Major Professor: Emanuel Donchin, Ph.D. Cynthia Cimino, Ph.D. David Drobes, Ph.D. Mark Goldman, Ph.D. Cathy McEvoy, Ph.D. Douglas Nelson, Ph.D. Date of Approval: November 17, 2004 Keywords: human-machine interface, ALS, assistive devices, event related potentials, locked-in syndrome Copyright 2004, Eric W. Sellers

PAGE 2

Dedication For my mother, Billie Lee, in loving memory

PAGE 3

Acknowledgements I would foremost like to thank my wife Treva, and my son Tristan, for providing me true happiness in life. Without Trevas patience, kindness, and unconditional support I would not have been able to complete this document. In fact, my entire family has always been supportive of my conquests, no matter how outrageous they may seem. I would specifically like to thank my parents, William and Billie, and my aunt and uncle, Joan and Peter. These four people have always encouraged me to pursue my dreams and have provided the means to do so. This work would not have been possible without the help and support of my advisor and committee members: Manny Donchin, Cindy Cimino, David Drobes, Mark Goldman, Cathy McEvoy, and Doug Nelson. Mannys boundless knowledge has been an invaluable resource for me, and has helped guide me every step of the way. Additionally, my entire committee provided many comments and suggestions that helped to design and refine this study. I would also like to thank Mary Evans, my external chair. Mary is a superb mentor who takes time out of an especially busy schedule to attend to insignificant questions about minor details. I am also grateful to all of the members of the Laboratory of Nervous System Disorders at the Wadsworth Center. In particular, Jon Wolpaw, Dennis McFarland, Gerwin Schalk, and Theresa Vaughan, have been especially kind and supportive. The Wadsworth Center provided the hardware and software (BCI 2000) for this project.

PAGE 4

Without their support this project may have taken additional years to complete. The programming help of Dennis McFarland was particularly useful. He has written several versions of data analysis code, and never complained once. I must also thank the participants in this study. They made a substantial contribution of time and effort to complete the required number of experimental sessions.

PAGE 5

Table of Contents List of Figures....................................................................................................................iii Abstract................................................................................................................................v Chapter 1: Introduction........................................................................................................1 P300 Component of the ERP Brief Background........................................................4 Pilot Data.......................................................................................................................9 Relevant Previous Research.........................................................................................14 P300 Habituation...................................................................................................14 Mode of Presentation.............................................................................................18 Task Manipulation.................................................................................................21 Memory load....................................................................................................21 Available attentional resources........................................................................22 Dual task requirements....................................................................................22 Other Issues Relevant to BCI Development................................................................24 Averaging across trials...........................................................................................24 Single trial classification........................................................................................25 Epoch overlap........................................................................................................25 Individual Differences...........................................................................................26 Summary and Goals of the Current Study...................................................................28 Chapter 2: Methods............................................................................................................32 Participants...................................................................................................................32 Data Acquisition & Processing....................................................................................32 Task, Procedure, & Design..........................................................................................33 Chapter 3: Results..............................................................................................................36 Waveform Analysis.....................................................................................................36 SWDA Analysis...........................................................................................................39 Deriving Weights...................................................................................................40 Applying Weights..................................................................................................42 Classification................................................................................................................42 Chapter 4: Discussion........................................................................................................54 Practical Issues.............................................................................................................57 Future Directions.........................................................................................................58 Conclusions..................................................................................................................61 References..........................................................................................................................63 i

PAGE 6

Appendices.........................................................................................................................75 Appendix A: Classification Results.............................................................................76 About the Author...................................................................................................End Page ii

PAGE 7

List of Figures Figure 1: Average waveforms for target (solid line) and non-target (dashed line) Stimuli. ................................................................................................................................5 Figure 2: Electrode montage used for data collection. .......................................................6 Figure 3: 6x6 P3 Speller matrix. .........................................................................................7 Figure 4: Example data from two ALS patients in a standard oddball task. ....................10 Figure 5: P300 amplitude and r 2 for target and non-target trials. .....................................13 Figure 6: Averaged waveforms for a target (YES) and three non-target stimuli. .............14 Figure 7: Oddball experiment data for all six participants. ..............................................34 Figure 8: Non-ALS participants average waveforms for each presentation mode in Session 1 and Session 10. .................................................................................................38 Figure 9: ALS participants average waveforms for each presentation mode in Session 1 and Session 10. .................................................................................................38 Figure 10: Topography for each presentation mode for one Non-ALS and one ALS participant. ................................................................................................................39 Figure 11: Mean accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 1. ....................................................................................44 Figure 12: Mean accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 2. ....................................................................................45 Figure 13: Mean accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 3. ....................................................................................46 Figure 14: Mean accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 1. ............................................................................................47 Figure 15: Mean accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 2. ............................................................................................48 iii

PAGE 8

Figure 16: Mean accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 3. ............................................................................................49 Figure 17: Group x Method interaction. ...........................................................................50 Figure 18: Group x Number of Stimuli interaction. .........................................................51 Figure 19: Session x Number of Stimuli interaction. .......................................................52 Figure 20: Mode of Presentation x Number of Stimuli interaction. .................................53 iv

PAGE 9

A P300-Based Brain-Computer Interface: Testing An Alternative Method of Communication Eric W. Sellers ABSTRACT The current study evaluates the effectiveness of a Brain-Computer Interface (BCI) system that operates by detecting a P300 elicited by one of four randomly presented stimuli (i.e., YES, NO, PASS, END). Two groups of participants were tested. The first group included three ALS patients that varied in degree of disability, but all retained the ability to communicate; the second group included three Non-ALS controls. Each participant participated in ten experimental sessions during a period of approximately 6 weeks. Sessions were conducted either at the participants home or in the lab. During each run the participants task was to attend to one stimulus and disregard the other three. Stimuli were presented auditorily, visually, or in both modes. Additionally, on each run, the experimenter would either tell the participant which stimulus to focus on, or ask the participant a question and the participant would focus on the correct YES/NO answer to the question. Overall, for each participant, the ERPs elicited by the target stimuli could be discriminated from the non-target stimuli; however, less variability was observed in the Non-ALS group. Comparing across sessions, the within session variability was lower than across session variability. In addition, waveform morphology varied as a function of the presentation mode, but not in a similar pattern for each v

PAGE 10

participant. Offline and simulated online classification algorithms conducted using step-wise discriminant analysis produced results suggesting the potential for online classification performance at levels acceptable for communication. Future investigations will begin to focus on testing online classification performance with real-time feedback, and continuing to examine stimulus properties to determine how to maximize P300 amplitude for individual users. vi

PAGE 11

Chapter 1: Introduction Previous research has shown that a P300 based Brain-Computer Interface (BCI) can function as an effective means of communication with able-bodied young adults, and wheelchair-bound healthy adults (Allison & Pineda, 2003; Donchin, Spencer, & Wijesinghe, 2000; Farwell, & Donchin, 1988). The study proposed here is intended to further investigate and test the efficacy of a P300 BCI device. The current study tests a system that presents stimuli in three different modes (auditory, visual, and auditory + visual). We will test differences among two groups of people (ALS patients and Non-ALS participants), and we will also test different methods of deriving classification algorithms to eventually be used in online, real-time classification. To date, there have been no published studies that examine these factors in conjunction with a BCI device. A BCI is a device that allows the user to communicate with the world without utilizing voluntary muscle activity (i.e., using only brain activity). A BCI is not a mind reading device; rather, the primary function is to communicate in a fashion analogous to speaking, writing, or typing. Such a system may prove to be quite useful for a completely paralyzed individual. A person who is fully conscious yet unable to perform any voluntary muscle movement is termed locked-in. A person may be rendered locked-in because of a trauma such as a brainstem stroke, spinal cord injury, or severe head trauma. For example, Jean-Dominique Bauby, a locked-in patient, put it succinctly in his book, The Diving-Bell and the Butterfly: 1

PAGE 12

You survive, but you survive with what is so aptly known as locked-in syndrome. Paralyzed from head to toe, the patient is imprisoned inside his own body, his mind intact, but unable to speak or move. In my case, blinking my left eyelid is my only means of communication. (Bauby, 1997, pp. 12). One of the most common causes of locked-in syndrome is Amyotrophic Lateral Sclerosis (ALS). Currently there are approximately 30,000 people in the United States living with ALS, and there are approximately 5,000 new cases per year (lougehrigsdisease.net). The disease is progressive in the sense that the patients paralysis becomes increasingly severe with time. More often than not the patients ultimately become completely paralyzed, even though their cognitive and emotional facilities remaining relatively intact, they are thus by definition locked-in. They no longer have the use of any muscles, including the muscles that control respiration. With advances in medical technology (e.g., lung ventilators), people are now able, and choose to live in a locked-in state for longer periods of time; thus, a means of communication is of great importance for the patient, their caregivers, and their loved ones. A BCI system may provide people who are locked-in a means of communication because their cognitive abilities are left relatively intact, while their ability to communicate is completely abolished because of paralysis. Currently the locked-in population will benefit the most from BCI systems. There are several reasons for this. First, speed and accuracy of the EEG-based BCI systems is not high and, therefore, the communication they allow is slow. As a consequence, communication systems that rely on any form of muscular control movement are more efficient. Obviously, speech is the fastest means of communication. After speech, typing is the most effective means of 2

PAGE 13

communication, in most situations. Following typing, simple muscle twitches can be used for letter, word, or binary command selection (Kubler, et al., 2001). And finally, with severe neurological disease, BCI methods may be the only remaining possible means of communication and, with the current state of technology, the rate of communication is further reduced from reliable muscle twitch communication (Kubler, et al., 2001). Because BCI communication is at a disadvantage as compared to communication methods that employ muscular control it is important to put a high premium on speed and accuracy. Improved speed and accuracy can be achieved in several different ways; most notably the system must maximize the signal to noise ratio. The current project will examine how a number of different stimulus variables affect the signal to noise ratio by focusing on the speed and accuracy of the BCI. In addition, how these variables affect the morphology of the response-locked event-related potentials (ERPs) will also be examined. The relationship between the experimental stimuli and the ERPs is an important factor in determining how well a BCI can perform with a given individual. This idea will be developed further below. Several different instantiations of electroencephalographic (EEG) BCIs are currently being developed and tested by research groups around the world, and they are being used with locked-in patients. In addition to the P300 BCI, the most notable non-invasive EEG-BCI systems are based on slow cortical potentials (SCPs; Birbaumer et al., 1999, 2000), or mu and beta rhythms (McFarland, Lefkowicz, Wolpaw, 1997; Pfurtscheller, et al., 1996; Wolpaw, Birbaumer, McFarland, Pfurtscheller, Vaughan, 2002; Wolpaw, McFarland, & Vaughan, 2000). Invasive systems that are based on 3

PAGE 14

cortical neuronal action potentials are also being developed (Kennedy & Bakay, 1998; Kennedy, Bakay, Moore, Adams, & Goldwaithe, 2000). In the current study, we will examine how extended use of a P300-based BCI affects the ERP. Second, we shall test the extent to which stimulus presentation mode (auditory, visual, and auditory + visual presentation) influences performance. Third, how the participants task affects the ERP will be investigated. Finally, the performance of three classification algorithms derived from step-wise discriminant analyses (SWDA) will be compared. The SWDA solutions will be derived using three different training data set sizes and the classification performance of each solution will be evaluated. Each of the experimental variables will be discussed in the following sections. The details related to the classification algorithms will be provided in the Methods and Results sections. First, however, before project specifics are discussed, a more detailed description of the P300 component of the ERP and the P300 BCI will be presented. P300 Component of the ERP Brief Background The P300 component of the ERP was discovered almost 40 years ago (Sutton, Braren, Zubin, & John, 1965), and after many years of research the robust nature of the component has been well established. Several comprehensive reviews of the P300 are available (e.g., Donchin, 1981; Pritchard, 1981; Donchin, Karis, Bashore, Coles, and Gratton, 1986; Fabiani, Gratton, Karis, & Donchin, 1987). In the most general terms presence of the P300 component can be ascertained by examining the amplitude, latency, and scalp distribution of a time-locked average of responses (Donchin, Kramer, & Wickens, 1982). The P300 is largest at the parietal electrode sites and is attenuated as the recording sites move to central and frontal locations (Donchin, 1981), Figure 1 shows a 4

PAGE 15

typical stimulus-locked ERP at Fz, Cz, and Pz, respectively (note that positive is plotted downward). Figure 2 shows a montage of the electrode names and locations used in the present study. It is also important to note that the neural substrate, or substrates, that generate the P300 is not assumed. What is assumed, however, is a scalp distribution that is invariant across trials, representing a fixed set of synchronously activated neural generators, and that their geometry is such that they appear as a potential difference to electrodes located on the scalp (Donchin, Ritter, & McCallum, 1978; Wood & Allison, 1981). 0.2 0.4 0.6 0.8 1 -4 -2 0 2 4 0.2 0.4 0.6 0.8 1 -4 -2 0 2 4 0.2 0.4 0.6 0.8 1 -4 -2 0 2 4 Amplitude in VTime (sec)FzCzPz Figure 1: Average waveforms for target (solid line) and non-target (dashed line) stimuli. P300 amplitude decreases as the electrode site moves from anterior (Fz) to posterior (Pz). Some necessary conditions must be met for a given task to elicit a P300. First, a random sequence of stimulus events must be presented. Second, a classification rule that 5

PAGE 16

separates the series of events into two categories must be applied. Third, the task must require using the rule. Fourth, one category of events must be presented infrequently (Donchin, & Coles, 1988). The P3 Speller paradigm, used in previous studies, and the four choice paradigm employed in the current meet these requirements. Fp1 Fp2 F4 F3 Cz P3 P8 P7 C4 T10 Fz Oz C3 T9 P4 Pz Figure 2: Electrode montage used for data collection. An oddball paradigm is often used to elicit a P300 response. Typically, in an oddball paradigm, events from two categories occur at random and the two categories have complementary probabilities. The subjects task is to focus on the stimulus presentation and when the less frequent event occurs a P300 is elicited. In addition, P300 amplitude can be determined by the subjective probability of the stimulus event (Donchin, & Coles, 1988; Donchin, & Isreal, 1980; Duncan-Johnson, & Donchin, 1977, 1981; Squires Petuchowski, Wickens, & Donchin, 1977). 6

PAGE 17

Donchin and colleagues (Donchin, et al., 2000; Farwell, & Donchin, 1988) have previously capitalized on the robust nature of the P300 to implement a BCI device that can communicate letters to a computer the P3 Speller. The P3 Speller presents a 6 x 6 matrix of letters, numbers and symbols to the user. At random, a row or column flashes every 125 ms. The subjects task is to focus on a character to be communicated. The character will flash 16.7% of the time (1 out of 6 columns and 1 out of 6 rows). This constitutes an oddball event that should produce a P300 component. Figure 3 illustrates a column and a row flash. The column and row presented in white correspond to the target character flashes, provided that the attended character is the letter P. In practice, however, the rows and columns would intensify in a random serial sequence. Because the stimulus elicits the P300 response it is an advantageous component to use for a BCI. For comparison, BCIs that use spontaneous EEG signals such as slow cortical potentials or mu rhythms for the control signals may have training periods for as long as 1 year (see Kubler, et al., 2001). In contrast, a P300 is elicited by infrequent stimuli and no training period is required. Donchin, et al. (2000) initially evaluated the P3 Speller offline, using stepwise discriminant analysis (SWDA) or a combination of discrete wavelet transformation (DWT) and SWDA. The SWDA and DWT/SWDA algorithms produce classification coefficients that work equally well, statistically. Once the offline analysis has been completed, the set of generated coefficients can be tested online. Donchin, et al. (2000) demonstrated that such coefficients classify the character correctly in an online mode 56% of the time. Additionally, 92% of the time either the row or column is correctly classified (i.e., the classification is half-correct). Offline analyses further demonstrated (using SWDA/DWT) that the system could perform at a rate of 5 characters per minute 7

PAGE 18

with approximately 90% accuracy. Donchin, et al. (2000) also point out that the communication rate is underestimated given the fact that spelling does not have to be completely accurate to be effective, and that the current version of the system failed to capitalize on sequential dependencies of the English language. In the future, it will be possible to incorporate spell correction and smart speller software. In addition, an icon driven interface could be implemented by presenting rows and columns of icons that flash at random. Such an addition would allow for command, or phrase selection in place of character selection. M N O QR Figure 3: 6x6 P3 Speller matrix. Each row and each column randomly intensify in a serial sequence. In the above example a P300 should be elicited when the fourth column and third row intensify, if the user wishes to communicate the letter P. Pilot research has indicated that the ALS population may benefit most from a system that allows the user to answer yes/no questions rather than compose words (Sellers, Schalk, & Donchin, 2003). For completely or nearly locked-in individuals, 8

PAGE 19

being able to compose text would be an added benefit, but a binary switch may be the most effective means of immediate communication. Sellers, et al. (2003) found that ALS patients elicited responses are more variable, and the patients may have more difficulty using a matrix that includes many items because of non-voluntary eye movements, as well as other factors to be discussed below. As such, the focus of the current project is on a version of the BCI that does not include the P3 speller; rather, the system tested here contains only four choices. Each choice is presented serially, either auditorily, visually (at fixation), or auditorily and visually. Each participant participated in a total of 10 experimental sessions that consisted of twelve 100 trial runs. On each trial the participant selected one of four possible target words (i.e., YES/NO/PASS/END). For example, the experimenter may have told the participant attend to YES on this trial. In this case, the stimulus YES should elicit a P300, while the other three options should not elicit a P300. Each stimulus was presented 25 times, at random, with an ISI of 1400 ms, for a total of 100 trials. The primary reason for focusing on a system that uses only four commands is that the current focus of the project is to help locked-in patients communicate. As the four-choice system is developed, the P3 Speller version of the system will continue to be developed in parallel. However, for practical purposes the immediate focus is to design a system that can be implemented, and used with a patient population, in as timely a fashion as possible. To that extent, the current project hopes to determine the likelihood of providing the locked in community with a BCI system that will allow a user to communicate at an acceptable rate of speed, and with an acceptable rate of accuracy. 9

PAGE 20

Pilot Data Prior to this study we acquired data from four ALS patients and several able-bodied college-aged participants. Most of the data had been obtained using the P3 Speller. It was encouraging that all of the subjects have exhibited a differential response to the target and non-target stimuli, in an oddball task. Figure 4 exhibits the average waveforms elicited by target and non-target stimuli in an oddball task for two ALS patients. Thus, although speller performance may be variable between subjects, the system satisfied the necessary condition for the successful operation of the BCI, namely that the target stimuli elicit an ERP that can be discriminated from the non-target ERP. Nonetheless, several questions that required further investigation emerged during the course of pilot testing. 0.2 0.4 0.6 0.8 1 -2 0 2 4 Amplitude in VTime (sec)A 0 0.2 0.4 0.6 0.8 1 -3 -2 -1 0 1 2 Amplitude in VTime (sec)B Figure 4: Example data from two ALS patients in a standard oddball task. Target item responses are presented as solid lines and non-target responses are presented as dashed lines. Panel A shows electrode Pz for one patient and panel B shows electrode Cz for another patient. A question of primary concern that we have to address is related to habituation or attenuation of the P300. Pilot data show that from session to session (and trial to trial) the signal is slightly different, but preliminary data indicate that habituation may not be a significant problem to overcome. For example, with one ALS patient we have collected 10

PAGE 21

data over 17 sessions and the P300 has remained quite consistent in its temporal properties; however, more detailed analysis is needed to determine the stability of the amplitude and scalp distribution of the response across days. In working with locked-in patients, lack of eye movements can be a substantial problem when using a BCI system that is based on a visual task. The current study will examine three modes of stimulus presentation, an auditory condition, a visual condition, and auditory + visual condition. Eye movements are not required to orient attention (Posner, 1980); however, without eye movements, and no means of communication, we cannot be sure that a locked-in individual can orient attention to the target item. If a P300 is elicited by the target stimulus we can assume that the patient was able to attend to the display. If a P300 is not elicited we have no way of knowing if this is due to the fact that the subject was unable to attend to the display, or that he could not adequately perform the task. With some ALS patients eye movements may be possible, even in a quite advanced stage of the disease, but the latency of the eye movements is quite variable because moving the eyes may be a demanding physical activity. By presenting all visual stimuli in the same location the current study reduces the potential adverse effects of needing to orient attention to a small area of a computer monitor. Another variable that requires investigation is the task that the subject is performing. In the most basic sense, the task is to attend to the item that is to be communicated. In practice, however, the BCI user will be using the system to answer a question by focusing on a given stimulus. As subtle as this difference may sound on the surface, it could potentially change the morphology of the ERP if it affects workload (Gopher & Donchin, 1987), if it increases memory load (Wintink, Segalowitz, & Cudmore, 2001), or if it creates a dual focus of attention (Kramer, Wickens, & Donchin, 11

PAGE 22

1983), i.e., focusing on the answer to the questions, and the actual question. The concern with such a variable is derived from the method used to conduct the majority of the experiments that have been conducted using the P3 Speller. The participant is typically given a sequence of letters and or characters to focus on (usually a word). This mode of presentation can be referred to as copy spelling, and it may not produce the same ERP responses as a mode in which the subject imposes their own volition onto the system. The issue needs to be examined further using a controlled experiment. A final question of interest is related to online classification performance. In the pilot studies, online classification has ranged from approximately 10% correct to >90% correct, depending on the particular subject and session. However, there is a caveat associated with this performance. Performance, in this case, refers to the percentage of correct online classifications. The online classification procedure used with the current online classifications has utilized an area calculation method. Whereas previous studies used by Donchin and colleagues have used methods such as SWDA, DWT, Peak Picking, and Area Calculation. Farwell and Donchin (1988) showed that the different classification algorithm accuracy varied for each subject and as a function of stimulus ISI; although, overall, the SWDA solutions classified at a higher level of accuracy. During pilot data acquisition, the data from a previous session served as a template for the area calculation to be used for classification in the subsequent session. Initially, each electrode in the data set was examined to determine the location that contained the maximum r 2 between the target and non-target intensifications. Once identified, the temporal window, for the specific electrode that maximized the r 2 would be located and the peak area would be weighted in an online classifier. Figure 5 shows an example of a waveform that could be used as a template and a corresponding r 2 value. 12

PAGE 23

It can be seen that the maximal window for the P300 response is from approximately 200 to 300 ms post stimulus presentation. The important point to be made regarding online classification is twofold. First, when the P300 response is large and consistent across space and time, online classification is simplified because of the increase in the signal to noise ratio. Second, online classification is only as good as the classifier being used. Additionally, many types of classification algorithms can produce high levels of correct online classification when the signal to noise ratio is high. In fact, visual inspection of the time-locked average waveforms can show a clear difference between target and non-target trials, as demonstrated in Figure 6. 0 200 400 600 800 -8 -6 -4 -2 0 P300 AmplitudeAmplitude in V 0 200 400 600 800 0 .1 .2 P300 r2Time (ms)Prop r2non-target target AB Figure 5: A) P300 amplitude and B) r2 for target and non-target trials. The target waveform (solid line) represents an average of 30 stimulus presentations, and the non-target waveform represents 150 stimulus presentations. 13

PAGE 24

0.2 0.4 0.6 0.8 1 -6 0 8 Amplitude in VTime (sec)YES NO PASS END Figure 6: Averaged waveforms for a target (YES) and three non-target stimuli. Each waveform represents 25 stimulus presentations. Relevant Previous Research Previous research that is relevant to the concerns identified during pilot data collection will be discussed in more detail in the following three sections. The issue of classification performance will be addressed further in the Methods and Results sections. P300 Habituation One of the important issues in developing a BCI based on the P300 ERP is to determine how, or if, the ERP will change with extended use, as such, participants in the current study completed ten, fifty-minute experimental sessions. Properties related to extended use are critical because if the ERP attenuates or habituates with extended use the system may need to be modified, or in the worst case scenario, such a system may not be feasible. Using the P300 is different from EEG signals such as slow cortical potentials (SCPs) or mu rhythms. In a P300 based system biofeedback information is not present. 14

PAGE 25

In mu and SCP systems the user learns to control the EEG signal to move a cursor in a display (Birbaumer, 1984; Pfurtscheller & Neuper, 1997; Wolpaw, McFarland, & Vaughan, 2000). Typically, the user can either learn to control the signal to some level of proficiency, or not. In contrast, the P300 is not controlled, it is elicited by stimulus properties, as such, if the stimulus no longer elicits the signal, the system will need to be modified to be effective. A somewhat limited amount of research has investigated the consequences of repeated, prolonged, exposure to the same experimental paradigm or stimuli. Currently, there is no strong consensus as to how the ERP will be affected by repeated use (Ravden & Polich, 1998); however, taken together, the evidence points to the plausibility of a P300 based BCI system. For example, when measuring the P300 of individual trials research has shown the ERP to be relatively stable (Cohen & Polich, 1997; Polich 1989). In addition, scalp distribution does not appear to be affected by task manipulations, and reliability within and between session for a given subject on a given task has been shown to be .70 or higher (Fabiani, Gratton, Karis, & Donchin, 1987). Moreover, test retest correlations for peak amplitude and latency are robust (Polich, 1986). Several studies have shown that P300 amplitude decreases across sessions (Pan, Takeshita, & Morimoto, 2000; Ravden & Polich, 1999; Wintink, et al., 2001). For example, Ravden and Polich (1999) showed that the P300 amplitude, but not latency, decreases across a ten block (approximately 60 minute) session of trials. They interpret this variation in terms of ultradian rhythm variation, which is thought to underlie oscillations in vigilance performance. Wintink, et al. (2001) also showed a decrease in P300 amplitude across time. They examined 5 trial blocks of target presentations and found the P300 amplitude decreases at Fz, Cz, and Pz, the largest decrease being at the Fz 15

PAGE 26

site. In addition to the decrease in amplitude, Wintink et al. (2001) also reported effects of memory load on P300 amplitude. As the memory load increased the amplitude increased at Fz, while it decreased at Pz, and Cz did not significantly change. The former result can be interpreted as the habituation of a novelty P3 effect (Fabiani & Friedman, 1995), while the latter can be attributed to increased frontal activity related to the increase in memory load. The memory load manipulation is relevant to the dual attentional focus conditions that will be employed in the present study. One of the most comprehensive studies investigating evoked potential changes over time was conducted over approximately two months and included eight experimental sessions (Kinoshita, Inoue, Maeda, Nakamura, & Morita, 1996). Kinoshita et al. (1996) found a decrease in the amplitude of the P300 across the first 6 experimental sessions, while latency remained relatively stable. The subjects were initially informed that the experiment would end after six sessions. Unbeknownst to the subjects, one month after the sixth session, they were again tested in two additional sessions. In the additional sessions P300 amplitude returned to the initial level. Polich (1989) has suggested that P300 habituation, or reduction in amplitude, may be associated with a reduction in attentional resources (see also, Wickens, Heffley, Kramer, & Donchin, 1980). It seems reasonable that the subject may become somewhat complacent (attentionally) with the task after six sessions, then, recover in a subsequent unexpected session. Another multiple session study conducted 12 sessions and nearly 2000 trials per session (Kramer, Schneider, Fisk, & Donchin, 1986). The study was designed to examine the effects of practice and the development of automaticity through the use of consistentor variable mapping. Mapping condition, the memory set size, and 16

PAGE 27

probability of an item from the memory set being presented was systematically manipulated using a visual search task. The important finding for the present study is that throughout all sessions a P300 was found for the memory set items, and the amplitude of the P300 did not significantly decrease in the variable mapping condition. Other attentional (Kramer, et al., 1986; Sirevaag, Kramer, Coles, & Donchin, 1989; Wickens, et al., 1980) and workload related (Gopher & Donchin, 1986) factors that are known to affect P300 latency and amplitude will be discussed below. The issues of maintaining and maximizing attention must be investigated in relation to the BCI system. This is critical because the system will be most accurate when P300 latency and amplitude remain relatively stable within and between sessions, allowing the classification algorithm to generalize across time. The primary goal of the P300 speller is to correctly classify target stimuli; to this end, as P300 amplitude increases target detection should be more reliable. Thus, it is imperative to use an experimental paradigm that can yield the largest possible P300 response. Medications may also have an affect on the amplitude of the P300. For example, lorazepam prolonged the latency and reduced the amplitude in a dose-related manner (Pooviboonsuk, Dalton, Curran, & Lader, 1996). In addition, cognitive and psychomotor performance was impaired by the benzodiazepine. As EEG based BCIs continue to be developed drug interactions will need to be considered given the fact that many patients will be taking a number of medications. For example, one nearly locked-in ALS patient who has been tested has difficulty staying awake and alert, presumably because of morphine prescribed for pain. 17

PAGE 28

Mode of Presentation Three modes of stimulus presentation are included in the present study, auditory only, visual only, and auditory + visual. It may be advantageous to use a conjunction of auditory and visual presentation for the current project because the results of psychophysical studies have shown that behavioral responses to multimodal stimuli, presented in close spatial and temporal proximity, produce more accurate and faster responses than unimodal stimuli. Not only has behavioral evidence indicated higher accuracy and faster responses but electrophysiological evidence of cross-modal enhancement has also been shown (Foxe et al., 2000). Therefore, because of the premium placed on discriminability between target and non-target stimuli, a variable that may enhance the EEG response should be investigated. Especially a variable such as the mode of presentation that has virtually no cost associated with using or not using a given modality. Studies examining the effects of presentation mode on accuracy and ERP amplitude have been conducted by Teder-Salejarvi and colleagues (McDonald, Teder-Salejarvi, & Hillyard, 2000; Teder-Salejarvi, et al., 2002). In these studies the subjects were instructed to respond to a more intense noise burst, a brighter flash, or both, within the context of a standard oddball task (target p = .15). Accuracy was higher and ERP amplitude was larger when auditory and visual stimuli were presented simultaneously than when they were presented in isolation (McDonald, Teder-Salejarvi, & Hillyard, 2000; Teder-Salejarvi, et al., 2002). This evidence is quite sufficient to warrant testing of the three presentation modes in a BCI system. Squires, Donchin, Squires, & Grossberg (1977) have also investigated the role of stimulus mode. They presented subjects with either auditory tones, visual flashes, or a 18

PAGE 29

combination of both. The results showed that P300 amplitude was not affected by mode of presentation when the two modes contained redundant information; however, P300 latency was longer for the visual presentation, as is typically the case (see Fabiani et al., 1987). Furthermore, the latency discrepancy can be removed by manipulating discrimination difficulty between the auditory pair and the visual pair of stimuli. Additionally, when the dual mode presentation was used, the P300 latency was similar to that of the auditory only condition. This result indicates that once enough information to classify a stimulus has been delivered the P300 response initiates (Donchin, Kramer, & Wickens, 1982). In the present study the information contained in the auditory and visual modes is completely redundant, as in the Squires, et al. (1977) study. Taken together, these results support the notion that presenting dual mode stimuli will not adversely affect the subjects response to the eliciting stimuli. There are also practical issues that need to be considered in developing a P300 BCI. For example, subjects may need to have augmented presentation because of loss of mobility. For individuals that may benefit the most from the system, eye movements may be difficult, or at the very least retarded, as compared to healthy age matched users. However, limited eye movements are clearly not a factor that would endanger the efficacy of a P300-based BCI. Previous research has shown that eye movements are not necessary to shift visual attention (Posner, 1980; Yantis, et al., 2002). Albeit, the most effective way is to employ visual attention is to shift gaze direction to the attended location. In any event, auditory presentation in not a complete solution either. Auditory presentation may be somewhat problematic because different stimuli (i.e., each different spoken word) may be processed, or recognized, with a different time course. Previous 19

PAGE 30

research has demonstrated that the latency of the P300 reflects the duration of the stimulus evaluation period (Magliero, et al., 1984; McCarthy & Donchin, 1981). This may pose a problem because latency of the P300 will be affected by 50 150 ms (or more) depending the stimulus processing time (Fabiani, et al, 1987). For example, if the spoken word YES is recognized at 100 ms after onset, a P300 should occur at approximately 400 ms. If the spoken word NO is recognized at 60 ms after onset, a P300 should occur at approximately 360 ms. The difference in response time introduces latency jitter into the data and will result in a decrease in P300 amplitude and a lengthening of the P300 latency window. Magliero et al. (1984) examined the effects of visual noise on the amplitude and latency of the P300. Subjects were presented with a 5 x 5 noise matrix that contained a target or non-target word in one of the rows. The remaining locations of the matrix contained either # signs or distractor letters, low and high visual noise conditions, respectively. The subjects task was to search the matrix for the word RIGHT or LEFT, one of the words was a to-be-counted target, and the other word was designated as a to-be-ignored non-target. The words were presented at random using complementary probabilities of .10 and .90. The low probability item elicited a P300 response in the low and high visual noise conditions; however, the amplitude was greater in the low noise condition and the base to peak latency of the response was much longer for the high noise condition, indicative of latency jitter. After adjusting the noise data for latency jitter, the no-noise to noise difference was approximately 300 ms. This suggests that targets, on average, were identified 300 ms slower in the noise condition that in the no-noise condition. 20

PAGE 31

Considering the above issues for and against using three modes of presentation one must remember the purpose of testing the different modes, to find a mode that is optimal for a given subject. Ultimately, some prospective BCI users may have somewhat compromised visual or auditory pathways. Therefore, combining auditory and visual presentation modes may allow subjects who are mildly visually or hearing impaired to compensate by using an aggregate of the available information. The added visual stimulus may produce enough of a needed benefit to reduce latency jitter; or, in the best case, the dual mode presentation may increase the amplitude of the P300 as suggested by previous research (e.g., Teder-Salejarvi, et al., 2002). Task Manipulation The current study has employed two different tasks. The first task is to focus on a target item to be communicated (i.e., YES or NO). The second task requires the subject to focus on a target item to be communicated, and hold a question (that the target item answers) in memory. Memory load, available attention resources, and dual task requirements may play a role in the current task manipulation. Memory load. The two task condition may increase memory load, and memory load has been shown to reduce the amplitude of the P300 (Kramer, et al., 1986; Wintink, et al., 2001). This could occur because the subject will have two tasks to perform simultaneously. First, they must watch or listen to the display to find the target items. Second, they must remember the question to be answered. Kramer et al. (1986) found that increasing the number of items in a memory set from one to four significantly reduced the amplitude of the P300. In partial disagreement, Wintink et al. (2001) 21

PAGE 32

reported that increasing memory load increases the P300 amplitude at Fz, while it decreased at Pz, and Cz did not significantly change. Available attentional resources. The overall amount of attentional resources available to focus on the stimulus may also be affected by the different task conditions. It is possible that holding a question in memory may require a large enough portion of available attention to reduce the amplitude of the P300 response. Assuming that the two tasks draw from the same pool of resources this may be a concern. Donchin, Kramer, & Wickens, (1982) demonstrated that the amplitude of the P300 is related to the demands that a task place on processing resources. If holding a question in memory and attending to a stimulus that correctly answers the question does not tax the attentional system more than only attending to a stimulus, the participants elicited P300 should not be affected. Ultimately, how the elicited response is affected by this manipulation is very important because in practice using a BCI device is dependent on being able to simultaneously hold a question in memory and focus on the correct stimulus item. Dual task requirements. Another possible way to think of the two task condition is that of a dual task. Admittedly, this is not a dual task in the traditional sense that requires two overt responses. Nonetheless, the participant needs to perform two separate cognitive acts. This question is tenable because previous research has demonstrated that the amplitude of the P300 is attenuated when subjects participate in dual-task experiments (Isreal, Chesney, Wickens, & Donchin, 1980). Furthermore, in the most extreme case, when subjects are told to ignore stimuli while performing another task, oddball stimuli do not elicit a P300 (Duncan-Johnson, & Donchin, 1977). In the present study we can 22

PAGE 33

assume that the user will be attending to the presented stimuli in some capacity. To this end, only the dual task studies are directly relevant. A series of studies conducted by Donchin and colleagues has addressed the dual task issue. To summarize, within a single experimental session, latency is relatively stable across experimental conditions. In contrast, when a secondary task is concurrently being performed, P300 amplitude is reduced (Gopher & Donchin, 1989; Kramer, Wickens, & Donchin, 1983, 1985; Isreal, et al., 1980; Isreal, Wickens, Chesney, & Donchin, 1980; Sirevaag, et al., 1989). Sirevaag et al. (1989) demonstrated that the sum of the P300 amplitude remains relatively constant while the amount allocated to a primary and secondary task is reciprocal. They used a tracking task for the primary task and an auditory discrimination task for the secondary task. The results indicated that as priority of the primary task increased the primary task P300 also increased and the secondary task P300 amplitude decreased. Amplitude was largest for the secondary task when the primary task was easiest, and the amplitude for the secondary task was lowest when the primary task was the most difficult. The results demonstrate how task difficulty and attention allocation affect P300 amplitude. The task manipulation is important in the current study because it is an important issue for the early sessions with a new participant. Much of the pilot data for new subjects in this lab, and in other labs, has been collected without good control measures related to what the participant is doing while focusing on the target item. For example, the task given the user may be count the number of yess. However, in practice, the user will be using the system to answer questions; if the user dumps the question from memory before the resulting answer is delivered from the BCI system it may be difficult for the user to participate in a conversation and answer a series of related follow-up questions. 23

PAGE 34

The same question must also be asked in relation to the 6 x 6 BCI speller that is also being developed. Does focusing on the word DOG interfere with attending to the letter D? On the surface it seems to be a subtle distinction, however, if there is a qualitative difference in P300 amplitude or latency for the two attentive conditions, it must be addressed. Currently we do not have a definitive answer to this question because the data has been collected in the copy speller mode (discussed above). The assumption is that there is not a qualitative difference but we have no empirical evidence to support the assumption. However, we do know that in healthy college-aged adults the 6 x 6 Speller does allow users to spell with an accuracy level of .80 at a rate of 5 characters per minute (offline simulation, Donchin, et al., 2000). Other Issues Relevant to BCI Development Averaging Across Trials In most cases, trials must be averaged together to detect the P300 ERP. Ongoing brain activity makes averaging necessary because scalp recorded EEG signals are a combination of all ongoing brain activity (Horst & Donchin, 1980). However, by time-locking averages to a known event (e.g., stimulus presentation) the brain activity not related to the stimulus presentation can be essentially averaged to zero, thereby not affecting the ERP. When the signal of interest is large (e.g., a 10 V P300) fewer trials will need to be averaged together to realize the ERP. In terms of the current study, subjects who elicit a large stable P300 will require fewer stimulus presentations before averaging is required because the signal to noise ratio will be higher for these subjects. The consequence of fewer trials is a faster rate of communication. 24

PAGE 35

Single Trial Classification A problem faced by all BCI systems is how to maximize the signal to noise ratio and classify selection as accurately and as fast as possible. Ideally, we would like to be able to present a series of unique stimuli; then, after all stimuli had been presented, apply a classification algorithm to determine which one was most likely to be the attended item (i.e., the message to be communicated). However, in practice this is not very likely. A main factor that contributes to this is related to the fact that the ERP is a small signal relative to the amount of ongoing EEG activity (Duncan Johnson, & Donchin, 1982). In addition, other factors can influence whether or not a single stimulus will elicit a response. For example, it is possible that, on any given trial, the user may blink, move, cough, or otherwise disrupt the EEG signal. It becomes necessary to present a series of repeated random stimuli to amplify the signal and to reduce the variability introduced into any single time-locked stimulus event. To further complicate the ability to classify using single trial classification, it has been shown that within an oddball sequence there are local sequential dependencies related to stimulus identity and expectancy that influence the elicited response (Squires, Wickens, Squires, & Donchin, 1976). Epoch Overlap The amount of epoch overlap also contributes to the signal to noise ratio. P300 latency is in the range of 300 to 750 ms, and depends on the time to recognize and evaluate the task-relevant event (Gopher & Donchin, 1986). Consequently, if a stimulus is presented every 125 ms, and each epoch contains 900 ms, each epoch will overlap with preceding and subsequent epochs. For example, assume that a P300 occurs on trial N, trial N-1 will exhibit the increase in amplitude at 725 ms and trial N+1 will exhibit an 25

PAGE 36

increase in amplitude at 175 ms. Averaging over multiple trials will reduce the noise associated with overlapping epochs. Another way to reduce this noise would be to present stimuli at a slower rate, as we have done in the current study. The ISI in the current study is 1400 ms. The deleterious effect of this solution is a reduction in the efficiency of the system; however, the goal of the current study is to assess whether or not the system can serve as an alternative method of communication for ALS patients. Increasing the overall speed with which communication can occur is a logical next step in the process. Farwell & Donchin (1988) examined the effects of ISI using the P3 Speller. Two levels of ISI were tested, 125 ms and 500 ms. Three of the four subjects tested were able to reach accuracy levels of 80% and 95% faster in the 500 ms ISI condition than in the 125 ms ISI condition. These results indicate that three of the four subjects were able to select more characters in a given amount of time even though the presentation rate per stimulus is four times longer. For these participants the increase in signal to noise ratio provided by the longer ISI is more beneficial than presenting more stimuli before averaging. The relationship between rate of stimulus presentation and maximizing the signal to noise ratio will need to be determined independently for each user because individual differences will certainly play a role in how fast (or slow) an individual will be able to process the stimulus display. Individual Differences The speed and accuracy of the system has to be adequate for the user to want to continue with the system. A balance between speed and accuracy will need to be established for each individual user. A reasonable assumption is that some users will 26

PAGE 37

want to communicate fast; more errors will be acceptable. Others will want to communicate more slowly but make sure the message they produce is accurate. Operant conditioning and shaping techniques are essential for the SCP and mu based systems, and users learn to control the EEG activity over weeks or months (cf. Kubler, et al., 1999; Pfurtscheller, et al., 2000). In these systems feedback is provided in real time as cursor movement on a computer display. The users ability to modulate the spectrum of the EEG determines whether or not cursor control can be achieved and maintained. In contrast, in the P300 system, a P300 is elicited without any prior training as long as the subject focuses attention on the target stimuli. The feedback in this case is a performance feedback for the process of attention and continuous real-time feedback is not required. The P300 based system provides feedback only after each character selection. After a predetermined number of stimulus presentations, the system will apply a classification algorithm to the set of possible stimuli and select the stimulus with the highest classification coefficient. For both types of systems the feedback is only as good as the classification algorithm that the system uses for evaluation; however, the algorithm is derived from previous data. This effectively means that classification is dependent upon how well the user can modulate brainwaves in a similar fashion across time for the SCP and mu systems, or how similar the elicited response is across time for the P300 system. The amount of variability in the users responses, and the speed at which the desired response can be elicited will determine how many stimulus presentations are necessary for a given level of classification accuracy. In relation to response variability, if the elicited responses are less variable, fewer stimulus presentations will be needed before classification. Similarly, to the extent that discriminable responses can be elicited quickly, the ISI between target flashes can be 27

PAGE 38

reduced. The number of stimulus presentations and the ISI between presentations can be optimized for each user. Some users will not be able to tolerate excessively fast presentation rates, while others may not be able to tolerate the increased error rate associated with the increased system speed. Careful monitoring of the situation by the researcher should yield the most effective solution on a case by case basis. Summary and Goals of the Current Study BCI devices based on the P300 ERP have demonstrated effective communication in completely functional and wheelchair bound adults (Allison & Pineda, 2003; Donchin, et al., 2000; Farwell, & Donchin, 1988). These studies used a 6 x 6 matrix presentation of characters (the P3 Speller) and the system was able to accurately identify the attended character with 56% to 100% accuracy depending on stimulus conditions, using online modes and/or offline simulations. A P300 based BCI has never been tested with an ALS population and a main impetus of the current study is to determine if such a system may be a viable communication option for ALS patients. In a locked-in state ALS patients are thought to be cognitively intact; however, they have lost the ability to communicate by traditional means. Unfortunately, effective communication with a completely locked-in patient has not been demonstrated. Although there is no a priori reason to expect the system will not work with ALS patients, an essential first step is to determine if the system can be effective with ALS patients who are not yet locked-in. At this stage of testing, we must be able to communicate with the participants because they must be able to provide their intended message. Initial testing with 2 locked-in patients (a 70-year old male, and a 54-year old female) showed differential responses to stimuli in two-stimulus oddball paradigm, but 28

PAGE 39

the P3 Speller performance was inconclusive. Given that the oddball data indicated the ability to attend to the target item, the decision was made to test a system that more nearly approximates a standard oddball sequence, hence the four-choice paradigm. It is necessary to include more than two stimuli in the sequence because the amplitude of the P300 is affected by the probability of a target stimulus presentation (e.g., Allison & Pineda, 2003; Duncan-Johnson, & Donchin, 1977). Using 4 stimuli provides a target probability of .25. The four stimuli were the words YES, NO, PASS, and END and they were presented auditorily, visually, or auditorily + visually. The users task was to focus on a given stimulus, or answer a yes/no question. Only the stimuli YES and NO were used as targets. In the current study PASS and END represented filler items that allowed the target stimulus probability to remain in a range acceptable to elicit a P300. Although PASS and END were not actually used in the current study they represent items that could be used in practice. Pass refers to a question that could not be answered, and by choosing this option the user would indicate they would like to pass, or move on, to another question. Similarly, the choice of End would indicate that the user would like to stop using the system. The participants were made aware of this before the study commenced. As stated above a primary goal of the current study is to determine if a P300 based BCI can serve as an effective communication device for an ALS population. Additionally, we are addressing several other factors related to the use and clinical application of such a device. By manipulating the mode of presentation, we are able to examine how auditory and visual presentation will affect performance. This is important for use with people who may have severely limited, or no ability to move their eyes. 29

PAGE 40

Another goal of the current study is to determine if the users task (i.e., focusing on a given item, or answering a specific question) will affect the elicited response. Previous research has shown that the effect of concurrently performing multiple tasks is reduced P300 amplitude (Gopher & Donchin, 1989; Kramer, Wickens, & Donchin, 1983; Isreal, et al., 1980; Isreal, Wickens, Chesney, & Donchin, 1980; Sirevaag, et al., 1989); however, the extent to which (or if) the current situation can be considered as a multiple task has not been ascertained. If the elicited responses are the same in the two conditions it will become a moot point, nonetheless it is a question that should be examined, because it has implications for how the system should be used and calibrated. An equally important goal of the current study is related to the methodological issue of deriving classification weights. Several methods of classification have previously been examined; for example, SWDA, peak picking, area measurements, and covariance were all tested by Farwell and Donchin (1988). SWDA and SWDA with discrete wavelet transformation (DWT) were performed by Donchin et al. (2000). The results indicated that the SWDA solution was nearly as good as, or better than the other methods. Other methods have recently been tested in offline simulations with a high degree of success. For example Kaper, Meinicke, Grossekathoefer, Lingner, & Ritter, (2004) showed classification accuracy of 100% using a Support Vector Machines (SVMs) algorithm. A problem with the SVMs solution is the computational power needed to implement the solution in an online mode. Here we decided to test three methods of deriving SWDA weights. The method used to derive the weights is another factor that has not been previously investigated; however, it may be useful in determining how to calibrate the system for a subsequent or current session. The three methods 30

PAGE 41

(discussed below) are meant to simulate a calibration period at the beginning of each experimental session, or two different amounts of data derived from previous sessions. 31

PAGE 42

Chapter 2: Methods Participants The participants were recruited from the University community and with the help of the Amyotrophic Lateral Sclerosis Association Florida Chapter. Three ALS patients who were able to communicate with the experimenter were included in the study (2 male, mean age 44.3). Three control participants were also included (1 male, mean age 33.7). All subjects signed an informed consent approved by the University of South Florida Institutional Review Board, IRB approval #100650. Data Acquisition & Processing The EEG was recorded using a cap ( Electro-Cap International, Inc. ) embedded with 16 electrodes covering left, right, and central scalp locations (Fz, Cz, Pz, Oz, Fp1, Fp2, F3, F4, C3, C4, P3, P4, P7, P8, T7, T8) based on the 10 20 system of the International Federation (Jasper, 1958). The recordings were referenced to the right earlobe, and grounded to the right mastoid. The EEG was amplified with a SA Electronics amplifier, digitized at a rate of 160Hz, was high-pass filtered at 0.1 Hz, and low-pass filtered at 50 Hz. The electrode impedance did not exceed 5 k. All aspects of data collection and experimental control were controlled by the BCI200 system, developed at the Wadsworth Center, New York State Department of Health (Schalk, et al., 2004). Signal processing (i.e., time domain averaging, generation of scalp topographies, generation of classification algorithms, etc.) was conducted offline using 32

PAGE 43

Matlab 7.0 and SPSS 11.0. Before offline analyses were performed, a moving average filter of four samples and a decimation factor of four samples were applied to the data. Both of these procedures can also be performed online before a classification algorithm is applied to the data. Analysis of variance (ANOVA) was conducted on the classification results (described below) using the following design: Group (Non-ALS vs. ALS) x Session (3 10) x Mode of Presentation (Auditory vs. Visual vs. Auditory + Visual) x Method (1 Run vs. 1 Session vs. 2 Sessions) x Number of Stimuli ( 1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 31). Task, Procedure, & Design Each subject initially participated in a typical oddball experiment. One stimulus was presented at random with a probability of .75 and the other at a probability of .25. Each stimulus was presented for 600 ms and the ISI was set to 1400 ms. The subjects task was to attend to the infrequent stimulus. The purpose of this experiment was to establish each subjects baseline response to a standard oddball sequence. Waveforms for the oddball experiment for each of the six participants are shown in Figure 7. In a typical oddball experiment the subject is instructed to attend to a sequence of events, in most cases two stimuli are used. The subject typically performs one of three different tasks while attending to the sequence of stimuli in an oddball experiment. One task is to press one button on trials in which a target is presented and a different button when non-target trials are presented. A second task is to press a button when a target stimulus is presented and not respond when a non-target is presented. A third task requires the subject to keep a running numeric tally, or make a mental note, of target trials, and filter out, or ignore, the non-target presentations. Because motor responses are 33

PAGE 44

not possible for locked-in patients, the current study has adopted the method of covertly attending to target stimuli. Participants were instructed to silently count target occurrences, or make a mental note of target occurrence. 0 0.5 1 -5 0 5 P300 Amplitude (Pz)Time (sec)Amplitude in V 0.2 0.4 0.6 0.8 1 0 5 10 15 Amplitude in VTime (sec)P300 Amplitude (Pz)non-target target 0.2 0.4 0.6 0.8 1 -2 0 2 4 Amplitude in VTime (sec) 0.2 0.4 0.6 0.8 1 -4 -2 0 Amplitude in VTime (sec) 0.2 0.4 0.6 0.8 1 0 5 10 15 Amplitude in VTime (sec) 0.2 0.4 0.6 0.8 1 0 2 4 6 Amplitude in VTime (sec)Non-ALS ALS Figure 7: Oddball experiment data for all six participants. The target waveforms (solid lines) represent an average of 50 stimulus presentations, and the non-target waveforms represent 150 stimulus presentations. After the initial oddball experiment subjects began testing with the 4 choice, single item at fixation paradigm. Each stimulus was presented with a probability of .25 and the subjects were asked to attend to one of the four stimuli. Stimulus presentation was 600 ms for the visual stimulus, and 600 ms for the auditory stimulus, the ISI was 1400 ms. Each experimental run consisted of 100 stimulus presentations (75 non-target, 34

PAGE 45

25 target). The stimuli were presented in blocks of four at random (one of each stimulus type), 25 times, for a total of 100 presentations. After each run a short break ensued, the duration of the break was determined by the participant, but was typically around 1 minute. All subjects participated in ten experimental sessions that lasted approximately 1 hour. Each session was composed of 12 runs that were counter-balanced across the following variables, mode of presentation, task, and target item. In total, each session included four runs in each of the three modes, six runs for the target YES, six runs for the target NO, and six runs in each of the two task conditions. The experimental sessions deviated from how the system would be implemented in a clinical setting in an important manner. The number of trials per run was held constant for all subjects and all runs. This was a necessary control so that each participant would have an equal amount of experimental data. In a clinical setting, the fewest number of trials that can be used to classify to the desired level of performance would be used. For example, in the current case, 100 trials were used in each run; thus, 25 instances of each item are available to be averaged together. If it is assumed that only 5 trials are needed before averaging to make sufficiently accurate classifications only 20 trials per run would be necessary. In this example subjects would be able to make 5 times the number of classifications in the same amount of time. Obviously, in a real world setting where both speed and accuracy are at a premium the minimal number of trials per run should be used. 35

PAGE 46

Chapter 3: Results Waveform Analysis The oddball experiment waveform traces are shown in Figure 7. The figure indicates that the non-ALS subjects elicited responses conform to what is expected in a standard oddball experiment. That is, the oddball item elicits a large positive deflection in the P300 window, roughly the 250 500 ms range. In contrast, the ALS patient data are not as typical. For example, ALS patient 1 exhibits a maximal positive deflection around 800 ms. Patient 3 does not produce a standard response; however, there is a clear difference between the oddball and the standard stimulus. In a practical sense, the actual shape of the waveform may not be critical in relation to whether or not a BCI system can function with a high degree of accuracy. The critical factor for BCI functioning is the presence of a consistent differential response between the target and non-target (or attended and disregarded) items. Although the reason for the atypical oddball response in the patient group is unclear, there are some possible explanations. First, individual differences in P300 latency and amplitude exist (Fabiani, et al., 1987), and the elicited response changes with age. For example, latency is slightly longer and amplitude is reduced for older adults as compared to young adults (Donchin, Miller, and Farwell, 1986). In addition, Goodin, Squires, Henderson, & Starr (1978) found a latency increase of 1.8 ms/year and an amplitude reduction of .2V/year. Furthermore, the scalp distribution of the P300 is equipotential along the midline in older adults while it is maximal at Pz and decreases 36

PAGE 47

along the midline in younger adults (Fabiani, & Friedman, 1995; Miller, Bashore, Farwell, & Donchin, 1987). These differences are clearly not large enough to account for the present data; nonetheless, changes occur with the normal aging process, and differences are found between subjects. Another possible explanation of the atypical oddball results is related to neuronal cell death of the motor cortex. A fMRI performed on a locked-in patient showed a high degree of cell death along the motor cortex. Imagined movement produced no visible change in the BOLD signal as compared to a no-imagined movement condition. In contrast, imagined speech showed a large difference in the BOLD signal as compared to a no-imagined speech condition (P. Kennedy, personal communication, October 2004). Waveform data for the 4-choice paradigm is presented in Figures 8 and 9. Each figure shows, for each of the three modes, averaged data from Session 1 and Session 10 for each of the six participants. Figure 8 shows data from the Non-ALS participants and Figure 9 shows data from the ALS participants. In general, the within mode responses remain similar across mode from session 1 to session 10. In contrast, within session responses exhibit larger differences across the three modes of presentation. Additionally, this pattern is more evident in the Non-ALS participants than it is in the ALS participants. The ALS participants responses are more variable across mode and time than the Non-ALS participants. This is corroborated by the classification data discussed below. The increased variability is also evident in the topographic representation presented in Figure 10. Each topographic representation displays the time point of the maximal r 2 between the target and non-target stimulus, labeled on the x-axis. Although the time of maximal r 2 is variable, the signal is spatially more stable for the Non-ALS participant than for the ALS patient. 37

PAGE 48

0.5 1 0 1 2 3 Amp. in VTime (sec) 0.5 1 0 1 2 3 Amp. in VTime (sec) 0.5 1 0 1 2 3 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 -2 0 2 4 Amp. in VTime (sec) 0.5 1 -2 0 2 4 Amp. in VTime (sec) 0.5 1 -2 0 2 4 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec) 0.5 1 -2 0 2 Amp. in VTime (sec) 0.5 1 -2 0 2 Amp. in VTime (sec) 0.5 1 -2 0 2 Amp. in VTime (sec) 0.5 1 -2 0 2 4 Amp. VTime (sec) 0.5 1 -2 0 2 4 Amp. in VTime (sec) 0.5 1 -2 0 2 4 Amp. in VTime (sec)Aud Vis Aud Vis NonA LS Participant 1 Session 1 Session 10 Session 1 Session 10 Session 1 Session 10 Participant 2 Participant 3 Figure 8: Non-ALS participants average waveforms for each presentation mode in Session 1 and Session 10. Target waveforms (solid lines) represent an average of 100 stimuli and non-target waveforms (dashed lines) represent an average of 300 stimuli (electrode Pz). 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 0 2 4 Amp. in VTime (sec) 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 2 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 -1 0 1 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec) 0.5 1 0 2 4 6 Amp. in VTime (sec)Aud Vis Aud Vis Participant 1 Session 1 Session 10 Session 1 Session 10 Session 1 Session 10 Participant 2 Participant 3 A LS Figure 9: ALS participants average waveforms for each presentation mode in Session 1 and Session 10. Target waveforms (solid lines) represent an average of 100 stimuli and non-target waveforms (dashed lines) represent an average of 300 stimuli (electrode Pz). 38

PAGE 49

r2 at 445.0 ms 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 r2 at 500.0 ms 0.005 0.01 0.015 0.02 0.025 r2 at 450.0 ms 0.002 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0.02 Non-ALS r2 at 650.0 ms 0.005 0.01 0.015 0.02 0.025 r2 at 465.0 ms 0.01 0.02 0.03 0.04 0.05 0.06 r2 at 600.0 ms 0.004 0.006 0.008 0.01 0.012 0.014 0.016 0.018 0.02 ALS VisualAud + Vis Auditor y Figure 10: Topography of each presentation mode for one Non-ALS and one ALS participant. The x-axis shows the time at which the r 2 between the target and non-target stimulus was maximized for each of the three modes. SWDA Analysis We chose to use the Step-Wise Discriminant Analysis (SWDA) method of classification for the current analysis because previous research (e.g., Donchin, et al., 2000; Farwell & Donchin, 1988) has demonstrated that SWDA performs equally well or better than several other methods of operationally defining P300 waveforms (Fabiani, et al., 1987). Previous research has not, however, examined how deriving SWDA weights from different subsets of data affects classification accuracy. Therefore, the analyses were conducted using three different methods of deriving SWDA weights. The offline analyses are a critical part of the project, as they compare how well each classification technique classifies the participants responses into the correct category. This is important in a BCI application because our goal is to achieve the highest possible 39

PAGE 50

classification accuracy, in the most efficient manner possible. Deriving weights frdifferent subsets of data and applying them to independent data sets for testing will allous to directly compare performance across the different methods of derivation. In general, two steps are involved in the process. The first step is concerned with deriweights using single-trial time-locked ERPs. The second step applies the weights to datasets that are created from independent samples via bootstrapping. Before describing how the functions are derived and how they are then applied to the bootstrapped data the method will be described briefly. SWDA is a linear classificat om w ving ion method that identifies variables, or features, that optimat ni, et eriving Weights t study the criteria were set to a minimum of 4 steps and a max of 10 d lly characterize differences among groups. In this case group is defined as target or non-target stimulus presentations. It is an iterative regression procedure thaselects features that discriminate between the groups of data. Initially, the feature that accounts for the largest amount of variance is partialled out and a regression weight is computed, the procedure is then repeated until a certain criterion is reached. The criterion can be related to the number of features selected and/or an F value (Fabiaal., 1987). D In the presen steps, or an F value of .10 to enter features, and .15 to eliminate features. In some cases it was necessary to relax these constraints (e.g., Method 1 uses only 100 stimuli and in some instances it was necessary to increase the F to enter, otherwise no features woulenter the solution). 40

PAGE 51

We recorded data from 16 electrode channels at a sample rate of 160 Hz; therefore, we have 16 x 160 features available to derive the classification weights. Although we have 2560 features at our disposal it is not necessary to include all of this information in the analysis for two primary reasons. First, the largest amplitude of the P300 is located along the midline electrodes. Second, using contiguous time points adds a substantial amount of redundant information to the analysis; this may result in the solution over-fitting the data. Over-fitting the training data may reduce the generalizability of the solution to a subsequent data set. Accordingly, three midline electrodes (Fz, Cz, and Pz) were used in the analysis, and the data were decimated by a factor of four. The time epoch used for all analyses was 900 ms. The use of these parameters results in a total of 108 spatial location x time features to be used in each analysis. Each set of SWDA weights was derived separately for each mode of presentation (auditory, visual, and auditory + visual), and applied to data sets which included only same mode data. In other words, each analysis was conducted three times, once for each presentation mode. In addition, three different methods were used to derive SWDA weights. Method 1 used only the first run of a session. One run consisted of 100 stimulus presentations; SWDA weights were derived using 25 attended and 75 non-attended stimulus presentations. The purpose of method 1 is to simulate a mode in which the system can be calibrated at the beginning of a session and the derived weights would be used online following the calibration period. Method 2 used the data from an entire session to derive the weights; in this case 100 attended and 300 non-attended stimuli were available for the analysis. Method 3 used data from two sessions to derive the weights; in this case 200 attended stimuli and 600 non-attended stimuli were available for the 41

PAGE 52

analysis. Method 2 and 3 simulate modes in which weights would be rendered offline, following a session (or multiple sessions). The derived weights would then be used in a subsequent session online. Applying Weights The weights were applied to data sets created via a standard bootstrapping method described below (Efron, & Tibshirani, 1993). Each data set consisted of 400 trials for sampling (100 trials for each stimulus type). One thousand sets of each of the four stimulus types were created for each number (N) of averaged samples from 1 to 31, incremented by 3. This results in 4000 total cases for each bootstrapped data set. For each mode of presentation and each experimental session the following steps were executed (modeled after Donchin, et al., 2000): 1) Obtain a random sample of N trials (stimulus presentations) for each of the four stimulus types (YES, NO, PASS, and END) by sampling with replacement from the set of 400 trials. 2) Compute the average for each stimulus type. 3) Apply SWDA weights to the appropriate features and select the stimulus with the maximum discriminant score. 4) If the selected stimulus is defined as the target count a hit, if one of the other three stimuli are selected count a miss. 5) Record the percentage of hits among the 1000 sets of samplings. The final result is the percent accuracy at each level of N trials. The results can be used to determine the optimal speed and accuracy level for each participant. Classification Tables of complete classification results for all participants and sessions are shown in Appendix A. Mean classification accuracy for each of the three methods and 42

PAGE 53

each of the three modes is presented in figures 11 16 (Figures 11 13 are the Non-ALS data). There were no significant differences based on the users task, that is, whether they focused on a given word or whether they answered a question by focusing on the word that correctly answered a question posed by the experimenter. Given that user task had no significant effects, the data were collapsed across task before the analysis was conducted. Classification accuracy was entered into a mixed design factorial ANOVA using the between groups variable Group (nonALS vs. ALS), and the within groups variables of Session (3 10), Mode of Presentation (Auditory vs. Visual vs. Auditory + Visual), Method (1 Run vs. 1 Session vs. 2 Sessions), and Number of Stimuli (1, 4, 7, 10, 13, 16, 19, 22, 25, 28, 31). The data presented in Figures 11 16 show that classification accuracy reaches a level of two times chance (50%) with as few as 7 stimulus presentations, in the worst case, and classification accuracy above 90% with as few as 13 stimulus presentations in the best case. For all participants classification accuracy begins to asymptote with approximately 13 stimuli, and continues to increase slightly throughout the tested range of 31 stimuli. Several effects obtained statistical significance. The main effect of Method yielded significant effects, (F(2,8)=9.64, MSe=109.5, p =.007). Method 1, using the first run of a session, and Method 3, using the aggregate of two previous sessions classified significantly better than Method 2 (data from the previous session). The main effect of Number of Stimuli was also significant (F(10,40)=135.57, MSe=171.4, p =.0001). As the number of stimuli averaged before classification was increased, classification accuracy also increased. This result indicates that as the number of stimuli averaged together increases, the waveform more nearly approximates the shape of the waveform created by the data used to derive the SWDA weights. 43

PAGE 54

Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Aud + Vis Mode204060801001471013161922252831Number of StimuliAccuracy Visual Mode204060801001471013161922252831Number of StimuliAccuracy Figure 11: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 1. 44

PAGE 55

Visual Mode204060801001471013161922252831Number of StimuliAccuracy Aud + Vis Mode204060801001471013161922252831Number of StimuliAccuracy Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Figure 12: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 2. 45

PAGE 56

Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Visual Mode204060801001471013161922252831Number of StimuliAccuracy Aud + Vis Mode204060801001471013161922252831Number of StimuliAccuracy Figure 13: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, Non-ALS participant 3. 46

PAGE 57

Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Visual Mode204060801001471013161922252831Number of StimuliAccuracy Aud + Vis Mode204060801001471013161922252831Number of StimuliAccuracy Figure 14: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 1. 47

PAGE 58

Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Visual Mode204060801001471013161922252831Number of StimuliAccuracy Aud +Vis Mode204060801001471013161922252831Number of StimuliAccuracy Figure 15: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 2. 48

PAGE 59

Auditory Mode204060801001471013161922252831Number of StimuliAccuracy 1Run 1Ses 2Ses Visual Mode204060801001471013161922252831Number of StimuliAccuracy Aud + Vis Mode204060801001471013161922252831Number of StimuliAccuracy Figure 16: Mean classification accuracy as a function of presentation mode and number of stimuli averaged, ALS participant 3. 49

PAGE 60

Although the main effect for Group did not reach significance, the Group x Method interaction did reach significance (F(2,8)=16.95, MSe=145.2, p =.0013). Overall, the Non-ALS group accuracy was higher (mean=62.45) than the ALS group accuracy (mean=50.99). There was no difference between Method 1 for the two groups; however, the Non-ALS group classification accuracy was higher for Method 2 and Method 3, see Figure 17. In addition, classification accuracy was highest for the ALS group with Method 1 (mean=56.83), and highest in the Non-ALS group for Method 3 (mean=65.91). This result indicates more response variability across sessions in the ALS group. The Group x Number of Stimuli interaction was also significant (F(10,40)=2.29, MSe=22.3, p =.031). As can be seen in Figure 18, as the number of stimuli averaged increases, the Non-ALS group accuracy increased at a faster rate than that of the ALS group. Group x Method Interaction0204060801001 RUN1 SES2 SESMethodAccuracy Non-ALS ALS Figure 17: Group x Method interaction. Two other interactions were significant. First, the Session x Number of Stimuli interaction was significant (F(70,280)=1.46, MSe= 6.67, p =.019). As shown if Figure 19, the obtained classification accuracy in Session 3 and Session 10 was highest, and the 50

PAGE 61

intermediate session accuracy, although similar in terms of the performance curves, was slightly lower. In addition, as the number of samples averaged together increases the differences between each sessions classification accuracy increase, classification accuracy spreads out across the range of stimulus samples averaged. This result is consistent with what previous research has found regarding habituation effects and a reduction in P300 amplitude (Ravden & Polich, 1999; Wintink, et al., 2001). The Mode of Presentation x Number of Stimuli interaction was also significant (F(20,80)=1.97, MSe=10.1, p =.018). As Figure 20 illustrates, accuracy levels for all three modes begin at the same level, and as the number of stimuli averaged increases, the accuracy increases more for the visual and auditory + visual modes than it does for the auditory mode. Group x Number of Stimuli Interaction0204060801001471013161922252831Num of StimuliAccuracy Non-ALS ALS Figure 18: Group x Number of Stimuli interaction. The above results provide an objective measure of how accurately the system will classify responses at each level of number of stimuli averaged. The following example is based on Non-ALS participant 1 data (Figure 11, Visual Mode panel). Method 2 reaches 75.5% accuracy with 4 samples. In this case, the system would have to present 4 samples 51

PAGE 62

of each of the four stimuli (16 presentations) before averaging. Given the ISI of 1400 ms used in the current study, the result corresponds to one classification every 22.4 seconds. If 19 samples are obtained before averaging classification accuracy increases to 92.4%; however, classification time increases to 106 seconds/selection. In practice the speed/accuracy tradeoff will be dependent upon the individual users performance preferences. Some users will choose to communicate at a higher rate, accepting more errors, while others may wish to be more accurate but proceed at a slower pace. Session x Num of Stimuli Interaction0204060801001471013161922252831Number of StimuliAccuracy ses3 ses4 ses5 ses6 ses7 ses8 ses9 ses10 Figure 19: Session x Number of Stimuli interaction. The focus of the current study was to determine if a P300 based system may be a viable option for an ALS population; not to optimize the speed of the system, and as such the rate of stimulus selection is not impressively rapid. In part, this is because the current system was configured to test the efficacy of spoken stimuli, and by using spoken stimuli we reduced the speed with which the system can operate. Recall that each stimulus was presented for 600 ms, this constraint was made necessary by the limitation of the spoken stimuli; however, it would not be necessary with visual only presentations, or tone bursts 52

PAGE 63

(in place of the spoken stimuli). The ISI can also be adjusted to increase the number of stimulus presentations in a given time period; however, the current study did not examine this variable. Recall that Farwell & Donchin (1988) examined the effects of ISI and found that a longer ISI provided a higher rate of communication for three of the four participants tested. There are several potential ways to increase the speed of the system that will be discussed below. Mode x Number of Stimuli Interaction0204060801001471013161922252831Number of StimuliAccuracy Aud Vis Aud+Vis Figure 20: Mode of Presentation x Number of Stimuli interaction. 53

PAGE 64

Chapter 4: Discussion The current study demonstrates that a P300-based BCI may be a viable option for a person who has lost the ability to communicate through the use of normal output pathways. This project has made substantial progress in answering several questions essential to the development of a BCI. In fact, the variables examined in this study have not been examined in relation to a BCI interface. The overarching question we sought to answer was can a P300-based BCI function as an alternative method of communication for a patient population. To achieve this goal an accuracy level of approximately 75% would have to be obtained so that the system would not enter into a recursive loop of incorrect or ambiguous answers. The current data suggest that, while classification performance appears to be higher for the Non-ALS group, as indicated by the group x method interaction, the patient population could benefit from such a system. Only one of the six participants did not reach this level of accuracy (maximum 61.6%). Variables related to practical issues, theoretical issues, and methodological issues were all tested within the framework of the current design. The primary variable related to practical issues was the mode of presentation variable. It is important to determine if a BCI device can function effectively using different presentation modalities, because it is quite possible that a user may have an auditory or visual deficiency. Previous research has shown that auditory and visual oddball task both elicit large P300 responses (Fabiani et al., 1987; Donchin, Kramer, & Wickens, 1982). However, because of the possibility of latency jitter on the P300 response associated with word recognition, words may not be 54

PAGE 65

the best method of presenting auditory information. In fact, Figure 20 shows that the auditory mode did not reach the same levels of classification accuracy as the visual or the auditory + visual mode. However, the auditory mode classification was as good as or better than the other two modes for three of the six participants. This result is encouraging because it demonstrates that users who may have a compromised visual system can still be a likely candidate for using a BCI. One luxury we have in using a system based on the P300 is a strong body, nearly 40 years, of theoretical research. Although we obtained null effects with the user task manipulation, it was nonetheless an important question to ask. As discussed above, previous research has shown that the effect of concurrently performing multiple tasks is reduced P300 amplitude (Gopher & Donchin, 1989; Kramer, Wickens, & Donchin, 1983; Isreal, et al., 1980; Isreal, Wickens, Chesney, & Donchin, 1980; Sirevaag, et al., 1989). Previous research has not examined whether or not the elicited response is either enhanced or attenuated by merely focusing on a stimulus item, or focusing on a stimulus item that answers a specific question. The obtained null results suggest that the two processes may not be different. In terms of a BCI device, this is good news. In so much as we can feel more confident that participants need not answer questions when collecting calibration data, they can simply focus on target stimuli. In turn, the elicited responses will generalize when they use the system for the intended purpose, to answer questions. Collecting data across ten sessions also provided a large enough sample of data to examine how the effects of repeated use affect classification accuracy. The classification data showed that significant classification differences are present across sessions; however, the waveform data suggest that these differences are not un-manageably large. 55

PAGE 66

For example, Figures 8 and 9 show a high degree of similarity in the shape of the waveforms across the three modes of presentation, for session 1 and session 10. In addition, the fact that classification accuracy for all sessions starts at approximately the same level, and fans out as the number of averaged stimuli is increased, is largely responsible for the significant interaction between sessions and number of stimuli averaged, see Figure 19. Moreover, classification accuracy for session 3 and 10 is slightly higher than the intermediate sessions. This result can be explained in terms of motivational and attentional issues. In the first few sessions anxiety will be at higher levels and performance may benefit from this heightened arousal level. Similarly, for session 10, the participants were aware of the fact that they were participating in the final session, and the desire to perform well may be greater than in intermediate sessions. In support of this proposal Kinoshita et al. (1996) found similar effects. In an initial session P300 amplitude was at the highest level, and in subsequent session the amplitude slightly decreased, while latency remained relatively constant. When a surprise final experimental session was performed P300 amplitude spontaneously returned to the same level as the initial session. The most important detail in Figure 19 is represented by the curves for the intermediate sessions. Essentially, they are identical, suggesting that classification accuracy is quite stable across the intermediate session (similar to the results of Kinoshita et al., 1996). In practice, after session 1 a user will always be in the intermediate range of sessions because use may be expected to continue indefinitely. The current study also examined methodological issues related to classification. Three methods of deriving SWDA weights were compared. The first method, using the first run of a session, was selected to simulate classification weights that could be derived online at the beginning of a session. This method examined how well the system could 56

PAGE 67

function using a minimal amount of calibration data, and would result in less off-line data processing. Interestingly, the ALS group performed best with the weights derived from method 1. This result indicates that the ALS groups elicited responses are more variable than the Non-ALS groups elicited responses. This is not entirely surprising because ALS patients suffer from fasciculation (twitching) and cramping of muscles, especially those in the hands and feet. On a day-to-day basis, symptoms may vary and can differentially affect attention and concentration leading to less genaralizable performance. In contrast, for the Non-ALS group, classification accuracy was highest with method 3. Method 3 used data aggregated from the previous two sessions. This result suggests that, over time, the elicited responses are quite stable and using more data to derive classification weights may be an optimal method for a non-disabled population. Practical Issues There are also some practical issues that will need to be addressed before in-home long-term use of a BCI system can be realistically achieved. One important issue is related to acquiring high quality EEG recordings. EEG artifacts can arise from involuntary muscle movements, or environmental sources. Although none of the participants in the current study required artificial respiration, electrical interference from necessary devices is a source of interference we have observed while working with the locked-in population. Particular attention to grounding amplifiers and moving recording wires can typically overcome such problems; however, each case will certainly present new and different challenges. Other issues that will need to be addressed are related to operating such a system. User friendly software that would allow a spouse or caregiver to easily operate a BCI has 57

PAGE 68

not yet been developed. At the present time a trained researcher is required at all recording sessions. We have been working closely with a group of dedicated individuals who have no formal training in EEG recording, and it has taken approximately 20 experimental sessions to record useable data. Although it is certainly only a matter of time, the process of developing a software package that can be executed without a long training period will have to be carefully designed and implemented. Future Directions As stated previously, the P3 Speller will continue to be developed in parallel with the 4 choice system. In fact, recently collected data, using method 3 described above, has resulted in online real-time classification accuracy between 94.1% and 100%, in seven consecutive experimental sessions. The participant was an ALS patient who still retains the ability to communicate, but cannot speak. Although each character was selected at a rate of .75 chars/minute, these results are very encouraging. For a person who has lost the ability to communicate, item selection at a rate of .75 chars/minute can be quite acceptable. This particular patient has been trained using mu rhythms, slow cortical potentials, and the P300 system. The P300 system was the most effective, and the patient has expressed wishes to continue working within the P300 framework. We are currently devising a new research plan that will move from basic research to practical application, and this patient will be the first to benefit from these advances. A study is also being designed, using the four choice paradigm, that will compare tone bursts and visual stimuli, and different ISIs using a design similar to the one employed in this study. This will eliminate issues related to latency jitter and stimulus duration that were inherent, and unavoidable, in the current study. Using tone bursts will 58

PAGE 69

also allow us to more closely examine the effects of ISI within this paradigm, and further calibrate the system based on optimal timing for each user. In addition, we will be able to incorporate real-time online classification based on previous data. Providing feedback to the user may be a valuable motivational tool that will keep participants engaged in the task, to the extent that classification accuracy is at an acceptable level for the user. Additional studies are also currently being designed to examine the performance of different classification algorithms. Support vector machines (SVMs) are a likely candidate (see Bennett & Campbell, 2003, for a review). The SVMs approach has recently demonstrated classification accuracy approaching 100% in offline simulations (Kaper, et al., 2004). Currently, however, online implementation of the SVMs solution is not available. If the efficacy of the method proves worthwhile, online implementation of such algorithms will be devised. Several other system features can improve system flexibility and help increase the information transfer rate. The interface could be designed to toggle between matrix driven menus. Because the P300 response is not dependent upon stimulus identity, such a system affords itself to be used with multiple levels of matrices. For example, a main matrix would contain icons to activate other matrices. This matrix would allow the user to use a spelling mode, a nurse mode, a food mode, or an entertainment mode. In practice, any mode that would benefit the user could be designed. From any matrix the user would always have an option to return to the main matrix. Another improvement that can increase the attractiveness and efficiency of the spelling system is smart spelling. To date this option has not been investigated; however, smart spelling technology is being used in effectively in other communication systems (e.g., REACH Smart Key Technology ). It is also easy to imagine a mode in 59

PAGE 70

which common phrases can be selected, as opposed to a mode that selects one character at a time. One of the most likely ways of improving the overall performance of a BCI system is to add an error detection mechanism to the system. One ERP component that is a candidate to incorporate an error detection algorithm is the error-related negativity (ERN). The ERN is a negative deflection in the ongoing EEG, recorded from central or frontal-central scalp locations (e.g., Cz, FCz, or Fz), that occurs approximately 100 ms after an error has been committed (Falkenstein, Hohnsbein, Hoormann, & Blanke, 1990; Gehring, Goss, Coles, Meyer, & Donchin, 1993; Gehring, Coles, Meyer, & Donchin, 1995). BCI applications are good candidates for incorporation error correction because many experimental paradigms that provide time locked feedback can potentially elicit an ERN. For example, the ERN has been observed in Eriksen flanker tasks (Gehring, et al., 1993; Holroyd & Coles, 2002), Go/No-Go tasks (Falkenstein, et al., 2000; Vidal, Hasbroucq, Grapperon, Bonnet, 2000), antisaccadic tasks (Nieuwenhuis, et al., 2001), and sentence verification tasks (Gehring, Coles, Meyer, & Donchin, 1995). Schalk, Wolpaw, McFalrland, & Pfurtscheller, (2000), have developed a preliminary error correction module that has been implemented with a mu rhythm system. The error potential has a positive deflection peaking at approximately 180 ms after feedback presentation. In summary, there are many potential ways to increase the effectiveness of BCI devices in general and the P300-based system in particular. The current study has helped to elucidate an important set of preliminary questions related to working with a patient population, and has, at the same time, provided a roadmap for several new issues to address. 60

PAGE 71

Conclusions BCI devices are beginning to allow people to communicate through non-traditional means. Although the current rate of information transfer is slow as compared to speaking, or writing, people who suffer from neuromuscular disabilities are now being offered an alternative method of communication. As with any new technology, as more research is conducted, the systems will be refined and performance will increase accordingly. In addition, it will become clear as to which systems and interfaces work best under different circumstances and conditions. At this stage of development it is important to have a dual focus research program. One focus should be on controlled studies that examine questions derived from sound scientific theory. The second focus should be on applying the findings to the population that will benefit most from the research. This focus can be somewhat less controlled and can keep the individual users interests at the forefront. Wolpaw, et al. (2002) is currently carrying out such a research plan based on this model. Such studies coupled with careful analysis of individual users data will help to ensure each user is provided the optimal system for the particular situation. Moreover, given the interdisciplinary nature of the development, collaboration between engineers, computer scientists, cognitive neuroscientists, psychologists, and medical doctors is an essential piece of the puzzle. It is also important that the overall goals do not overshadow the purpose of the systems; that is, a new opportunity to communicate, and in some cases binary decision systems may be appropriate or all that is available. However, for the locked-in population, a degree of autonomy is regained or kept by allowing a user to select characters and compose words. Evidence of this is provided by patients preferences to 61

PAGE 72

continue training with a spelling system, even when performance is less than optimal (Kubler, et al., 2001). Thus, it is important to continue with the development of BCI systems that can be flexible enough to move out of the lab and into patients homes. 62

PAGE 73

References Allison, B.Z., & Pineda, J.A., (2003). ERPs evoked by different matrix sizes: Implications for a brain computer interface (BCI) system. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 11, 110-113. Bauby, J. D. (1997). The Diving Bell and the Butterfly. Random House, New York. Bennett, K.P., & Campbell, C. (2003). Support vector machines: Hype or hallelujah? SIGKDD Explorations, 2, 1-13. Birbaumer, N. (1984). Operant control of slow brain potentials: A tool in the investigation of the potentials meaning and its relation to attentional dysfunction. In: T. Elbert, B. Rockstroh, W. Lutzenberger, & N. Birbaumer (Eds.), Self-regulation of the Brain and Behaviour, 1 st Ed (pp. 227-239). Berlin, Heidelberg, New York: Springer. Birbaumer, N., et al. (1999). A spelling device for the paralyzed. Nature, 398, 297-298. Birbaumer, N., et al. (2000). The thought translation device (TTD) for completely paralyzed patients. IEEE Transactions On Rehabilitation Engineering, 8, 190-193. 63

PAGE 74

Cohen, J., & Polich, J. (1997). On the number of trials needed for P300. International Journal of Psychophysiology, 25, 249-255. Donchin, E. (1981). Surprise!surprise?. Psychophysiology, 18, 493-513. Donchin, E., & Coles, M.G.H. (1988). Is the P300 component a manifestation of context updating? Behavioral and Brain Sciences, 11, 355-372. Donchin, E., & Isreal, J.B. (1980). Event-related brain potentials: Approaches to cognitive psychology. In R.E. Snow, P.A. Federico, & E.E.E. Montague (Eds.), Aptitude, Learning, and Instruction: Cognitive Process Analyses, Vol. 2 (pp. 47-82). Hillsdale, NJ: Lawrence Erlbaum Associates. Donchin, E., Karis, D., Bashore, T.R., Coles, M.G.H., & Gratton, G. (1986). Cognitive psychophysiology and human information processing. In M.G.H. Coles, E. Donchin, & S.W. Porges (Eds.), Psychophysiology Systems, Processes and Applications (pp 244-267). New York: Guilford Press. Donchin, E., Kramer, A.F., & Wickens, C. (1982). Probing the cognitive infrastructure with event-related brain potentials. In Proceedings of the AIAA Wrokshop on Flight Testing to Idnetify Pilot Workload and Pilot Dynamics (pp. 371-387). Washington, DC: U.S. Government Printing Office. 64

PAGE 75

Donchin, E., Miller, G.A., & Farwell, L.A. (1986). The endogenous components of the event-related potential a diagnostic tool? In D.F. Swaab, E. Fliers, M. Mirmiran, W.A. Van Gool, & F. Van Haaren (Eds.), Progress in Brain Research, Vol. 70 (pp. 87-102). Amsterdam: Elsevier. Donchin, E., Ritter, W., & McCallum, W.C. (1978). Cognitive psychophysiology: The endogenous components of the ERP. In E. Callaway, P. Tueting, & S.H. Coslow (Eds.), Event-Related Brain Potentials in Man (pp. 349-411). New York: Academic Press. Donchin, E., Spencer, K.M. & Wijesinghe, R. (2000). The mental prosthesis: Assessing the speed of a P300-based brain-computer interface. IEEE Transactions on Rehabilitation Engineering, 8, 174-179. Duncan-Johnson, C.C., & Donchin, E. (1977). On quantifying surprise: The variation in event-related potentials with subjective probability. Psychophysiology, 14, 456-467. Duncan-Johnson, C.C., & Donchin, E. (1981). The relation of P300 latency to reaction time as a function of expectancy. In Kornhuber, H. H., & Deecke, L. (Eds.), Motivation, Motor and Sensory Processes of the Brain: Electrical Potentials, Behavior and Clinical Use. Process in Brain Research vol. 54 (pp. 717-722). Amsterdam: Elsevier. 65

PAGE 76

Efron, B., & Tibshirani, R.J. (1993). An Introduction to the Bootstrap. New York: Chapman and Hall. Fabiani, M., & Friedman, D. (1995). Changes in brain activity patterns in aging: The novelty oddball. Psychophysiology, 32, 579-594. Fabiani, M., Gratton, G., Karis, D., & Donchin, E. (1987). Definition, identification, and reliability of measurement of the P300 component of the event-related brain potential. Advances in Psychophysiology, 2, 1-78. Falkenstein, M., Hohnsbein, J., Hoormann, J., & Blanke, L. (1990). Effects of errors in choice reaction tasks on the ERP under focused and divided attention. In C. Brunia, A. Gaillard, & A. Kok (Eds.), Psychophysiological Brain Research (pp. 192-195). Tilburg, the Netherlands: Tilburg University Press. Falkenstein, M., Hoormann, J., Christ, S., Hohnsbein, J. (2000). ERP components on reaction errors and their functional significance: a tutorial. Biological Psychology, 51, 87-107. Farwell, L.A., & Donchin, E. (1988). Talking off the top of your head: Toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and clinical Neurophysiology, 70, 510-523. 66

PAGE 77

Foxe, J.J., Morocz, I.A., Murray, M.M., Higgins, B.A. Javitt, D.C., & Schroeder, C.E. (2000). Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping, Cognitive Brain Research, 10, 77-83. Gehring, W.J., Coles, M.G.H., Meyer, D.E., & Donchin, E. (1995). A brain potential manifestation of error-related processing. In G. Karmos, M. Molnar, V. Csepe, I. Czigler, & J. E. Desmedt (Eds.), Perspectives of Event-Related Potentials Research (EEG Suppl. 44) (pp. 261-272). Amsterdam: North-Holland/Elsevier Science. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E., & Donchin, E. (1993). A neural system for error detection and compensation. Psychological Science, 4, 385-390. Goodin, D., Squires, K., Henderson, B., & Starr A. (1978). Age related variations in evoked potentials to auditory stimuli in normal human subjects. Electroencephalography and Clinical Neurophysiology, 44, 447-458. Gopher, D., & Donchin, E. (1986). Workload an examination of the concept. In L. Kaufman and K. Boff (Eds.), Handbook of Perception and Human Performance (pp. 41.1 41.49). New York: John Wiley. Holroyd, C.B., & Coles, M.G.H. (2002). The neural basis of human error processing: Reinforcement learning, dopamine, and the error-related negativity. Psychological Review, 109, 679-709. 67

PAGE 78

Horst, R.L., & Donchin, E. (1980). Beyond averaging: II. Single-trial classification of exogenous event-related potentials using stepwise discriminant analysis. Electroencephalography and Clinical Neurophysiology, 48, 113-126. Isreal, J.B., Chesney, G.L., Wickens, C.D., & Donchin, E. (1980). P300 and tracking difficulty: Evidence for multiple resources in dual-task performance. Psychophysiology, 17, 259-273. Isreal, J.B., Wickens, C.D., Chesney, G.L., & Donchin, E. (1980). The event-related brain potential as an index of display monitoring workload. Human Factors, 22, 212-224. Jasper, H. (1958). The ten twenty electrode system of the International Federation. Electroencephalography and Clinical Neurophysiology, 10, 371-375. Kaper, M., Meinicke, P., Grossekathoefer, U., Lingner, T., & Ritter, H. (2004). BCI competition 2003 data set IIb: Support vector machines for the P300 speller paradigm. IEEE Transactions on Biomedical Engineering, 51, 1073-1076. Kennedy, P.R., & Bakay, R.A.E., (1998). Restoration of neural output from a paralyzed patient by a direct brain connection. NeuroReport, 9, 1707-1711. 68

PAGE 79

Kennedy, P.R., Bakay, R.A.E., Moore, M.M., Adams, K., and Goldwaithe, J. (2000). Direct control of a computer from the human central nervous system. IEEE Transactions on Rehabilitation Engineering, 8, 198-202. Kinoshita, S., Inoue, M., Maeda, H., Nakamura, J., & Morita, K. (1996). Long-term patterns of change in ERPs across repeated measurements. Physiology & Behavior, 60, 1087-1092. Kramer, A.F., Schneider, W., Fisk, A., & Donchin, E. (1986). The effects of practice and task structure on components of the event-related brain potential. Psychophysiology, 23, 33-47. Kramer, A.F., Wickens, C.D. & Donchin, E. (1983). An analysis of the processing requirements of a complex perceptual-motor task. Human Factors, 25, 597-621. Kramer, A.F., Wickens, C.D. & Donchin, E. (1985). Processing of stimulus properties: Evidence for dual-task integrality. Journal of Experimental Psychology: Human Perception and Performance, 11, 393-408. Kubler, A., et al., (1999). The thought translation device: A neurophysiological approach to communication in total motor paralysis. Experimental Brain Research, 124, 223-232. 69

PAGE 80

Kubler, A., Kotchoubey, B., Kaiser, J., Wolpaw, J.R., & Birbaumer, N. (2001). Brain-computer communication: unlock the locked-in. Psychological Bulletin, 127, 358-375. Magliero, A., Bashore, T.R., Coles, M.G.H., & Donchin, E. (1984). On the dependence of P300 latency on stimulus evaluation process. Psychophysiology, 21, 171-186. McCarthy, G., & Donchin, E. (1981). A metric for thought: A comparison of P300 latency and reaction time, Science, 211, 77-80. McDonald, J.J., Teder-Salajarvi, W.A., & Hillyard, S.A. (2000). Involuntary orienting to sound improves visual perception, Nature, 407, 906-908. McFarland, D.J., Lefkowicz, A.T., & Wolpaw, J.R. (1997). Design and operation of an EEG-based brain-computer interface (BCI) with digital signal processing technology. Behavioral Research Methods, Instruments, and Computers, 29, 337-345. Miller, G.A., Bashore, T.R., Farwell, L.A., & Donchin, E. (1987). Chapter 1: Research in geriatric psychophysiology. In K.W. Schaie, & C. Eisdorfer, (Eds.), Annual Review of Gerontology and Geriatrics, Vol. 7, (pp. 1-27). New York: Springer. Pan, J., Takeshita, T., & Morimoto, K. (2000). P300 habituation from auditory single-stimulus and oddball paradigms. International Journal of Psychophysiology, 37, 149-153. 70

PAGE 81

Pfurtscheller, G., Flotzinger, D., Pregenzer, M., Wolpaw, J.R., & McFarland, D. (1996). EEG-based brain computer interface. Medical Progress Through Technology, 21, 111-121. Pfurtscheller, G., & Neuper, C. (1997). Motor imagery activates primary sensorimotor area in humans. Neuroscience Letters, 239, 65-68. Pfurtscheller, G. et al. (2000). Current trends in Graz brain-computer interface (BCI) research. IEEE Transactions on Rehabilitation Engineering, 8, 216-219. Polich, J. (1986). Normal variation of P300 from auditory stimuli. Electroencephalography and Clinical Neurophysiology, 65, 236-240. Polich, J. (1989). Habituation of P300 from auditory stimuli. Psychobiology, 17, 19-28. Pooviboonsuk, P., Dalton, J. A., Curran, H. V., Lader, M. H. (1996). The effects of single doses of lorazepam on event-related potentials and cognitive function. Human Psychopharmacology, 11, 241-252. Posner, M.I., (1980). The orienting of attention. Quarterly Journal of Experimental Psychology, 32, 3-25. 71

PAGE 82

Pritchard, W. (1981). The psychophysiology of P300. Psychological Bulletin, 89, 506-540. Ravden, D., & Polich, J. (1998). Habituation of P300 from visual stimuli. International Journal of Psychophysiology, 30, 359-365. Schalk, G., Wolpaw, J. R., McFalrland, D. J., & Pfurtscheller, G. (2000). EEG-based communication: presence of an error potential. Clinical Neurophysiology, 111, 2138-2144. Schalk, et al. (2004). BCI2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering, 51, 1034-1043. Sellers, E., Schalk, G., & Donchin, E. (2003). The P300 as a typing tool: Tests of brain computer interface with an ALS patient. Psychophysiology, 40 (suppl. 1), S77. Sirevaag, E.J., Kramer, A.F., Coles, M.G.H., & Donchin, E. (1989). Resource reciprocity: An event-related brain potentials analysis. Acta Psychologica, 70, 77-97. Squires, N.K., Wickens, C., Squires, K.C., & Donchin, E. (1976). The effect of stimulus sequence on the waveform of the cortical event-related potential. Science, 193, 1142-1146. 72

PAGE 83

Squires, N.K., Donchin, E., Squires, K.C., & Grossberg, S. (1977). Bisensory stimuluation: Inferring decision-related processes from the P300 component. Journal of Experimental Psychology: Human Perception and Performance, 3, 299-315. Squires, K., Petuchowski, S., Wickens, C. & Donchin, E. (1977). The effects of stimulus sequence on event related potentials: A comparison of visual and auditory sequences. Perception & Psychophysics, 22, 31-40. Sutton, S., Braren, M., Zubin, J., & John, E.R. (1965). Evoked-potential correlates of stimulus uncertainty. Science, 150, 1187-1188. Teder-Salejarvi, W.A., McDonald, J.J., Di Russo, F., & Hillyard, S.A. (2002). An analysis of audio-visual crossmodal integration by means of event-related potential (ERP) recordings. Cognitive Brain Research, 14, 106-114. Wickens, C.D., Heffley, E.F., Kramer, A.F., & Donchin, E. (1980). The event-related brain potential as an index of attention allocation in complex displays. Proceedings of the Human Factors Society, 24 th Annual Meeting, (pp. 297-301). Santa Monica: Human Factors Society. Wintink, A.J., Segalowitz, S.J., & Cudmore, L.J. (2001). Task complexity and habituation effects on frontal P300 topography. Brain and Cognition, 46, 307-311. 73

PAGE 84

Wood, C.C., & Allison, T. (1981). Interpretation of evoked potentials: A neurophysiological perspective. Canadian Journal of Psychology, 35, 113-135. Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., & Vaughan, T.M. (2002). Brain-computer interfaces for communication and control. Clinical Neurophysiology, 113, 767-791. Wolpaw, J.R., McFarland, D.J., & Vaughan, T.M., (2000). Brain-computer interface research at the Wadsworth Center. IEEE Transactions on Rehabilitation Engineering, 8, 222-225. Yantis, S., Schwarzbach, J., Serences, J.T., Carlson, R.L., Steinmetz, M.A., Pekar, J.J., & Courtney, S.M. (2002). Transient neural activity in human parietal cortex during spatial attention shifts. Nature Neuroscience, 5, 995-1002. 74

PAGE 85

Appendices 75

PAGE 86

76 Appendix A: Classification Results Non-ALS Participants Participant 1 Session 3 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 50.0 60.3 59.2 56.7 57.1 56.6 55.3 55.8 54.3 54.0 53.4 1Ses 45.6 61.8 64.2 68.7 69.4 68.8 69.0 67.9 68.2 69.0 69.2 2Ses 44.5 57.8 63.0 70.8 71.4 70.2 69.7 71.0 68.8 69.1 70.2 Visual 1Run 63.4 79.5 84.7 87.9 89.7 91.9 91.0 92.4 93.1 93.8 94.7 1Ses 63.5 79.3 87.0 88.5 91.1 93.0 94.8 96.4 97.4 97.7 97.6 2Ses 56.1 80.3 88.1 90.8 93.9 95.8 96.7 98.3 98.2 99.0 99.2 Aud+Vis 1Run 55.7 74.3 80.2 83.0 88.7 90.7 89.9 93.9 93.6 94.1 95.1 1Ses 53.6 76.9 87.2 92.4 95.4 95.2 96.8 97.3 98.7 98.4 99.2 2Ses 45.8 68.1 77.1 82.4 86.7 87.6 91.5 91.2 93.4 94.6 94.7 Session4 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 36.3 40.2 42.6 46.1 42.2 48.6 50.3 48.8 49.7 50.7 48.5 1Ses 40.3 49.5 54.7 60.3 60.7 60.7 63.2 63.5 62.9 63.9 62.7 2Ses 39.3 49.0 51.9 51.3 51.2 50.3 51.0 50.6 50.6 50.3 50.2 Visual 1Run 57.2 77.1 83.0 87.6 89.6 92.9 94.1 95.2 95.5 96.4 96.4 1Ses 48.5 70.4 79.1 84.4 88.8 90.7 94.8 94.7 95.8 95.3 97.2 2Ses 49.6 71.5 81.1 87.1 89.2 90.8 94.4 95.7 95.8 95.1 96.6 Aud+Vis 1Run 39.7 56.4 65.0 65.6 71.1 76.1 76.3 78.2 79.0 81.0 81.9 1Ses 29.6 36.9 40.8 46.6 44.8 49.1 48.0 48.2 50.2 52.4 55.0 2Ses 38.4 48.3 55.7 59.6 64.9 66.7 67.2 68.9 73.0 73.6 72.6 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 33.3 35.9 33.8 34.1 35.0 36.4 37.1 37.3 36.8 38.3 39.3 1Ses 29.4 33.8 32.1 31.6 32.6 31.7 31.9 31.9 32.2 29.8 30.5 2Ses 34.6 42.4 47.5 47.7 52.2 50.5 52.7 54.6 53.4 54.7 54.3 Visual 1Run 48.7 61.7 69.1 74.1 75.5 78.2 82.4 80.4 81.4 83.6 82.9 1Ses 45.8 61.3 74.5 77.9 84.7 88.1 87.8 90.7 92.9 93.6 95.0 2Ses 49.4 74.4 83.3 88.9 91.7 95.6 96.7 97.7 98.5 98.6 98.9 Aud+Vis 1Run 44.7 68.8 73.9 76.9 79.9 80.9 82.8 81.0 83.8 83.0 84.0 1Ses 43.1 56.7 63.1 70.2 73.5 76.3 75.3 78.8 80.4 81.5 81.7 2Ses 47.7 65.3 74.7 83.7 84.7 88.2 89.5 91.9 93.1 94.5 95.0

PAGE 87

77 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 45.6 62.1 67.9 67.8 71.4 72.0 72.5 70.7 74.1 73.4 73.6 1Ses 35.4 50.0 56.9 60.5 63.9 66.6 70.8 72.3 73.3 75.2 77.0 2Ses 43.2 52.7 62.2 61.4 63.5 65.4 70.9 68.9 69.4 69.6 70.6 Visual 1Run 28.4 33.1 37.0 43.6 43.5 44.0 43.0 43.4 45.5 47.9 49.2 1Ses 35.9 47.8 57.8 64.2 67.6 72.6 76.1 77.3 81.5 80.9 82.4 2Ses 45.8 67.8 78.3 80.3 88.0 89.8 89.8 91.2 93.9 94.8 95.2 Aud+Vis 1Run 25.0 29.4 27.7 29.9 26.8 27.9 27.4 28.0 29.6 27.8 30.5 1Ses 41.0 52.1 63.1 64.4 67.3 71.1 72.9 75.6 76.6 80.1 80.5 2Ses 47.5 62.8 72.1 73.4 77.0 75.7 79.7 79.3 75.4 79.8 79.9 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 38.5 46.5 53.3 51.0 53.1 53.6 52.9 51.3 52.6 57.5 55.0 1Ses 38.9 40.2 44.4 47.3 50.3 52.5 54.9 51.4 52.2 54.6 56.5 2Ses 33.2 39.1 41.3 43.1 43.2 45.0 45.6 45.0 44.7 46.8 46.9 Visual 1Run 61.8 84.5 93.1 96.6 98.8 98.4 99.4 99.8 99.7 99.9 99.8 1Ses 59.1 79.2 85.9 88.8 93.4 95.7 96.4 97.5 97.5 98.9 98.3 2Ses 59.6 81.4 89.7 94.3 96.3 97.7 98.8 98.9 99.3 99.4 99.8 Aud+Vis 1Run 55.5 78.9 85.2 90.2 92.9 94.9 96.4 96.6 98.6 98.1 99.0 1Ses 38.7 50.5 59.8 65.1 63.0 66.8 73.6 75.6 76.5 76.1 77.5 2Ses 44.2 57.1 62.0 69.4 71.0 71.2 73.2 76.4 74.1 73.5 76.3 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 43.9 53.1 61.0 63.7 64.5 65.6 65.3 66.3 67.6 67.7 67.9 1Ses 31.1 33.3 33.4 28.0 30.6 28.2 27.7 25.8 23.5 26.6 22.1 2Ses 42.6 55.7 60.6 63.0 66.4 66.2 67.9 66.9 68.6 69.8 69.2 Visual 1Run 46.0 65.0 74.1 80.1 83.3 85.8 87.4 89.5 89.3 93.6 94.1 1Ses 59.3 74.0 82.2 86.7 85.3 90.0 89.1 91.8 92.3 92.6 92.7 2Ses 63.3 79.2 87.3 90.8 91.5 94.3 94.8 94.8 96.4 97.6 97.3 Aud+Vis 1Run 57.6 83.2 93.4 96.3 97.5 99.1 99.4 99.6 99.8 99.8 100.0 1Ses 51.5 77.0 85.4 90.5 95.2 95.4 97.6 97.7 98.2 99.0 98.6 2Ses 58.7 85.3 93.5 97.2 98.2 99.1 99.5 99.8 99.9 99.9 100.0

PAGE 88

78 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 51.9 72.0 83.0 86.3 91.5 92.0 92.6 95.2 96.1 96.0 96.8 1Ses 51.0 63.9 74.6 79.3 81.3 82.0 87.1 87.0 86.7 85.0 86.8 2Ses 46.7 54.9 62.1 63.8 65.5 66.8 69.4 66.6 70.6 70.8 69.2 Visual 1Run 29.5 27.7 26.0 27.2 26.1 29.7 25.6 25.7 27.3 26.7 26.0 1Ses 50.4 68.1 74.3 80.2 85.5 88.4 90.4 94.2 95.5 94.8 96.1 2Ses 49.9 68.2 69.7 77.6 78.4 83.0 84.4 84.8 87.6 87.9 89.0 Aud+Vis 1Run 39.2 46.5 50.4 53.0 51.4 52.4 51.8 51.6 51.7 52.5 50.8 1Ses 54.6 78.7 85.0 91.9 93.9 97.0 97.8 98.8 98.9 99.3 99.5 2Ses 47.9 71.2 79.8 86.3 89.0 92.9 94.0 97.3 97.9 97.3 98.6 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 45.8 65.4 70.5 73.5 75.9 76.9 80.1 80.3 79.7 83.4 83.1 1Ses 50.7 68.2 74.9 81.7 86.8 88.1 90.7 92.2 94.2 95.6 96.0 2Ses 48.6 64.0 67.6 72.0 77.8 78.8 79.6 82.9 84.4 85.2 85.4 Visual 1Run 62.9 88.1 94.7 98.3 98.7 99.5 100.0 99.9 100.0 99.9 100.0 1Ses 36.6 49.3 57.7 60.5 64.7 70.5 77.0 77.4 78.4 80.2 81.4 2Ses 59.1 81.5 88.9 93.7 95.5 96.8 98.0 99.0 99.4 99.5 99.8 Aud+Vis 1Run 45.9 59.7 66.7 70.7 75.7 77.8 79.2 81.6 81.9 81.1 83.2 1Ses 46.6 65.8 75.7 82.3 88.3 90.7 93.6 95.5 94.6 96.9 97.8 2Ses 53.5 73.3 85.8 91.1 94.0 97.5 97.8 99.2 98.8 99.6 99.9 Participant 2 Session 3 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 53.3 72.1 84.2 91.1 93.5 96.0 96.9 97.6 97.0 99.0 98.5 1Ses 33.8 39.5 46.7 49.8 51.2 51.1 54.5 52.9 54.1 53.2 52.4 2Ses 35.1 45.4 49.7 51.7 54.3 58.6 59.4 57.6 60.9 60.2 58.9 Visual 1Run 58.1 82.9 93.5 96.9 99.2 99.2 99.9 99.8 100.0 99.9 100.0 1Ses 41.4 58.2 67.1 76.0 82.0 86.1 88.7 91.5 93.6 93.4 95.3 2Ses 56.6 79.5 89.8 95.3 97.2 98.2 99.6 99.2 99.7 99.5 100.0 Aud+Vis 1Run 36.1 40.0 44.2 48.1 50.3 51.7 53.6 52.9 53.4 54.5 57.2 1Ses 33.1 52.4 64.6 68.9 73.6 80.0 83.3 87.5 89.5 89.5 92.2 2Ses 49.9 62.5 67.7 71.8 79.1 80.6 83.7 85.2 88.7 90.1 91.1

PAGE 89

79 Session4 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 33.1 38.5 42.3 44.5 45.9 48.5 47.5 50.8 50.2 49.0 50.6 1Ses 35.7 48.9 53.9 58.1 62.8 63.8 67.9 69.6 74.2 68.2 74.0 2Ses 38.1 54.3 60.0 64.7 69.4 74.1 77.7 79.6 83.1 83.3 86.4 Visual 1Run 31.4 37.7 36.3 39.2 40.8 41.6 41.7 43.8 44.9 45.8 44.6 1Ses 28.1 31.6 31.7 30.8 31.7 31.5 33.9 34.5 33.9 33.7 35.5 2Ses 34.6 40.6 42.2 45.4 47.4 47.2 54.6 55.8 53.9 55.8 57.9 Aud+Vis 1Run 31.6 38.7 43.7 42.7 49.0 50.6 49.9 52.5 57.2 56.1 57.8 1Ses 37.5 56.6 60.6 63.8 69.2 74.6 76.2 81.8 82.0 84.5 86.9 2Ses 26.3 31.3 31.2 29.6 31.9 32.5 35.9 35.5 34.3 37.0 34.6 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 41.7 57.7 63.1 69.8 73.3 73.8 74.6 75.8 76.4 78.2 78.6 1Ses 27.5 32.6 35.8 36.6 42.6 44.1 48.2 46.1 46.7 49.3 50.9 2Ses 35.4 49.7 59.5 62.8 67.1 69.7 71.2 73.8 76.8 75.3 77.7 Visual 1Run 36.6 45.1 50.2 52.6 52.2 51.0 52.9 52.1 52.7 52.1 51.7 1Ses 31.1 37.7 45.4 50.3 53.2 56.8 59.7 63.7 64.3 66.9 68.3 2Ses 42.4 59.4 70.8 78.7 86.1 89.5 91.2 91.9 93.9 96.1 97.2 Aud+Vis 1Run 44.9 54.2 61.9 63.2 69.0 69.6 71.9 72.8 76.5 74.1 75.4 1Ses 44.1 56.9 68.9 74.7 78.8 83.4 86.9 86.8 89.5 91.3 93.0 2Ses 44.5 62.2 74.1 79.7 84.7 86.8 91.7 91.3 94.4 95.7 97.1 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 46.6 61.4 68.0 76.1 80.5 79.5 83.2 84.1 85.2 88.7 86.6 1Ses 33.2 45.1 48.4 51.2 56.4 57.3 60.9 62.5 62.0 62.1 65.5 2Ses 39.4 51.0 60.6 64.7 71.5 71.7 74.5 75.9 77.5 80.3 82.5 Visual 1Run 39.3 47.3 51.7 56.1 64.2 64.0 69.0 69.4 73.4 74.1 77.6 1Ses 40.0 61.1 71.1 77.8 81.4 83.6 88.2 89.9 90.4 92.0 92.7 2Ses 28.0 34.6 36.2 39.8 42.0 39.7 45.8 45.3 47.8 50.4 47.6 Aud+Vis 1Run 34.6 38.5 46.2 45.1 48.7 49.9 52.5 52.3 50.5 52.1 53.4 1Ses 31.0 38.6 43.6 45.7 44.9 51.0 53.0 50.4 50.5 52.9 54.6 2Ses 32.5 39.8 46.6 47.4 45.1 52.5 54.5 53.5 53.1 55.7 56.4

PAGE 90

80 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 33.3 36.9 40.2 39.4 44.6 49.7 47.5 50.2 49.7 51.8 52.6 1Ses 31.1 43.5 48.9 47.2 53.3 56.1 54.9 58.9 59.2 58.5 60.6 2Ses 37.9 48.0 50.6 52.1 54.8 59.4 60.2 60.4 66.2 65.0 67.1 Visual 1Run 52.1 77.6 87.1 90.9 94.5 96.0 97.5 97.5 98.1 99.1 99.0 1Ses 44.6 62.0 74.3 81.2 87.9 90.6 92.3 95.2 95.9 96.8 98.6 2Ses 50.2 75.5 88.1 93.0 96.0 97.9 98.4 99.1 99.5 99.8 99.4 Aud+Vis 1Run 42.5 60.8 65.8 68.5 67.4 70.9 69.8 71.0 72.2 72.2 74.4 1Ses 50.1 75.3 84.8 90.7 92.7 96.1 96.6 97.0 98.6 99.4 99.2 2Ses 47.6 71.3 82.6 88.1 93.1 95.2 96.5 97.9 97.3 98.8 98.9 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 37.2 45.7 52.2 57.3 61.9 66.8 68.5 68.5 71.9 74.2 73.2 1Ses 31.6 38.9 39.2 44.1 45.5 46.9 52.2 49.6 51.8 51.6 53.5 2Ses 31.5 46.1 50.1 56.4 62.1 67.0 67.5 67.4 71.1 71.2 75.7 Visual 1Run 39.5 50.8 60.0 65.7 69.3 76.3 77.7 79.3 81.0 84.6 85.6 1Ses 39.5 53.1 66.3 70.3 77.8 81.5 82.9 87.2 89.8 89.9 92.4 2Ses 45.4 61.5 75.8 82.1 87.6 90.4 92.8 95.0 96.9 97.5 98.6 Aud+Vis 1Run 40.3 59.4 68.1 69.5 74.7 75.2 77.2 80.5 81.5 78.5 82.0 1Ses 33.9 43.4 47.8 50.1 55.2 55.3 59.1 62.6 63.9 65.8 68.2 2Ses 38.1 52.6 59.3 69.5 73.2 71.9 74.7 79.2 81.2 81.8 84.6 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 40.3 51.7 58.8 64.5 65.4 67.1 65.3 67.2 65.2 67.9 66.7 1Ses 33.3 44.0 50.8 54.2 56.6 59.2 61.1 65.4 67.6 69.1 72.1 2Ses 36.9 51.0 57.8 64.2 66.1 71.7 76.1 78.3 81.5 81.7 83.6 Visual 1Run 44.6 71.4 80.3 88.9 93.5 94.3 95.3 97.0 98.6 98.9 99.0 1Ses 33.3 44.3 52.6 58.1 62.1 66.3 67.9 72.1 75.2 75.3 76.1 2Ses 38.1 50.9 63.0 70.6 75.1 79.2 82.0 84.0 87.8 90.8 87.7 Aud+Vis 1Run 33.7 40.0 40.9 41.3 43.5 46.0 47.4 47.8 46.3 46.6 48.3 1Ses 32.5 47.0 51.6 55.3 60.0 64.0 66.8 68.0 70.8 68.6 71.0 2Ses 35.4 42.3 47.0 50.6 51.2 57.8 58.3 60.7 60.9 60.7 59.4

PAGE 91

81 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 30.4 37.7 39.9 43.9 42.8 47.9 48.5 50.2 53.0 53.1 53.9 1Ses 33.2 39.9 44.0 46.4 48.2 48.2 49.6 50.9 51.1 52.3 51.2 2Ses 33.8 40.2 42.9 43.2 45.1 46.1 48.4 48.8 49.3 50.4 50.0 Visual 1Run 34.6 39.9 40.5 45.3 43.7 47.2 48.9 50.0 50.3 50.8 51.0 1Ses 35.5 39.3 45.0 49.5 50.2 53.4 55.7 58.5 56.5 60.1 57.7 2Ses 36.3 46.5 55.2 59.3 59.8 61.0 63.1 67.2 65.4 66.7 64.1 Aud+Vis 1Run 42.2 56.5 62.9 65.1 67.1 69.5 66.1 68.0 69.1 68.5 68.4 1Ses 39.1 59.5 62.4 68.0 70.2 71.7 72.2 75.5 75.9 77.4 77.2 2Ses 37.6 58.8 69.2 76.7 79.8 84.0 84.5 86.8 90.9 89.5 91.2 Participant 3 Session 3 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 37.5 52.1 56.3 66.4 69.6 71.7 76.9 80.3 82.7 82.9 85.8 1Ses 35.8 44.2 47.7 52.7 52.9 58.2 56.9 58.9 60.2 61.7 62.0 2Ses 37.8 49.4 51.4 55.4 54.2 59.7 60.1 63.3 65.2 62.6 64.0 Visual 1Run 44.5 58.4 63.7 65.4 70.4 71.7 70.9 72.5 73.4 72.9 71.1 1Ses 33.4 46.3 51.0 54.4 53.5 57.7 54.7 55.1 57.1 56.3 56.1 2Ses 35.1 48.8 54.5 57.8 57.1 60.9 59.6 61.3 62.0 62.4 59.4 Aud+Vis 1Run 46.0 65.8 75.2 82.3 86.4 88.4 91.0 93.5 95.1 96.1 95.2 1Ses 46.0 59.0 69.2 76.3 79.6 81.3 85.2 88.3 88.3 90.7 89.9 2Ses 45.4 63.5 71.0 79.0 82.2 84.9 88.4 90.0 90.0 91.5 92.8 Session4 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 31.4 31.4 41.5 41.4 44.9 44.9 47.7 47.7 48.4 49.9 49.1 1Ses 25.1 32.3 29.5 30.4 33.6 35.2 32.1 34.1 28.1 31.1 29.1 2Ses 31.7 37.5 39.6 44.2 47.5 48.0 53.2 51.4 50.0 54.5 52.0 Visual 1Run 28.0 28.5 30.6 28.1 28.9 30.6 31.1 33.2 30.1 33.4 32.0 1Ses 26.6 32.3 34.4 36.2 36.8 41.3 40.3 43.3 41.7 42.1 43.9 2Ses 31.8 39.5 42.7 45.8 49.5 49.2 53.1 51.2 54.6 53.9 57.6 Aud+Vis 1Run 40.7 49.0 52.8 55.5 55.8 57.4 55.8 54.2 55.4 54.6 54.7 1Ses 35.6 48.7 57.0 60.7 64.5 69.5 67.9 71.2 73.3 76.1 78.6 2Ses 42.9 51.0 54.5 60.5 62.3 64.2 62.8 65.9 65.6 69.0 70.7

PAGE 92

82 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 34.1 40.9 46.3 48.6 49.4 51.0 49.7 56.2 55.3 55.4 57.7 1Ses 27.4 30.6 32.9 32.6 32.8 36.0 33.1 33.6 36.0 37.1 37.8 2Ses 32.9 33.1 38.2 37.2 38.3 39.0 39.6 38.6 43.3 42.5 41.7 Visual 1Run 27.3 30.7 28.5 31.3 31.2 29.9 31.8 30.0 28.8 31.6 33.0 1Ses 29.1 33.0 36.7 39.3 43.2 43.8 47.3 51.8 49.1 52.9 51.2 2Ses 38.5 47.9 52.3 56.7 62.9 66.2 67.2 69.3 69.3 71.4 71.4 Aud+Vis 1Run 37.7 46.2 53.8 57.6 63.4 67.6 69.2 69.4 74.5 77.2 76.4 1Ses 35.4 50.3 60.9 66.0 69.2 74.0 72.6 74.5 77.1 80.2 82.2 2Ses 41.1 51.3 57.6 62.0 64.1 65.3 67.6 66.8 67.4 70.7 72.8 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 33.6 47.5 53.4 58.4 63.7 63.8 64.8 67.1 67.5 70.0 68.4 1Ses 28.0 30.1 30.2 30.1 34.3 33.8 33.9 33.9 36.1 37.0 38.8 2Ses 32.3 38.4 39.6 42.6 46.1 44.7 47.6 51.8 50.9 51.7 55.0 Visual 1Run 30.9 35.2 35.3 41.3 43.6 46.2 47.2 46.8 47.5 49.7 49.2 1Ses 31.0 32.0 33.7 37.0 39.2 40.3 39.5 39.5 43.9 44.5 43.3 2Ses 37.8 47.0 55.1 57.9 65.2 70.7 65.5 72.7 71.2 76.1 76.5 Aud+Vis 1Run 40.8 53.5 55.9 56.1 54.5 53.9 54.7 53.8 53.4 53.1 52.9 1Ses 36.2 45.3 50.3 52.5 51.7 50.6 51.6 51.6 52.5 50.6 51.7 2Ses 49.0 60.7 64.9 67.2 65.3 64.2 64.2 61.3 63.3 61.1 62.2 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 33.0 35.1 35.2 36.6 40.7 39.8 40.1 43.3 39.5 41.0 43.7 1Ses 35.0 36.9 42.3 46.7 45.6 46.5 49.2 52.5 50.5 52.7 50.8 2Ses 26.9 28.6 29.4 32.5 33.9 34.9 31.2 33.7 34.5 36.6 37.9 Visual 1Run 40.0 53.4 56.3 59.5 65.3 62.5 66.1 67.0 66.4 70.9 67.3 1Ses 27.8 33.3 36.6 36.6 40.4 43.1 44.9 42.7 42.4 45.5 46.1 2Ses 30.8 45.1 54.0 55.2 60.0 61.2 67.6 66.5 68.9 70.6 72.7 Aud+Vis 1Run 41.7 55.2 64.5 67.8 72.2 76.2 76.8 77.8 82.5 82.7 83.0 1Ses 42.2 57.5 68.7 70.4 72.0 77.9 77.0 77.9 80.9 83.3 82.3 2Ses 46.7 66.0 76.3 81.3 83.6 88.3 91.3 91.3 91.9 94.6 94.8

PAGE 93

83 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 28.2 30.6 34.0 34.1 37.2 37.2 39.0 40.1 40.8 41.4 42.3 1Ses 31.8 37.3 41.1 42.7 42.3 45.6 46.3 44.1 46.2 48.5 48.1 2Ses 32.2 42.8 45.9 46.0 45.9 51.2 50.0 52.2 50.2 51.9 52.5 Visual 1Run 31.5 33.3 35.5 36.8 35.9 35.7 37.4 39.4 39.7 39.3 39.9 1Ses 31.1 31.5 30.7 37.2 39.7 41.3 41.0 41.9 42.8 41.0 45.4 2Ses 30.2 33.8 35.5 40.4 42.6 44.9 44.6 44.7 46.0 47.8 48.9 Aud+Vis 1Run 29.9 34.6 39.9 45.1 45.8 48.8 48.6 51.2 49.7 51.8 52.8 1Ses 27.1 34.7 32.9 34.4 35.0 37.9 37.7 37.8 40.0 39.1 42.2 2Ses 39.0 40.2 43.5 44.8 47.6 47.1 46.5 45.8 47.4 45.0 45.6 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 48.4 53.8 55.1 54.5 55.9 55.3 54.3 54.6 52.5 51.8 51.4 1Ses 44.6 58.8 66.2 70.4 75.5 77.4 78.4 84.4 84.5 84.6 88.5 2Ses 49.5 61.8 70.6 74.8 79.8 82.2 84.5 84.8 87.6 86.5 91.3 Visual 1Run 33.8 42.7 44.7 49.0 49.7 49.8 51.0 51.9 52.0 51.6 50.6 1Ses 26.7 23.9 20.8 20.2 20.9 20.4 19.5 20.7 20.4 25.3 21.6 2Ses 31.3 35.5 37.2 40.4 41.7 42.8 44.4 47.4 46.3 47.3 46.3 Aud+Vis 1Run 42.6 52.2 64.1 66.2 76.2 77.7 80.5 82.6 82.1 85.9 88.5 1Ses 45.9 70.4 81.2 84.6 91.0 93.5 95.3 95.9 97.3 98.2 97.6 2Ses 49.0 69.1 79.0 86.3 90.1 91.0 93.1 94.9 95.0 97.6 97.0 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 42.2 53.2 57.4 56.5 57.6 57.9 55.7 56.5 55.3 56.9 56.2 1Ses 33.6 44.8 54.1 62.2 62.2 64.3 67.9 71.1 72.3 73.4 76.2 2Ses 39.5 51.8 60.4 65.5 69.7 71.7 74.8 79.3 79.5 80.6 83.1 Visual 1Run 31.6 34.0 35.5 37.2 37.5 42.6 41.1 44.2 45.1 44.9 46.7 1Ses 27.6 33.3 32.1 32.9 35.8 39.7 38.9 41.1 37.8 37.4 38.7 2Ses 26.8 29.6 32.0 31.8 32.5 35.6 36.8 36.2 36.2 38.5 38.2 Aud+Vis 1Run 44.0 54.2 59.8 63.0 60.9 60.9 62.4 61.2 61.3 59.7 59.6 1Ses 25.1 32.6 32.8 33.5 35.9 37.0 39.7 36.6 39.2 42.8 37.9 2Ses 33.8 42.0 43.4 47.2 47.3 48.8 49.5 48.8 49.8 50.2 49.9

PAGE 94

84 ALS Participants Participant 1 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 49.8 69.5 80.6 82.5 84.3 85.5 87.3 89.1 89.6 91.9 91.9 1Ses 51.4 70.7 82.9 86.4 87.7 91.1 91.6 94.1 95.4 97.2 97.4 2Ses 49.0 66.9 80.9 83.4 88.1 90.2 91.2 93.7 95.4 95.9 96.4 Visual 1Run 41.0 52.1 54.5 57.9 58.1 56.7 57.0 57.5 57.8 56.9 56.7 1Ses 37.5 46.8 48.9 49.2 51.2 50.1 51.1 51.7 51.6 51.0 51.1 2Ses 26.8 33.1 33.1 34.3 31.7 35.0 35.4 35.4 37.8 36.5 38.4 Aud+Vis 1Run 45.1 57.1 61.9 64.2 65.0 63.9 65.1 65.0 65.3 64.3 64.4 1Ses 38.5 42.8 43.3 47.1 47.3 52.4 53.0 52.5 53.2 53.3 52.5 2Ses 33.4 46.6 46.6 50.7 50.4 54.0 51.9 53.7 54.7 53.2 54.5 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 40.8 50.7 59.9 64.9 66.8 68.6 69.2 70.7 71.4 72.6 74.8 1Ses 35.3 41.6 48.7 54.7 59.2 62.3 65.0 69.0 68.8 73.0 71.5 2Ses 40.3 54.7 62.5 68.7 72.9 73.3 76.7 79.7 81.6 83.7 84.4 Visual 1Run 51.5 70.6 74.9 80.4 82.2 83.0 84.3 86.4 88.2 88.9 88.9 1Ses 30.3 41.0 46.7 51.5 51.1 53.3 54.7 55.1 54.8 54.8 55.6 2Ses 42.1 61.3 66.5 74.8 80.0 82.8 86.0 89.8 91.3 90.1 91.4 Aud+Vis 1Run 45.9 63.6 72.4 76.2 80.6 80.9 84.5 87.6 86.9 86.4 88.6 1Ses 27.8 33.2 35.1 39.8 39.1 40.7 42.0 42.2 42.1 45.8 45.5 2Ses 30.9 41.3 46.1 52.0 54.8 60.2 60.4 60.5 63.1 64.4 66.1 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 32.5 42.6 46.2 50.3 54.7 53.0 55.1 55.2 55.6 56.7 58.7 1Ses 29.2 32.8 32.5 35.5 32.9 33.5 37.8 35.6 35.5 33.5 36.3 2Ses 36.6 47.2 54.5 56.3 61.0 64.0 66.7 68.1 75.3 71.6 75.5 Visual 1Run 33.7 42.6 48.8 49.5 55.2 54.0 61.0 62.8 63.1 65.7 65.1 1Ses 28.8 33.2 34.8 37.2 38.1 42.0 40.5 40.6 39.7 41.7 42.6 2Ses 29.5 28.6 28.8 28.6 25.9 27.8 27.6 27.5 28.9 25.7 24.6 Aud+Vis 1Run 31.5 37.2 40.9 44.1 45.7 49.2 48.9 48.5 50.1 49.4 50.7 1Ses 37.1 46.7 55.8 60.2 62.2 66.1 66.8 68.8 72.7 68.6 72.0 2Ses 28.5 34.2 34.6 34.0 36.9 39.1 35.2 37.1 37.4 39.2 38.5

PAGE 95

85 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 41.7 56.8 60.5 66.4 68.1 71.9 72.6 70.8 76.2 75.9 75.6 1Ses 27.9 30.0 29.6 30.1 32.1 30.7 31.1 31.0 28.0 30.6 30.4 2Ses 32.7 44.2 46.6 51.1 54.1 55.7 58.6 60.1 64.6 65.5 66.6 Visual 1Run 40.7 49.0 55.6 61.1 61.4 69.2 67.2 66.5 70.8 71.7 71.9 1Ses 31.0 37.9 39.1 40.2 41.9 43.9 48.2 49.1 51.4 49.3 53.0 2Ses 34.9 42.7 48.6 53.8 53.3 58.7 62.4 66.1 68.3 66.5 71.7 Aud+Vis 1Run 32.1 39.0 40.8 41.0 46.1 46.8 49.0 49.8 48.6 48.8 49.3 1Ses 30.6 37.3 39.8 45.2 43.0 45.7 49.4 48.4 51.5 51.7 52.1 2Ses 31.3 37.7 39.0 41.1 41.4 45.3 45.7 45.9 49.0 51.1 46.8 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 30.5 31.1 31.1 32.2 32.7 34.9 35.8 37.9 37.0 36.6 36.7 1Ses 35.8 45.4 51.6 56.8 63.5 66.5 67.4 70.9 72.4 73.5 78.1 2Ses 36.1 51.3 58.3 66.9 71.9 73.3 75.1 78.5 81.3 82.8 82.2 Visual 1Run 41.2 52.8 62.2 66.3 71.2 76.2 78.5 84.1 85.0 88.0 87.6 1Ses 28.2 32.4 35.8 35.5 37.0 33.6 36.5 37.7 37.4 37.6 37.5 2Ses 31.3 37.7 41.3 43.6 45.9 45.1 49.4 52.1 51.3 54.8 55.9 Aud+Vis 1Run 35.6 44.4 50.3 54.5 60.6 61.2 63.3 68.0 67.4 68.6 70.3 1Ses 28.8 27.8 32.3 34.0 34.2 35.7 37.8 37.1 39.3 39.3 39.5 2Ses 31.1 34.5 32.5 40.0 40.7 40.1 39.5 43.5 45.4 45.0 44.7 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 35.7 41.3 47.4 47.3 51.3 50.9 51.1 52.9 52.0 53.3 54.6 1Ses 26.2 26.3 31.1 29.6 32.9 28.9 32.7 32.7 31.3 34.3 32.7 2Ses 32.8 44.6 51.4 56.5 60.1 66.2 67.5 70.3 72.9 74.5 72.6 Visual 1Run 34.5 41.5 47.5 53.1 56.8 59.4 63.3 66.4 70.2 70.3 72.3 1Ses 43.0 59.6 66.6 73.6 78.9 82.5 87.6 87.7 91.2 91.5 93.3 2Ses 43.2 63.6 75.4 80.1 85.2 87.8 92.2 94.5 94.9 96.0 97.4 Aud+Vis 1Run 41.2 50.6 55.0 59.7 59.2 63.0 64.2 65.6 65.0 66.7 66.2 1Ses 38.1 47.6 53.5 62.0 67.2 67.4 68.5 69.8 75.0 76.6 77.1 2Ses 38.4 49.0 55.2 60.5 65.4 69.9 72.2 73.3 72.7 79.2 79.5

PAGE 96

86 2Ses 32.9 42.3 44.5 46.8 51.8 51.4 57.0 56.5 59.7 62.0 63.3 Visual 1Run 44.9 56.6 57.2 61.7 63.3 63.7 62.8 65.5 62.8 64.1 61.7 1Ses 37.3 43.7 49.6 57.9 58.8 61.5 68.9 71.4 69.3 74.3 75.8 2Ses 44.8 57.3 64.7 70.4 74.3 76.6 80.8 84.7 85.2 87.0 88.2 Aud+Vis 1Run 34.0 48.9 56.3 60.6 62.2 66.8 71.6 74.7 75.5 76.2 76.6 1Ses 38.8 56.3 61.3 67.7 66.7 68.6 70.7 70.3 69.0 71.0 69.6 2Ses 42.9 53.3 61.0 63.4 67.1 67.1 69.1 68.8 67.7 71.3 69.7 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 30.2 40.7 46.2 43.4 46.8 48.7 49.5 50.8 50.4 50.3 50.4 1Ses 26.0 37.7 37.5 42.1 45.6 46.4 49.5 47.7 52.0 54.5 55.7 2Ses 30.3 39.3 39.7 41.9 46.6 47.2 51.7 50.7 55.6 56.1 56.0 Visual 1Run 30.5 44.1 46.7 50.5 50.0 50.6 49.8 52.2 51.0 51.4 51.0 1Ses 22.9 30.3 31.1 33.1 32.5 33.4 35.4 35.7 33.7 33.7 36.2 2Ses 28.4 34.6 35.3 38.9 42.2 45.8 40.5 43.5 46.5 48.1 45.9 Aud+Vis 1Run 34.1 40.6 47.4 46.9 49.1 50.9 53.0 55.0 53.5 54.9 56.0 1Ses 30.5 40.9 44.7 47.5 47.9 50.7 48.5 50.3 51.4 51.1 53.5 2Ses 35.1 46.4 47.0 50.6 55.0 54.7 53.8 53.5 55.4 54.4 56.5 Participant 2 Session 3 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 25.8 29.6 27.1 28.1 29.7 33.0 33.2 31.7 33.4 33.5 34.5 1Ses 32.7 38.6 45.3 55.1 56.1 61.6 64.5 65.2 70.4 70.7 69.1 2Ses 33.5 41.2 49.5 54.4 57.7 62.4 67.8 67.6 72.5 73.0 72.8 Visual 1Run 33.9 38.6 36.6 41.3 47.7 46.9 47.7 49.4 52.3 53.6 52.4 1Ses 43.3 53.1 62.4 66.7 72.2 75.7 76.5 78.4 80.5 81.0 82.9 2Ses 34.1 46.4 53.1 59.3 62.4 66.1 70.0 70.8 73.8 75.8 76.2 Aud+Vis 1Run 39.6 52.7 55.8 63.6 67.0 72.2 76.2 77.9 81.7 83.0 84.4 1Ses 32.8 35.8 37.5 37.7 40.4 41.4 42.4 42.1 41.4 45.0 41.1 2Ses 28.0 29.6 31.8 30.5 33.4 33.7 33.7 33.4 31.9 33.1 31.9 Session4 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 43.5 64.1 71.9 75.1 80.9 81.9 84.0 86.5 86.9 88.3 89.6 1Ses 27.5 34.7 35.8 37.3 40.5 43.0 45.3 45.1 48.5 46.5 49.6

PAGE 97

87 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 44.3 55.8 63.1 66.0 71.7 75.1 74.6 76.9 76.9 76.3 78.8 1Ses 26.2 31.6 32.2 29.6 34.7 34.1 35.6 35.9 35.5 36.5 35.5 2Ses 28.3 24.9 27.1 30.2 27.6 30.3 30.8 34.4 30.1 32.4 32.6 Visual 1Run 36.5 47.3 52.3 55.0 57.5 59.6 60.4 61.7 61.2 62.0 63.5 1Ses 33.8 44.9 53.0 54.5 58.3 63.1 64.0 66.3 68.6 71.0 70.9 2Ses 30.7 41.7 44.2 47.6 49.9 51.8 53.7 52.8 53.5 57.5 60.9 Aud+Vis 1Run 35.9 50.6 57.1 58.6 62.0 63.7 65.4 70.5 68.0 70.1 69.7 1Ses 33.1 41.8 44.7 48.3 53.4 55.5 58.6 57.8 59.5 59.8 61.5 2Ses 32.3 41.0 40.2 41.7 45.2 51.7 49.1 51.3 54.4 55.9 54.9 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 36.1 47.5 56.4 58.6 64.1 64.9 66.4 66.0 69.1 68.9 71.3 1Ses 25.4 31.7 34.3 36.7 34.2 36.9 40.3 41.9 42.8 40.7 41.3 2Ses 27.4 25.9 24.3 24.5 24.5 24.4 22.7 23.0 25.2 24.5 23.0 Visual 1Run 31.1 36.6 42.6 41.9 46.4 45.1 44.9 45.9 46.8 47.8 47.7 1Ses 27.4 34.9 39.6 39.6 42.1 42.7 42.2 46.7 47.9 49.3 48.7 2Ses 27.7 32.7 36.3 38.2 38.0 40.0 42.0 41.6 45.7 45.6 47.7 Aud+Vis 1Run 35.9 36.5 34.5 37.8 37.9 36.9 39.7 36.0 39.2 37.4 38.2 1Ses 30.6 33.8 34.4 34.6 38.6 42.6 43.4 46.3 48.6 46.4 51.1 2Ses 25.4 30.8 31.5 33.4 35.0 35.5 36.7 37.4 37.6 33.3 35.3 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 39.0 44.9 56.0 59.4 62.4 62.6 65.8 70.6 73.1 74.4 72.7 1Ses 28.4 30.8 30.7 32.8 35.4 32.2 34.7 33.3 32.8 31.4 34.4 2Ses 30.6 33.0 34.3 35.1 39.5 38.5 38.9 40.9 37.8 38.1 39.6 Visual 1Run 39.6 53.4 60.0 62.0 65.6 70.6 73.8 75.2 76.9 76.2 78.3 1Ses 30.6 34.8 35.5 36.5 38.1 36.7 38.1 41.6 40.9 42.2 43.6 2Ses 37.9 49.5 59.6 67.4 69.1 74.4 75.1 79.6 83.8 81.2 86.5 Aud+Vis 1Run 43.0 57.7 63.4 72.1 74.0 75.9 77.8 77.4 79.5 80.5 79.8 1Ses 34.7 41.1 45.9 54.7 55.9 58.0 62.3 62.7 66.4 69.2 66.8 2Ses 31.3 36.7 41.3 43.4 47.2 49.8 50.8 52.0 52.7 51.7 54.5

PAGE 98

88 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 48.0 68.9 81.2 88.2 90.6 94.6 95.7 96.6 98.2 99.3 98.8 1Ses 46.5 61.5 70.0 71.3 75.2 75.0 77.2 77.5 78.5 77.4 80.2 2Ses 38.3 54.6 61.4 66.4 65.7 71.1 74.5 74.8 78.5 78.0 78.0 Visual 1Run 38.5 48.0 57.4 59.0 63.1 65.4 68.6 66.9 66.4 67.9 68.3 1Ses 32.3 44.5 53.6 56.9 58.9 64.0 65.5 65.3 67.1 69.0 71.8 2Ses 34.7 41.3 48.2 52.2 51.6 56.4 56.2 56.0 57.2 58.0 59.8 Aud+Vis 1Run 40.4 60.2 65.6 75.8 79.1 82.1 86.1 89.0 91.1 94.3 92.6 1Ses 27.5 28.9 32.1 36.7 34.5 35.2 35.7 36.5 40.4 42.7 43.3 2Ses 23.0 23.1 23.8 26.8 25.1 22.2 21.9 23.7 21.3 21.7 22.2 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 52.6 67.6 75.5 79.9 84.0 84.0 85.8 87.6 88.8 90.7 89.1 1Ses 40.5 47.4 49.1 54.8 53.6 57.5 57.5 56.6 57.3 59.5 55.7 2Ses 46.9 61.0 69.4 75.4 76.7 80.1 82.5 83.5 86.5 86.7 86.6 Visual 1Run 33.5 49.9 54.0 61.2 61.8 66.9 71.7 74.0 72.4 74.6 77.5 1Ses 29.6 30.7 33.1 38.0 33.5 35.7 34.1 32.9 35.2 36.5 37.7 2Ses 38.2 46.0 53.9 60.3 63.3 64.7 67.1 67.5 73.5 75.3 77.7 Aud+Vis 1Run 35.6 39.6 46.2 45.2 48.5 51.9 53.3 53.7 53.1 54.6 53.4 1Ses 32.1 35.1 41.8 44.3 48.0 50.1 52.3 56.8 59.3 59.8 59.2 2Ses 25.0 31.2 35.8 38.2 40.0 40.1 43.9 44.6 45.4 45.3 46.7 Participant 3 Session 3 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 37.7 53.8 59.2 65.7 72.0 75.5 78.2 78.7 81.1 82.9 83.5 1Ses 28.7 33.8 34.7 40.2 41.1 42.0 40.7 42.7 42.1 44.5 44.2 2Ses 24.9 22.1 23.5 22.6 21.6 23.1 22.3 21.2 20.7 21.1 20.7 Visual 1Run 31.0 35.9 39.5 43.2 45.7 44.6 44.2 47.0 45.6 47.4 50.3 1Ses 26.1 28.5 29.5 29.3 29.3 30.1 30.8 31.3 31.2 32.2 30.3 2Ses 29.5 30.0 29.0 34.4 32.6 33.0 32.7 33.7 32.4 33.8 35.6 Aud+Vis 1Run 45.0 58.1 65.1 68.6 67.5 71.5 71.2 74.0 74.9 74.2 76.1 1Ses 28.2 40.2 44.6 50.4 52.4 55.3 56.0 60.0 60.3 58.5 61.2 2Ses 33.4 44.6 50.1 58.5 62.3 67.5 69.6 73.2 73.6 76.6 79.3

PAGE 99

89 Session4 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 30.9 34.0 35.8 39.1 40.0 41.0 39.8 44.4 44.5 46.3 46.9 1Ses 25.1 26.7 25.6 23.6 24.1 26.7 22.6 25.1 25.0 23.1 24.1 2Ses 23.7 26.4 27.8 28.5 27.6 25.1 27.5 27.0 28.2 30.7 30.4 Visual 1Run 40.1 46.2 50.9 53.2 54.7 54.8 54.3 54.1 55.6 54.2 52.2 1Ses 31.2 39.1 46.0 45.9 50.1 52.4 55.0 54.8 57.6 58.2 59.5 2Ses 31.0 32.6 37.6 41.0 41.9 43.6 47.0 45.5 49.1 49.1 48.3 Aud+Vis 1Run 32.2 36.0 39.0 42.9 45.5 46.6 46.6 50.8 50.5 51.1 51.2 1Ses 29.0 33.1 34.6 36.8 38.9 36.4 40.9 39.1 38.6 41.1 43.1 2Ses 26.7 26.1 27.4 28.0 26.3 28.1 25.8 25.6 26.3 26.6 25.5 Session 5 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 37.4 44.1 44.8 50.7 54.2 58.7 56.8 58.8 60.2 61.8 61.9 1Ses 28.8 31.1 32.7 31.6 32.9 33.9 33.6 31.5 27.9 32.3 33.6 2Ses 27.0 22.9 23.1 27.4 25.8 25.4 26.7 24.8 27.5 28.1 27.5 Visual 1Run 35.6 42.4 46.8 48.0 48.4 49.7 49.9 50.1 51.7 50.6 50.1 1Ses 24.8 27.5 29.1 27.1 29.6 30.0 30.6 28.7 31.1 32.3 33.6 2Ses 24.9 30.6 34.2 33.0 35.3 33.3 36.4 35.6 37.6 40.5 37.5 Aud+Vis 1Run 36.3 39.8 41.8 44.2 47.7 50.9 53.3 55.0 53.9 56.0 57.2 1Ses 29.0 32.4 30.6 32.8 33.9 32.0 34.0 37.2 36.7 35.4 34.0 2Ses 35.7 43.1 46.7 48.0 49.7 53.4 54.4 57.3 59.6 57.6 60.7 Session 6 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 32.7 38.3 41.1 45.4 45.8 47.2 47.9 48.4 49.9 49.0 50.2 1Ses 25.7 27.8 24.5 27.5 26.6 26.0 25.2 26.3 25.7 24.4 26.7 2Ses 26.7 28.2 25.6 27.9 27.3 25.8 27.4 27.9 27.9 25.7 27.8 Visual 1Run 29.4 36.8 39.8 44.1 46.4 46.9 47.4 48.8 49.2 50.4 51.2 1Ses 30.9 41.2 49.7 50.2 56.6 56.8 62.5 64.5 66.4 66.7 68.4 2Ses 31.7 36.0 40.3 41.5 41.2 43.6 47.8 48.6 48.8 46.5 49.9 Aud+Vis 1Run 37.8 48.6 58.9 61.6 69.0 71.2 75.8 80.1 82.5 80.9 85.7 1Ses 25.9 25.0 26.0 22.9 25.5 25.8 23.9 26.4 21.6 23.0 22.9 2Ses 32.3 37.8 41.5 42.3 46.8 45.9 47.3 51.9 49.0 49.5 53.6

PAGE 100

90 Session 7 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 34.2 40.0 45.2 47.1 49.7 51.2 54.5 53.2 55.3 55.2 52.4 1Ses 30.5 34.1 35.5 38.5 35.9 35.5 38.6 39.2 40.4 36.0 35.3 2Ses 29.7 37.8 37.6 41.6 40.4 41.5 42.1 42.3 44.3 42.6 41.7 Visual 1Run 33.7 40.0 44.6 43.3 46.7 48.9 45.8 49.0 51.9 51.4 54.6 1Ses 34.1 44.6 43.1 47.5 51.3 53.9 55.5 61.1 57.8 62.5 65.0 2Ses 31.1 40.5 44.3 47.0 51.3 55.5 56.3 57.5 57.6 60.8 59.8 Aud+Vis 1Run 34.7 44.3 47.2 50.3 52.7 55.3 56.5 54.7 58.7 57.4 58.3 1Ses 26.6 29.1 28.2 28.6 31.9 29.4 29.5 31.6 33.7 32.9 34.7 2Ses 28.1 29.9 31.6 34.9 35.3 35.2 37.1 36.5 36.7 38.8 35.7 Session 8 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 37.3 49.4 53.0 55.1 60.7 59.0 61.5 62.5 65.0 63.9 63.6 1Ses 25.8 26.1 27.5 26.9 26.5 27.4 29.9 31.3 30.8 33.6 28.5 2Ses 26.9 28.3 30.4 30.3 33.9 33.2 35.8 36.6 37.7 39.6 34.9 Visual 1Run 28.8 30.2 33.0 35.3 36.8 38.6 41.7 40.8 43.3 45.1 43.9 1Ses 32.7 39.2 38.2 43.3 49.5 45.8 47.9 53.0 49.8 51.6 52.8 2Ses 25.8 28.1 27.1 27.6 27.3 27.8 25.3 28.4 25.8 25.4 26.3 Aud+Vis 1Run 31.6 32.9 36.4 36.6 39.4 39.8 40.4 42.1 41.9 41.2 42.5 1Ses 24.4 27.9 25.2 28.1 28.2 26.8 28.3 27.2 28.9 28.4 28.8 2Ses 25.0 25.6 24.9 23.9 27.1 25.3 22.6 24.2 23.8 24.2 24.2 Session 9 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 35.3 45.2 49.3 53.4 53.6 54.7 54.9 54.9 54.9 54.1 54.0 1Ses 30.9 34.5 35.6 41.6 42.1 41.1 43.2 46.2 45.7 45.5 46.4 2Ses 25.8 29.5 29.0 32.1 31.7 30.9 36.1 33.2 34.2 39.4 37.1 Visual 1Run 38.9 46.9 55.0 57.9 60.3 63.4 64.2 65.6 68.2 68.0 68.7 1Ses 35.1 43.2 48.2 53.1 58.1 61.6 65.4 68.1 68.8 69.5 74.3 2Ses 32.2 41.9 46.2 51.4 53.8 57.0 60.1 61.9 62.3 64.4 67.3 Aud+Vis 1Run 37.0 46.8 54.0 57.6 61.1 59.1 65.2 65.7 66.8 71.3 70.8 1Ses 30.5 38.0 42.5 43.6 45.8 46.8 49.6 51.6 53.0 53.2 58.0 2Ses 30.1 32.8 35.1 38.2 41.7 41.9 46.7 44.1 46.3 47.4 49.6

PAGE 101

91 Session 10 Num of Samples Auditory 1 4 7 10 13 16 19 22 25 28 31 1Run 40.5 52.3 59.1 58.2 61.6 60.3 58.9 60.0 58.0 59.3 58.4 1Ses 31.0 41.8 46.0 51.5 57.7 61.9 63.5 68.4 67.7 74.0 72.9 2Ses 29.1 34.9 37.0 43.6 44.3 46.6 47.5 51.9 53.8 57.1 54.5 Visual 1Run 36.6 46.8 52.1 53.9 57.6 56.1 58.8 57.8 58.7 60.2 60.0 1Ses 31.1 43.1 52.3 57.2 61.0 62.8 66.3 72.9 73.5 74.3 78.6 2Ses 27.8 35.5 40.0 40.7 44.2 45.7 42.5 46.6 47.7 49.5 51.6 Aud+Vis 1Run 37.9 43.2 47.2 49.6 51.6 51.7 53.5 52.0 52.3 52.1 51.3 1Ses 34.6 43.2 47.3 52.7 54.8 58.5 64.2 66.1 69.4 66.7 68.1 2Ses 35.9 50.2 53.2 61.9 63.3 68.7 71.5 73.3 75.6 74.5 77.8

PAGE 102

About the Author Eric received the B.A. degree (Psychology) from the University of South Florida in 1995 and the M.A. degree (Cognitive and Neural Sciences), also from the University of South Florida, in 1999. In 2002 he shifted his research focus from visual perception to cognitive psychophysiology, and began work on a project to develop a Brain Computer Interface (BCI) system. Currently, he is a research affiliate in the Laboratory of Nervous System Disorders at the Wadsworth Center of the New York State Department of Health. When he is away from the lab, he spends most of his time chasing his toddler son, or engaging in some other type of sport or exercise.