On P300 Detection using Scalar Products

Transcription

1 Vol. 9, No. 1, 218 On P3 Detection using Scalar Products Monica Fira Institute of Computer Science Romanian Academy Iasi, Romania Liviu Goras Institute of Computer Science, Romanian Academy Gheorghe Asachi Technical University of Iasi, Romania Anca Lazar University of Medicine and Pharmacy Grigore T. Popa of Iasi, Romania Abstract Results concerning detection of the P3 wave in EEG segments using scalar products with signals of various shapes are presented and their advantages and limitations are discussed. From the point of view of the computational complexity, the proposed algorithm is a simple algorithm, based on a scalar product and searching for the max value of 6 calculated values. Because we considered that the human subject is not a robot that precisely generates P3 and that there is also a human component of error in the involuntary generation of such waves, we have also calculated the rate of classification of character in the human visual field. To validate the proposed method, electroencephalography recordings from the competition for Spelling BCI Competition III Challenge 25 - Dataset II have been used. Keywords Electroencephalographic (EEG); brain computer interface; P3; spelling paradigm; classification; signal processing I. INTRODUCTION The use of electroencephalographic (EEG) signals as a vector of communication between man and machine continues to be a challenge in the brain-computer interface (BCI) paradigm aimed at achieving proper interpretation of brain specific electrical activity signals and parameters [1], [2]. A definition of a BCI, subsequently accepted by most researchers in the field, is a communication system in which the messages or the commands sent to the exterior environment by an individual do not pass through the normal output paths of the brain, paths constituted by the peripheral nerves and muscles [3]. The purpose of BCI is to establish a communication system that translates human intentions, represented by appropriate signals - into control signals for an output device, such as a computer or neuroprosthesis. Since BCI can potentially provide a link between the brain and the physical world without any physical contact [1] its main objectives are to process the EEG waveforms and to generate the necessary signals to control some external systems. The most important application is to stimulate paralyzed organs or bypass disabled parts of the human body. BCI systems may appear as the unique communication mode for people with severe neuromuscular disorders such as spinal cord injury, amyotrophic lateral sclerosis, stroke and cerebral palsy [4], [5]. The correspondence between EEG patterns and computer actions constitutes a machine learning problem since the computer should learn how to recognize a given EEG pattern. As for other learning problems, in order to solve this problem, a training phase is necessary, in which the subject is asked to perform prescribed mental activities and a computer algorithm is in charge of extracting the associated EEG patterns. After the training phase is finished the subject should be able to control the computer actions with his or her thoughts. This is the major goal for a BCI system. In this work we investigate the possibility of detecting P3 waves using scalar products between EEG segments and various waveforms with shapes similar to P3 ones on data from the BCI III competition. II. SPELLING PARADIGM AND THE DATA SET: DESCRIPTION The P3 speller paradigm makes use of waves that are expressions of event related potential produced during decision making process. One of the first examples for BCI is the algorithm proposed by Farwell and Donchin [6] that relies on the unconscious decision making processes expressed via P3 in order to drive a computer. According to the P3 speller paradigm described in [1] the subject should watch a 6x6 matrix containing all letters and digits (as shown in Fig. 1) and should focus the attention on a character in the matrix. The protocol contains two steps: Step 1: The matrix is presented to the subject for 2.5 seconds. Step 2: All lines and all columns are randomly highlighted each for 1 ms with a pause of 75 ms. Fig. 1. Classical P3 spelling paradigm described by Farwell-Donchin 1988 [1]. 99 P a g e

2 Vol. 9, No. 1, 218 Thus, even though we did not expect better classification results than those reported in [7] this simple approach offers interesting insights. Basically the aspects we considered were: Fig. 2. Placement of the 64 electrodes. The procedure consists in repeating 15 times Step 2 (15 epochs) for each character, followed by a pause of 2.5 seconds (Step 1). For a given character, there will be 6x2x15=18 intensifications: 2x15 will contain the target character (once when the column is highlighted, second for the line it belongs to, repeated for 15 epochs) and the all the others are supposed not to contain it. The data for the BCI III competition has been recorded from two different subjects in five sessions. The raw signals have been processed with a band-pass filter in the range.1 6Hz and sampled at 24Hz. During each session, the subjects were asked to spell words and for each word a run was defined. All EEG signals of a 64- scalp electrodes placement (Fig. 2) have been continuously collected during an acquisition session. The data set is split in the train set (containing 85 characters) and the test set (containing 1 characters) for each subject. An extended description of the dataset is available in the BCI competition paper [1]. The winners of the competition proposed a method based on an ensemble of classifiers [7] of linear Support Vector Machine type. They were trained on a reduced part of the available data by performing a channel selection operation. The reported classification rate was 95.5% for 15 sequences and 73.5% for 5 sequences [7]. III. METHOD The aim of the investigations presented in what follows is to check the advantages and limitation of finding a prototype P3 waveform and to detect the presence of a P3 wave in an EEG segment by using the well-known scalar product. The main idea is that if an EEG segment presents a P3 waveform, it can be considered as a component orthogonal to the subspace of all the others in the segment. Thus, taking the scalar product of an EEG segment with a P3 prototype, we should obtain a high/small value if the segment exhibits/does not exhibit P3. The difficulty of the method consists in the fact that the shape of the P3 components is not the same for all segments. a) Train and test signals pre-processing. b) Generating various prototype shapes for the P3 waveform. c) Using the above waves to compute scalar products for one or several channels for the test case. d) Analyse and compare results. The pre-processing stage consists of EEG signals segmentation into slices of 1s length beginning at the start of the flashing intervals (1ms at 75ms distance) and averaging of the EEG segments corresponding to a flash i.e., to each line or column that has been intensified 15 times. More precisely, at this stage the pre-processing is based on the average of the selected channels for the 15 repetitions in order to calculate the P3 pattern from all 85 epochs and for calculation of 12 average signals for each epoch (corresponding to the 6 lines and 6 columns). Moreover, averaging has been used not only for one channel but also for several channels as it will be shown later. After averaging, for each line/column, a pattern of the EEG results which either is known to exhibit or not P3 for the train signals or should be classified as exhibiting or not P3 for the test signals. Thus, for classification on the testing set (epoch) we constructed 12 test signals for each character (corresponding to the 6 lines and 6 columns) by averaging all 15 repetitions. In other words, by averaging all 15 intensifications corresponding to each row or column we obtained a pattern characteristic to the respective line or column. The obtained patterns have been smoothed with a 1 samples window median filter. In the next steps various shapes for approximating the P3 pattern from the information in the test waveforms have been generated. The aim was to find which shapes give the best classification results using the scalar product with testing signals and to make a comparative analysis not only regarding the best shape ( prototype ) but also choosing the most significant channels for classification. After the scalar product between the averaged flashing pattern and the P3 prototype pattern is computed, the lines and columns containing P3 are determined according to the highest values for the scalar product. Another aspect that has been investigated is related to the hypothesis that it is not unlikely for subjects to produce P3 signals when adjacent lines to the character the patient was thinking at. Thus we have computed the so-called Partial Classification rates representing the classification rate of the lines and of the columns containing P3. Indeed, it is possible to correctly determine the line containing P3 corresponding to a given character while the column is wrong such that the final classification of the character is wrong. In other words, we did not make the intersection of the line and column containing the P3, but separately the classification rate for lines and for columns. 1 P a g e

3 Vol. 9, No. 1, 218 A B C D E F G H I J K L M N O P Q R S T U V W X Y Z _ Fig. 3. Panel for the classical P3 spelling paradigm, a desired character (colored in dark grey) and the characters from visual field (colored in light grey). We have also evaluated the co-called Visual_Field Classification which has been introduced based on the hypothesis that a subject may also generate P3 at highlights (illuminations) of adjacent lines or columns to the chosen (target) character. Indeed it is not unlikely that there is an error due to the human visual field and attention of the subject when adjacent lines/columns are illuminated. Indeed, since the human subject is not a robot or a machine that responds precisely, each subject has a degree of error in generating P3 depending on the state of fatigue, stress or attention, visual field, etc. In order to better understand the Visual_Field Classification concept, we have presented in Fig. 3 the character matrix, a desired character and the characters considered in the visual field which are taken into account. This is why in addition to the final classification rate, there were calculated the Partial and Visual_Field Classification rates. IV. EXPERIMENTAL RESULTS For the evaluation of the analyzed methods we used the dataset II of the BCI Competition III 25 -P3 Spelling. The data for the BCI III competition has been recorded from two different subjects but in this paper we used only teh recordings from one subject because we followed the presentations of the method and the presentation of the results from both subjects would make it difficult to follow the ideas. According to the above discussion we remind the reader that, for the scalar product method, the shape of the P3 prototype pattern is the one that mainly influences the final classification results. Thus, several such prototype patterns have been generated and tested. To make a global image the types of patterns that have been used are listed below. Average P3 Pattern - calculated by averaging all EEG segments containing P3 Five P3 patterns calculated based on the K-Means algorithm applied to all segments containing P3 Stylized pattern by summing up three Gaussians Stylized pattern by 1 Gaussian Stylized P3 Pattern by decimating and then interpolating the average P3 pattern defined first in this list. In what follows we will discuss the results obtained with the above prototype patterns. A. Average P3 Pattern The first pattern has been obtained by averaging all segments known/supposed to exhibit P3 in the training set. For the selected channels, there were extracted and normalized all data samples between and 1 sec next to the beginning of a flash. From a training set of 85 spelling characters, for each analysed channel, there were taken 12*15*85 = 153 signals of 24 samples. These 15 3 signals are split into P3 signals and non-p3 signals, resulting two subsets, the subset of 255 signal with P3 and the subset of 1275 signals without P3. From all subsets P3 signals, by averaging, the P3 pattern shown in Fig. 4 was achieved. Such a pattern can be considered a first prototype for the shape of the P Fig. 4. A P3 pattern obtained by averaging. At a first thought we might hope that the scalar product of the above wave would make a significant difference between P3 and non P3 segments. The reasons this did not happen are at least two: a. the pattern in Fig. 4 is an average and b. each waveform exhibiting P3 has its own shape which will not necessary give a high scalar product with the above pattern. To keep things as simple as possible, we selected a list of most important channels by making the scalar product with segments extracted from each one [8], [9]. More precisely we wanted to find which channels are the best in terms of the classification rate. At this stage we used one single channel and the proposed algorithm. In Table I the classification rate for several channels are presented. TABLE I. The P3 patterns CLASSIFICATION RATES FOR 18 CHANNELS Channel Classification % Channel Classification % F Cz F C F CP FC3-2 4 CP FC CPz FCz Pz FC2-5 5 P C POz C PO P a g e

4 Cluster (IJACSA) International Journal of Advanced Computer Science and Applications, Vol. 9, No. 1, 218 TABLE II. CLASSIFICATION RATES FOR AVERAGE P3 PATTERN USING VARIOUS COMBINATIONS OF CHANNELS AND VARIOUS DECISION METHODS Final Classification Partial classification 3, 4, 5, 1, 11, 12, 17, 18, 19, 17, 19 6% 56.47% 58.82% 89.41% Visual_Field Classification 86.47% 88.23% 84.7% 87.5% 85.88% Analysing the results the following channel combinations and averaging have been chosen for classification comparison (), (3, 4, 5, 1, 11, 12, 17, 18, 19) and (, 17, 19). Again, we stress the fact that using averages we cannot expect high classification rates. We have tried simulations with a different number of channels, that is, more or less channels, hoping that classification rates will change significantly. Using scalar products of this pattern with average segments from various channels, the classification rates reported in Table I have been obtained. We have thus built on the idea that, when multiple-channel EEG signals are averaged, a kind of P3 filter is obtained so that using EEG signals from multiple channels increases the classification performance. Thus, taking into account the results presented in Table I for the proposed algorithm, we have tested various combinations of handled channels the classification performance being presented in Table II. B. Five P3 Patterns based on the K-Means Algorithm Observing that a single pattern cannot define very well all cases of EEG segments that contain P3, we tested an alternative in which more patterns (in our case five) were used. Thus, by means of the K-mean algorithm and the EEG segments with P3 from the training database, we defined the class centroids as the (five) prototype patterns for P3 wave. In the classification stage, we calculated the scalar products between the EEG segments and each of such a pattern and the values of scalar products are summed which were summed in order to find the line and the column containing the P3 wave. In Fig. 5 the patterns obtained with K-means algorithm and the silhouette plot for the clustered data are presented. It can be seen that the patterns exhibit rather different forms. The silhouette refers to a method of interpretation and validation of consistency within clusters of data. The technique provides a succinct graphical representation of how well each object lies within its cluster. The silhouette ranges from 1 to +1, where a high value indicates that the object is well matched to its own cluster and poorly matched to neighboring clusters. If most objects have a high value, then the clustering configuration is appropriate. If many points have a low or negative value, then the clustering configuration may have too many or too few clusters Fig. 5. P3 patterns obtained with the K-means algorithm applied on EEG segment with P3 and silhouette plot from the clustered data. In Table III the classification results obtained with K- means patterns are presented. It can be noticed that the final classification rate is similar with the case with average P3 pattern but the partial classification rate and Visual_Field Classification are slightly better comparatively with Table II. TABLE III. CLASSIFICATION RATES FOR K-MEANS PATTERNS USING VARIOUS COMBINATIONS OF CHANNELS AND VARIOUS DECISION METHODS Final Classification Partial classification Visual_Field Classification Silhouette Value 3, 4, 5, 1, 11, 12, 17, 18, 19, 17, % 57.64% 57.64% 89.41% 88.23% 87.5% 86.47% 84.11% 87.64% C. Stylized Pattern by Summing up Three Gaussian Functions In several papers [1], [11] authors proposed as a P3 pattern a waveform model based on mathematical expressions of Gaussian functions. We have investigated the above approach by defining the model by summing up 3 Gaussian functions (see Fig. 6 and Table IV for results). 12 P a g e

5 Vol. 9, No. 1, 218 TABLE IV. CLASSIFICATION RATES FOR STYLIZED PATTERN COMPOSED OF GAUSSIAN USING ONE COMBINATIONS OF CHANNELS AND VARIOUS DECISION METHODS Final Classification 7.5% Partial classification 42.35% Visual_Field Classification 64.11% Fig. 6. P3 pattern composed by three Gaussian functions. D. Pattern Defined by 1 Gaussian Function We have also investigated the case of one Gaussian function and shown in Fig. 7. The final classification rate is low compared to the case with average P3 Pattern, but the partial classification rate and Visual_Field Classification are much better (see Table V) The P3 pattern with 3 Gaussian Fig. 7. Gaussian P3 pattern. TABLE V. CLASSIFICATION RATES FOR GAUSSIAN P3 PATTERN USING ONE COMBINATIONS OF CHANNELS AND VARIOUS DECISION METHODS Final Classification 3.58% Partial classification 75.29% Visual_Field Classification 77.5% E. Stylized P3 Pattern by Decimating and then Interpolating the Average P3 Pattern The last attempts to find prototype P3 patterns starting from the average P3 pattern from Fig. 4 were to construct a stylized pattern by decimating the average P3 Pattern and rebuilding the pattern waveform by cubic spline data interpolation (see Fig. 8) PATTERN p3 REAL OBTINUT PRIN INTERPOLARE Decimate 1:2 Decimate 1:1 Fig. 8. Stylized P3 Pattern by decimating with factor of 1:2 and 1:1. TABLE VI. CLASSIFICATION RATES FOR STYLIZED P3 PATTERN BY CUBIC INTERPOLATION USING ONE COMBINATIONS OF CHANNELS AND TWO P3 WVEFORMS Decimate 1:2 Decimate 1:1 Final Classification 56.47% Partial classification 87.5% 9.58 Visual_Field Classification 85.29% The final classification rate is similar with average P3 Pattern, but the partial classification rate and Visual_Field Classifications are much better. In fact, the partial classification rate is for 1:1 decimation has the highest values of all. V. CONCLUSIONS In this work we have analyzed and tested a very simple algorithm for the P3 detection based on the values of the scalar product between EEG segments and several prototype waves. The proposed algorithms are of low complexity and basically reduce to finding a maximum among 6 calculated values. Moreover, we have computed so called partial and visual field classifications. As expected, the results are significantly influenced by the shape of the prototype waveform. Thus, the best final classification results have been obtained with the averaged P3 wave and average of channels 3, 5 and 11 which gave the best results for the Visual_Field Classification as well (marked grey in Table II). The results confirm that a value of 86.47% for the Visual_Field which is higher than the Final Classification value with about 26%. This value shows that the probability of generating a P3 for characters around the target one is much higher compared to the case when this error would be uniformly distributed. Thus the probability of generating a P3 for characters in in the vicinity of the target one is higher. Another interesting result is that through decimation and interpolation the results are rather similar to the case of the initial prototype shape; moreover, for partial classification with 1:1 decimation rate they are the best of all (marked grey in Table VI). We envisage further investigations might envisage results of the proposed method applied to other EEG as well as ECG databases. REFERENCES [1] Blankertz, B.: BCI competition III webpage, P a g e

6 Vol. 9, No. 1, 218 [2] Blankertz, B., Mueller, K.-R., Curio, G., Vaughan, T., Schalk, G., Wolpaw, J., Schloegl, A., Neuper, C., Pfurtscheller, G., Hinterberger, T., Schroeder, M., Birbaumer, N.: The BCI competition 23: Progress and perspectives in detection and discrimination of EEG single trials. IEEE Trans. Biomed. Eng. 51(6), (24) [3] J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller, and T.M. Vaughan, Braincomputer interfaces for communication and control. Clin Neurophysiol, 113, Jun., , (22). [4] Schalk, G., McFarland, D., Hinterberger, T., Birbaumer, N., Wolpaw, J.: BCI2: a general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering 51(6), (24) [5] Serby, H., Yom-Tov, E., Inbar, G.F.: An improved p3-based braincomputer interface. IEEE Trans. Neural Syst Rehabil Eng. 13(1), (25) [6] L. A. Farwell & E. Donchin, "Talking off the top of your head: A mental prosthesis utilizing event-related brain potentials", Electroencephalogr. Clin. Neurophysiol. 7 (6): , 1988 [7] A. Rakotomamonjy, V. Guigue, BCI Competition III: Dataset II- Ensemble of SVMs for BCI P3 Speller, IEEE Transactions on Biomedical Engineering,Volume:55, Issue: 3, 28, pp [8] K. A. Colwell, D. B. Ryan, C. S. Throckmorton, E. W. Sellers, L. M. Collins, Channel Selection Methods for the P3 Speller, J Neurosci Methods, July 3; 232: 6 15, 214 [9] Minpeng Xu, Hongzhi Qi, Lan Ma, Changcheng Sun, Lixin Zhang, Baikun Wan, Tao Yin, Dong Ming, Channel Selection Based on Phase Measurement in P3-Based Brain-Computer Interface, PLOS ON, April, Volume 8, Issue 4, 213 [1] Lucie Daubigney and Olivier Pietquin, Single-trial P3 detection with Kalman filtering and SVMs, European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning. Bruges (Belgium), April 211 [11] Setare Amiri, Ahmed Rabbi, Leila Azinfar and Reza Fazel-Rezai (213). A Review of P3, SSVEP, and Hybrid P3/SSVEP Brain- Computer Interface Systems, Brain-Computer Interface Systems - Recent Progress and Future Prospects, Dr. Reza Fazel-Rezai (Ed.), InTech, DOI: / Available from: 14 P a g e