Mathew Salvaris, Caterina
Cinel, Riccardo Poli, Luca Citi,
School of Computer Science and Electronic Engineering
University of Essex
CO4 3SQ, Colchester, UK
In recent years, various visual protocols have been explored for P300-based BCI. In stimulus-driven BCI paradigms such as P300 BCIs it is vital to optimise the stimulation protocol as much as possible in order to achieve the best performance. Due to the inherent variability between subjects and the complex nature of the brain it is unlikely that an optimal protocol will be identified through a single iteration of random exploration. That is why in this paper we explore 8 different visual protocol configurations based on recent literature, in the hope of identifying key features that can later be used to create further improved protocols. Results indicate that luminosity changes, the standard method of stimulation used in visual P300 BCI protocols, do provide the best performance of the variations presented here.
Over the last few years there has been a healthy interest in trying to improve P300-based BCIs through modifications of the protocol used [1,2,3]. These modification come from trying to mitigate the temporal limitations of the P300 signal  or simply by observing the possible effects of the alterations to the visual makeup of the protocol [2,3]. For example, in  a number of modifications were made to the visual properties of a P300 matrix speller, including, the size of the letters, the distance between the letters and the colours used. Gonsalvez et al.  explored the effects of the Target to Target Interval (TTI) on the P300 and found a direct correlation between TTIs and P300 amplitudes. Martens et al. , Salvaris and Sepulveda  and Citi et al.  explored the effects of very short TTIs in P300-based BCIs and concluded that temporally proximal target events are harder to detect than target events separated by greater temporal distance. Although the reason for this effect is not entirely clear, a possible explanation is repetition blindness and attentional blink . In  the possibility of repetition blindness was explored and, although the effects on accuracy did match those of repetition blindness, it was not enough to unequivocally commit to that explanation. In  one of the modifications shown to mitigate the effects of repetition blindness was saliency between the two target events. In  it was shown that the repetition blindness-like effect present in their data could be mitigated by simply using a differing visual protocol and they attributed this difference to the apparent motion of the stimuli used. Others [9,10] have limited the temporal proximity of possible target events thereby avoiding these effects altogether.
In this paper we explore a number of novel visual protocols within the framework of a P300-based BCI mouse . The changes made concerned the SOA, the duration of the change and type of stimulation. The various visual protocols created were intended to investigate the effects of different TTI values, apparent motion and target event saliency on epoch classification.
Data were collected from 8 participants with an average age of 26.5. Subjects 1 and 2 carried out all 8 visual protocols. After that, however, it was determined that the experiment lasted too long and that fatigue might prevent other subjects from completing the experiment. Therefore, a Latin square approach of experimental design was used to split the 8 experiments into 4 experiments which would allow for the assessment of up to second order interactions.
In each session participants were presented with visual displays showing 8 circles (with a diameter of 1.5cm) arranged around an imaginary circle at the centre of the display as in Figure 1. Each session was divided into runs, which we will call direction epochs.
Each participant carried out 16 direction epochs for each of the chosen visual protocol configurations. Participants 1 and 2 carried out 8 visual protocol configurations while the rest carried out 4 visual protocol configurations. With 16 direction epochs, the 8 possible directions were covered twice over. Within an experiment the direction epochs and visual protocols were randomised, the only constraint being that, within 4 or 8 direction epochs (depending on the number of visual protocols conducted by the subject), each of the possible visual protocols be conducted only once.
During a direction epoch, the participant was greeted by a blank screen and after a predefined period the stimuli appeared near the centre of the screen. A red arrow then appeared for 1 second pointing to the target. Subjects were instructed to count the number of state changes for that target. After 2 seconds the random activation of the stimuli started. This stopped after 20 to 24 trials, with a trial consisting of the individual activation of each of the 8 stimuli. After the direction epoch had been completed, the subject was requested to verbally communicate the number of times the target stimulus changed state.
Participants were seated comfortably at approximately 80 cm from an LCD screen, their neck supported by a C-shaped inflatable travel pillow to reduce muscular artifacts. Data were collected from 64 electrode sites using a BioSemi ActiveTwo EEG system. The EEG channels were referenced to the mean of the electrodes placed on either earlobe. The data were initially sampled at 2048 Hz.
Classification was carried out using an ensemble of 6 linear SVMs, with each SVM trained on a subset of the collected data across all the channels. The classification method is similar to the one used by Rakotomamonjy and Guigue  except that no channel selection was carried out and all 64 channels were used instead. The data were filtered between 0.15 and 30 Hz and initially downsampled to 128 Hz. Then, from each channel an 800 ms epoch was extracted and further decimated to 32 Hz.
The classification results were estimated using 16 fold cross-validation, with each direction epoch being a cross-validation fold. The data were initially compared in two groups. One group consisted of the experiments carried out using the slower SOA and the other one was for the faster SOA. In Figures 2 and 3 the corresponding Receiver Operating Characteristic (ROC) curves of the experiments across all the subject are shown. The classification accuracy values in these plots, like all others presented in the paper, were calculated based on single-epoch classification, i.e., on whether the data (presented to the classifier from all the channels in an 800 ms post-event epoch) contained a P300 or not. From Figure 2 it is evident that the best protocol is the standard flash protocol, while the worst protocol is the persistent colour protocol. These observations are also reflected in the Area Under the Curve (AUC) values shown in Table I. A similar conclusion can be drawn from observing the results for the faster protocols (see Figure 3 and Table II).
|Flash Colour||Flash Distance||Pers. Colour||Pers. Distance|
|Flash Colour||Flash Distance||Pers. Colour||Pers. Distance|
|Flash Col.||Flash Dist.||Pers. Col.||Pers. Dist.|
|Flash Col.||Flash Dist.||Pers. Col.||Pers. Dist.|
The results from a multiple-comparison Kolmogorov-Smirnov test can be seen in Tables III and IV for the slow and fast protocols, respectively. From the p-values presented in Table III it is apparent that the two protocols that differ the most from the rest are the flash colour and the persistent colour, the flash colour protocol being the best performing and the persistent colour protocol being the worst performing, according to the AUC values presented in Table I. A similar conclusion cannot be drawn for the fast protocols based on the results of the Kolmogorov-Smirnov test (see Table IV).
The top two slow protocols (based on AUC) were then compared to the top two fast protocols. Of course, this is a little unfair since in theory, due to the effects of TTI, the slower protocols should always out-compete faster protocols in single epoch classification, as a result of the increased amplitude in the P300. A quick observation of the AUC-results' tables confirms this. Evidently, the faster protocols with an SOA half that of the slower ones can carry out twice as many events within the same time period. So, the AUC scores for the top two fast protocols were recalculated by obtaining the mean score of two temporally adjacent epochs of the same stimulus. This was done on the score output of the previously trained classifiers and did not involve retraining the classifiers on the combined data. These new AUC results can be seen in Table V and the results of the Kolmogorov-Smirnov in Table VI.
|Fast Flash Col.||Fast Flash Dist.||Slow Flash Col.||Slow Pers. Dist.|
|F. Flash Col.||F. Flash Dist||S. Flash Col.||S. Pers. Dist|
|F. Flash Col.||-||1.00000||0.87302||0.07937|
|F. Flash Dist.||1.00000||-||0.87302||0.00794|
|S. Flash Col.||0.87302||0.87302||-||0.00794|
|S. Pers. Dist.||0.07937||0.00794||0.00794||-|
The results seem to confirm the hypothesis that the slower protocols with the greater TTI range do indeed provide greater separability between the target and non-target epochs. What is interesting, however, is that if we compensate for this, by taking the mean of the score of two epochs (of the same stimulus event) in the faster protocols for every one in the slower one, we end up with comparable results for the flash colour protocols. For the faster protocols the top two performing protocols provide very similar results. The same cannot be said for the top two slow protocols, with the slow persistent distance protocol being significantly different from the fast flash distance protocol and the slow flash colour protocol. Overall the faster protocols perform better than their slower counterparts within the same time period, probably due to the error correcting effects of combining multiple trials. Another observation between the fast and slow protocols is that the fast protocols show a greater amount of variance between subject performance than their slower counterparts, possibly indicating that the optimum SOA might be subject-specific.
The protocols that utilised apparent motion did not seem to provide any advantage over using luminosity changes in terms of the AUC values. It is possible that the range of movement chosen for the apparent movement was not large enough and that greater displacement may provide better results, although the apparent motion protocols seem to result in less inter subject variance.
The persistent protocols generally seem to provide far worse results than the flash protocols. This could possibly be attributed to subject confusion, since each subject carries out a mixture of flash and persistent protocols and at very short intervals the persistent stimuli often appear as flash stimuli (two successive persistent state changes appear as a single momentary state change). This in turn may increase memory load, adversely affecting the amplitude of the P300. In future experiments we will explore the advantages and disadvantages of having more than two states, thereby decreasing the possibility of confusion with the flash protocols and increasing inter-target saliency. Another interesting possibility may be to test persistent protocols in isolation to flash protocols on naive subjects, removing the possibility of this confound.
We would like to thank the UK Engineering and Physical Sciences Research Council (grant EP/F033818/1) for financial support.