BCI++: an object-oriented BCI Prototyping Framework

Transcription

1 BCI++: an object-oriented BCI Prototyping Framework Luca Maggi 1, Sergio Parini 2, Paolo Perego 1 and Giuseppe Andreoni 1 Sensibilab, Politecnico di Milano 1 Indaco Dept., Politecnico di Milano, ITALY 2 Bioengineering Dept., Politecnico di Milano, ITALY luca.maggi@polimi.it Abstract This work aimed at providing a set of tools for the development of home Brain Computer Interfaces based on a low cost acquisition and processing device which could be used by both the end-user in their domestic activities and by the researcher in order to develop new protocols and algorithms and all the software modules necessary to ease the interfacing of the system to external devices (eg. Home automation systems). The framework was validated by setting up a BCI system based on the SSVEP protocol. Six subjects out of ten healthy users were able to use the system in a home automation and in a gaming application. 1 Introduction A Brain Computer Interface (BCI) is a man machine interface which allows to establish a new communication channel between the brain and the computer. This technology could be used in order to restore or to supersede lost body functions and it could potentially represent a substantial improvement to the quality of life of disabled people. In the last years many efforts were produced in order to demonstrated the applicability of the brain computer interface to daily life by moving the BCI systems from a controlled laboratory environment to a more complex context like a domestic one. During this out-of-lab transfer procedure many technological, man machine interaction, signal modification and financial problems must be faced; the aim of this work is that of contributing at reducing the burden of those activity by providing a set of tools studied to ease the development of novel BCI systems or the porting of existing systems. When compared to other existing tools, the main difference in our approach is the final goal of considering the acquisition device like a common personal computer peripheral thus the device should perform both signal acquisition and signal processing. 2 Materials & Methods The proposed methodological approach for the development of a domestic BCI is here presented step by step: Data-set recording; Offline algorithm prototyping in Matlab ; Online test; Identification of the possible user-specific customization; High level profiling of the most important blocks (Memory, execution time); Optimization of the algorithm and the protocol; Porting into a lower level language (C); Execution by the ARM7 CPU of the EEG device; User Interface customization. The framework provides both the final device and all the intermediate tools necessary for the setup of the protocol and the algorithms following the depicted development approach.

2 The framework is composed of: A low cost acquisition unit; A mathematical C library (C4M); A dynamic mathematical engine (C4MEngine); An hardware interface module; A graphic user interface (AEnima); A home automation system interface The signal acquisition is performed using KimeraII, a low cost Bluetooth EEG unit composed of an EEG amplifier and a digital acquisition and elaboration unit. The EEG amplifier was developed in the laboratory implementing a patented topology for the signal conditioning. Thought being 3.3V battery powered, the device is robust against artefacts and out-of-band noise thanks to the adopted novel circuitry which provides a fast recovery from saturation and enhanced dynamic reserve [1]. The signal acquisition module was specifically designed and produced by SxT Sistemi per Telemedicina s.r.l. (Lecco - Italy), it features a Real Time Operating System (RTOS) and a specifically designed firmware. The system was controlled by an ARM7 32bit microcontroller which accomplishes the following tasks: Analog to digital conversion using an external 8-channel ADC; Local data storage on a micro-sd card in a PC compatible format (FAT16); Communication with the PC or a PDA by an onboard Bluetooth module (SPP profile); On board data processing using a specifically designed mathematical library written in C language. The connection with the acquisition unit and, in the early stages of the development, the signal processing is performed by the Hardware Interface Module (HIM): a software designed in order to assist the researcher during the prototyping of a new BCI system. It provides a solid structure for the acquisition, storage and visualization of the signal, for the communication with the BCI user interface and for the real-time execution of algorithm both in C and Matlab environment. This software was written in C++ language using the crossplatform wxwidgets library. The source code was structured in order to promote application specific customization using hereditary and polymorphism techniques. The basic version supports several kind of instruments; some are real, other are virtual and are useful for debug and simulation purposes. They are: Kimera version II and I; GTec G.MobiLab; An ideal signal generator; An instrument simulator which load previously recorded signal from a file and play it ad the same sample frequency; An interactive file player which can load pieces of dataset depending on the operator s needs. It is possible to add a new instrument by deriving a specific class from the basic Instrument class: for example, in four hours it was possible to enable the HIM with the G.Mobilab instrument (G.Tec, Graz, AU). The HIM also provides a basic algorithm class which handles all the operation related to data buffer management, performance monitoring and, if necessary, the communication with Matlab. In order to test a new algorithm the researcher only have to focus on writing the specific code, in C or Matlab language according to his need and the HIM will manage all the other issues. In order to simplify the problem of porting BCI algorithms to different platforms, especially those not supporting Matlab based development, the C4M Mathematical library was developed. The C4M library is a powerful tool for the efficient porting of generic algorithms on single chip embedded system, which have limited performances in terms of speed and memory resources, when compared with those available in a standard PC. This library was already described in other works [2]. A mathematical engine provides a command line high level language which allow the test of new functions directly on the final hardware device: using a terminal software and a serial port it is possible to use the a standard personal computer in order to interact with the mathematical engine running on the device in a manner which is similar to a simplified Matlab version. The graphical user interface module, AEnima is an independent application and was written using a multiplatform and multi-api based graphical engine in order to provide a more realistic and challenging experience to the user. A TCP/IP socket based layer managed the communication with the hardware interface module which can be executed by both the same personal computer which executes the HIM and by another one. The following figure shows the structure of application: The core of the system: it is an open source high performance realtime 3D engine written and usable in C++. It is cross-platform, using D3D, OpenGL and its own software renderer;

3 The protocol class implements the user application and the stimulation protocol; The protocol dealer is a coordination object used for messages dispatch to handle the interaction between different user applications. Figure 1. Structure of the AEnima module A specific software module was implemented in order to provide an interaction layer with an home automation system. A MyHome gateway was provided by BTicino spa (Erba, Italy) and a basic demonstrator was set-up in our laboratory. The standard physical communication layer of the used gateway is RS-232; in order to maximize the ease of installation a specific RS-232 to Bluetooth module was designed. 3 Results The consistency of this tool chain was verified by using BCI++ to implement an SSVEP based BCI system. In the next paragraph a detailed description of the system will be given in order to demonstrate that all the specific modules were used in implementing a real-time working system. The framework proved a good flexibility and the wide set of debugging instruments dramatically simplified the debug and the test of a new system. Starting from a previous experience and a set of algorithms for the SSVEP protocol it was possible to set-up a complete BCI system in the Sensibilab laboratory in CampusPoint@Lecco (Lecco, Italy) in a few weeks. The visual stimulation system consisted of four cubic spotlights with high efficiency LED, the cubes were fixed to the edge of the screen ideally associating each light to a direction: up, down, left or right. The stimulation device autonomously generated and was controlled by the AEnima module. The following figure shows the structure of the SSVEP classification algorithm. Figure 2. Structure of the algorithm A spatial filtering block (SFB) was used in order to reduce the dimensionality of the eight acquired monopolar leads. The SFB computes a custom derivation on the basis of the analysis of the calibration data-set. The spatial filtering is computed performing a linear combination of the acquired channel. A band pass filter between 5 and 40Hz eliminated the out of band noise. The features were the result of a ratio between the amplitude of a stimuli time-synchronous averaging of the signal, compared to the estimation of the amplitude of the raw signal. For each frequency the algorithm extract one feature. The resulting estimation is a numerical value which theoretically ranges from 0, in case of total disruptive interference within the main window, to 1, when the signal has only one periodic component at the

4 same period of the stimulation. The classifier consisted of a regularized linear discriminant analysis (RLDA) based on the modified samples covariance matrix method. The RLDA included a boosting algorithm based on a cyclic minimization of the classification error on the training set and an algorithm for outliers rejection. The classification algorithm was conceived for use in an asynchronous protocol, thus the system was trained in order to identify five different classes referred to as LEFT, RIGHT, UP; DOWN and NULL for the non-stimulus or non-command class. The both a Matlab and a C language versions of the algorithm were developed and were activated by the HIM, with the same results. A classification post processing was inserted in the user interfaces in order to achieve a better usability. The adopted SSVEP protocol was composed of the following phases each one corresponding to a specific user interface implemented with AEnima: the first three are needed for the set-up and for the validation of the classifier, the second group are for free will use and can be considered as real applications. The screening phase was studied in order to understand the best user specific stimulation frequencies. One LED flashed a increasing frequencies between 6 and 17Hz and an off-line analysis GUI allowed the identification of the four best stimulation frequencies. The training or calibration phase guided the user in a four direction sequence studied in order to record a dataset for the calibration of the classifier. The test was an eight symbols sequence for online evaluation of the classifier performances. BCI-User is a main menu for application selection: the user can control a cursor in order to select one of the four applications (AstroBrainFight, Home automation, communication and media player) which are graphically represented by a 3D box with and appropriate texture. The AstroBrainFight (Figure 3) game was studied to test a cursor like control: the user can control a spaceship in order to collide with another one put in a semi-random position of the screen. The task for the user was to reach 15 collision in the minimum time. The Home automation protocol was implemented to demonstrate the usability of the home automation system, set-up in the laboratory. An interface similar to the one described for BCI-user, allowed the user to activate some devices like light and a fan. Figure 3. AstroBrainFight game. The used drives the ship in order to reach the other one The system was tested on ten healthy subjects between 22 and 55 years old, 9 of them were males. The standard testing procedure was composed of: One screening session; One calibration session without feedback; One testing session with feedback; If the testing was successfully carried out: One AstroBrainGame with feedback session; One home automation usability test. During the first part of the testing procedure the subject was verbally instructed about the structure of the graphic interface. In order to increase users participation and the interest in the protocol, they were informed that a hall of fame of the best AstroBrainGame players was being formed. Some subject enjoyed the system and asked for other BCI sessions after the standard one: in those cases the database was updated, but the score was not recorded in the results table. Seven subjects out of ten generated an SSVEP and six of them were able to train a classifier and use the system (subject HE, showed an SSVEP response, but had problems with electrodes). Subject HL was able to use the system, but the test times were not considered in the statistics because he did not completely understand the

5 instruction. The average time to complete the eight symbols test was: 68,2 seconds (std 8,2 s). The best performance was 59 seconds. We have to point out that this speed test was not a measurement of the maximum performance reached by a skilled user with an user optimized classifier, but the transfer rate obtained by a first time BCI user. All the six subject that acceded to the second part of the protocol were able to control the home automation system even after having switched the lights on, thus the system results strong to environmental light variation. Subject SSVEP Frequencies (Hz) Test time Game time MyHome L R U D HA Yes 9,14 6,9 5, Used with lights HB Yes 8 9,10 9,8 11, Used with lights HC No HD No HE Yes X x x x Only screening HF Yes 8 11,1 12,2 14, Used with lights HG No HI Yes 9,14 9,8 11, Used with lights HL Yes 9,8 11,1 12,8 15,1 170* 461* Used HM Yes 8 9,8 11,1 12, Used with lights Table 1. Subject performances using the SSVEP based BCI 4 Discussion & Conclusions The most relevant aspect of the developed framework is the possibility for unskilled developers (eg. Master thesis students) to develop and test their own work and to actively help increasing the number of available instruments in the framework. The usability of the framework was demonstrated in the implementation of the SSVEP based system: during the development of the system a Bioengineering master thesis student without previous C++ programming experience, was successfully involved in the development two new BCI-driven applications (Home automation and AstroBrainFight). When Matlab algorithms are provided, the implementation of a new testing protocol usually takes a few hours for the user interface customization and a few hours for testing purposes; thus evaluation of a new operative protocol isn t constrained by technical issues. The final goal is to make this fascinating technology available at home by any potential user with the help of their family members. The next steps planned in order to reach this objective are in the following directions: software platform distribution; application to disabled people; usability enhancement for people without technological background; availability of low cost acquisition devices. In the immediate future all the software platform will be available free-of-charge to other research group and further efforts will be done in documenting and disseminating the obtained results in order to give a better visibility to this project, in the hope that the community will help to complete the framework in all the specific aspects References [1] Maggi L., Piccini L., Parini S., Andreoni G. and Panfili G., Biosignal acquisition device: A novel topology for wearable signal acquisition devices, Proceedings of Biosignals , January 27-31, Funchal (Portugal) [2] Mazzucco L., Parini S., Maggi L., Piccini L., Andreoni G., Arnone L., A platform independent framework for the development of real-time algorithms: application to the SSVEP BCI protocol, Proceedings of International BCI Workshop 2006, September 21-24, 2006, Graz (AU).