EEG and Video Data Acquisition with ASA. Tutorial

EEG and Video Data Acquisition with ASA Tutorial January 19, 2009 A.N.T. Software BV Enschede, The Netherlands e-mail info@ant-neuro.com phone +31 (0)53-4365175 fax +31 (0)53-4303795 internet www.ant-neuro.com

DISCLAIMER We have attempted to write this tutorial as accurately as possible. However, mistakes are bound to occur, and we reserve the right to make changes to ASA, which may render parts of this manual invalid. We assume no liability as a result of the use or application of this product. No part of this manual may be copied or reproduced without the express permission of the authors. Document version: EEG and Video Data Acquistion with ASA tutorial 1.0.0.7 January 19, 2009 1/19/2009 ANT Software BV, Enschede, Netherlands 2 of 38

Introduction This tutorial intends to introduce you to the EEG Video Data Acquisition with ASA. We have divided the tutorial in two parts: The first chapter relates to EEG/ERP Data Acquisition. In the following chapters 2 to 5 the video view is described. In the first part you will learn how to record EEG/ERP data with ASA. The second part gives you an introduction of the additional video features: how to record video data together with EEG data, how to review video data synchronized with EEG data, and how to edit video data. A.N.T. Software BV Enschede, The Netherlands e-mail info@ant-neuro.com phone +31 (0)53-4365175 fax +31 (0)53-4303795 internet www.ant-neuro.com 1/19/2009 ANT Software BV, Enschede, Netherlands 3 of 38

Table of Contents 1 EEG Data Acquisition with ASA... 5 1.1 Preparation of the ASA Data Acquisition... 6 1.2 Workflow of EEG/ERP Data Acquisition with ASA... 7 1.3 Create a new recording montage... 15 2 Video View... 18 3 Video data recording... 19 3.1 Preparation steps for additional video data recording... 19 3.2 Video Application Options Dialog... 20 3.2.1 Audio Settings control group... 22 3.2.2 Time Settings control group... 22 3.2.3 Video Device Calibration control group... 23 3.3 Video toolbar... 25 3.4 Perform video data acquisition... 26 4 Video Review... 27 4.1 Perform Video-EEG review... 27 4.2 Video toolbar... 28 4.3 Video drop-down menu... 29 4.4 Zooming and shifting... 29 5 Video Editing... 30 5.1 Video Editing Application Options Dialog... 30 5.1.1 Video Edit Output control group... 32 5.1.2 Compressor Settings control group... 32 5.2 Video editing dialog... 33 5.2.1 Video Cutting group... 34 5.2.2 Video Export Settings group... 34 5.3 Video cut markers... 35 5.4 Perform video edit... 35 5.4.1 Set video clip markers... 36 5.4.2 Start video cutting... 37 6 Adding a new experiment... 38 1/19/2009 ANT Software BV, Enschede, Netherlands 4 of 38

1 EEG Data Acquisition with ASA ASA can be extended with the functionality to record EEG/ERP data. In order to record EEG/ERP data, the Acquisition toolbar and the AcquisitionMarkers toolbar need to be enabled (see Figure 1). Figure 1: Acquisition Toolbar (above) and AcquisitionMarkers toolbar (below) in the toolbar section. If these toolbars are not visible in the toolbar section, they can be enabled by using the menu options View / Toolbars / Acquisition Toolbar and View / Toolbars / AcquisitionMarkers Toolbar. 1/19/2009 ANT Software BV, Enschede, Netherlands 5 of 38

1.1 Preparation of the ASA Data Acquisition 1. Connect the EEG amplifier with the power supply 2. Connect the amplifier with the recording PC. In case of record video data together with EEG data prepare and connect the video camera. 3. Switch on the recording PC, amplifier, and power supply of the amplifier 4. Prepare the subject and connect the electrodes (including the ground electrode, GND) with the amplifier according to the recording montage 5. In case you want to record ERP data and apply stimuli to the subject with a separate stimulation PC, start this stimulation PC and prepare the stimulation. Note that the stimulation PC needs to be connected with the amplifier by a cable. 1/19/2009 ANT Software BV, Enschede, Netherlands 6 of 38

1.2 Workflow of EEG/ERP Data Acquisition with ASA Start a new study (menu option File / New). Step 1: Setup Recording Press the button Setup Recording in the Acquisition Toolbar to start the data acquisition procedure with ASA Figure 2: Setup Recording button A dialog opens within ASA (see Figure 3) 1/19/2009 ANT Software BV, Enschede, Netherlands 7 of 38

Figure 3: ASA EEG/ERP Recording dialog 1/19/2009 ANT Software BV, Enschede, Netherlands 8 of 38

Dialog control File Name Meaning File Name is created automatically by ASA Subject Name Subject ID Sampling Rate Montage Cascading Sampling Rate <= max. sampling rate of the amplifier Recording montage; if the appropriate recording montage is not available it must be defined before starting the data acquisition procedure. Note that the recording montage must match with the amplifier configuration. Combination of 2 amplifiers in order to record EEG/ERP data with a large number of channels After you have affirmed or changed the parameters, press the OK button. 1/19/2009 ANT Software BV, Enschede, Netherlands 9 of 38

Step 2: Select the Recording Montage EEG and Video Data Acquisition with ASA Figure 4: EEG online preview. In the preview mode incoming data is only displayed on the screen but not saved into the EEG data file. You recognize the preview mode by the moving current latency marker and the white background color. Furthermore, the preview mode is indicated by Not Recording! in red letters. It is recommended to check the impedances. In order to do this check the Check Impedances control. 1/19/2009 ANT Software BV, Enschede, Netherlands 10 of 38

Step 3: Impedance Check EEG and Video Data Acquisition with ASA The impedance view displays an impedance value for each electrode. You need to wait for a few seconds before the actual impedance values are shown. The numbers shown in the circles are impedance values in kilo Ohm (kohm). Color-coding is used as well. You should try to reach values below 25 kohm (blue color). After all impedance values have reached acceptable levels, uncheck the Check Impedances control and press the Close button to stop the impedance check procedure and to resume to the EEG preview. Figure 5: Impedance Check view showing the impedance values color coded. 1/19/2009 ANT Software BV, Enschede, Netherlands 11 of 38

Step 4: ASA EEG/ERP Recording EEG and Video Data Acquisition with ASA After closing the impedance check, the impedance check dialog is replaced by a dialog to start the recording, i.e. saving the incoming data to the EEG data file. Markers can be set during the recording using the buttons on the AcquisitionMarkers toolbar. In case you record ERP data: When you start the recording by pressing the Start button, the stimulation is started at the same time. You can choose between presenting AEP (auditory) or VEP (visual) stimuli. Figure 6: EEG online view showing the incoming EEG data in recording mode. The recording mode is recognizable by the yellow background color. The recording mode (writing to file) can be stopped by pressing the Stop button. You can start again a recording (a new segment in the resulting EEG data file). In case of recording ERP data, this will be second run. That means AEP events get the trigger codes AEP_2. 1/19/2009 ANT Software BV, Enschede, Netherlands 12 of 38

Select Trigger statistics from the ASA Data Acquisition toolbar to show the trigger statistics of the current ERP experiment. Figure 7: Trigger statistics during an ERP experiment 1/19/2009 ANT Software BV, Enschede, Netherlands 13 of 38

After stopping the recording (select Stop Online Processing from the ASA Data Acquisition toolbar), the EEG data view and the event list is shown in order to review the recorded EEG/ERP data offline. Figure 8: EEG data view (offline) and event list to review the recorded data offline. 1/19/2009 ANT Software BV, Enschede, Netherlands 14 of 38

1.3 Create a new recording montage A montage is an arrangement of EEG channels. In a montage channel labels and properties of channels like sensitivity, trace color, and filter parameters are defined. With help of the Montage editor (menu option Montage / Montages / Edit Montage ) you can create, edit, copy, and delete montages. A recording montage is a montage, which can be used for EEG data acquisition. Each channel has a channel number. The arrangement of the recording montage needs to match with the electrode arrangement of the amplifier used. To create a new recording montage, open the montage editor using the menu option Montage / Montages / Edit Montage Figure 9: Montage editor. In the top left control group Montage, there is the New button to create a new montage. 1/19/2009 ANT Software BV, Enschede, Netherlands 15 of 38

Press the New button in the top left control group to crate a new montage. You can also select a montage, press the Copy button to copy the montage in order to create a montage based on an already existing montage. After having pressed the New button, a dialog opens: Figure 10: Montage editor : New Montage dialog. In the New Montage dialog you can name the new montage. If you want to create a recording montage, check the Recording Montage control. The lower control group Default properties allows you to assign new properties of all channels of the new montage. Press the OK button to save the name and the properties of the new montage. Now you can start to assign channels to the new montage. 1/19/2009 ANT Software BV, Enschede, Netherlands 16 of 38

Figure 11: Montage editor. Assigning channels. To assign channels to the new montage, you can use the right New button and insert manually the channel label. An easier way is to select to click on a label of the left electrode list. This way the label will be transferred to the right channel list of your montage. With help of the Up/Down buttons on the right you can change the sequence of channels, with the Edit button you can change the properties of a channel, and with the Remove button you can remove a channel from you montage after having selected it. After having finished the channel list of the montage, press the OK button (top right). Now you can use the montage. 1/19/2009 ANT Software BV, Enschede, Netherlands 17 of 38

2 Video View ASA makes it possible to record and review video data synchronous with the EEG data from within the same application. Additionally cut off video sequences is available. Figure 12: User interface with Video-EEG review 1/19/2009 ANT Software BV, Enschede, Netherlands 18 of 38

3 Video data recording Video data can be recorded together with EEG data. A camera which can be connected to the recording computer is needed and from within ASA the different video (and audio) devices can be used. If wanted video data from an external camera is written to video files during recording of EEG data. The recording takes place on the same computer where the EEG amplifier is connected. Though the video files don t have to be written to the same machine. Any accessible computer in the local area network can be used as file location. During EEG and video recording video markers are set within the EEG data indicating at which point in EEG video data is available. These markers are vital for the linkage between EEG and video data, because EEG data and video data are written to separate files and the markers are the connection. Annotation: Video files are momentarily written to AVI files only. Because of that it is not possible to record video data to MPEG1/2 format. 3.1 Preparation steps for additional video data recording In case video data shall be acquired simultaneously with EEG/MEG data following steps have to be performed additionally (We assume that you already installed the driver of the video camera): 1. Connect video camera to computer. (In case of additional video compression device the camera has to be connected via the device.) 2. Switch on video camera. 3. Select the menu option Tools / Options... to open the application option dialog and select the Video tab. 4. Set video properties as described in Video Application Options Dialog 1/19/2009 ANT Software BV, Enschede, Netherlands 19 of 38

3.2 Video Application Options Dialog Figure 13: Video tab of the application options dialog 1/19/2009 ANT Software BV, Enschede, Netherlands 20 of 38

General control group Recording Mode: There are 3 options. EEG and Video Data Acquisition with ASA 1. Start With EEG: Video recording is started automatically when starting EEG data acquisition. 2. Start Manually: Video recording has to be started manually by pressing the Video Recording button in the view toolbar. 3. Disable Recording: Video recording is disabled. Video Window Enable Docking: You can check whether the video window shall be dockable like other ASA docking windows or if it shall always be floating on top. Video File Location: Select folder into which recorded video files shall be written. By setting this folder manually to.\, the video files get written into the same location than the EEG data files. Video Settings control group This control group relates to properties referring to the video camera and the hardware/software compressor. Annotation: Uncompressed video data results in enormous file sizes. Therefore you should always use some sort of compression. Because video compression is very performance consuming when doing it via software compression it is strongly recommended to use a hardware compression device. Video Device: The currently used video device which delivers the video stream which shall be recorded. The device can be either a directly to the computer connected video camera or some hardware compression device. Annotation: Compressing video data is performed either by using a hardware compressor or a software compressor. In case of using a hardware compressor you need not to use a software compressor in addition. Video Compressor: If you do not use a hardware compressor, a software compressor is recommended to use. By pressing the arrow button a drop-down list appears listing all video compressors which are available on this computer. By installing additional compressors (often called codecs) these can be used. Compressor Settings button: By pressing the button a dialog appears which lets you configure the compressor parameters. These dialogs are vendor specific and therefore vary in look and set of parameters. Some compressor vendors don t even distribute any dialog and no settings can be done (does only happen rarely). 1/19/2009 ANT Software BV, Enschede, Netherlands 21 of 38

In case a hardware compression device is selected the properties can be set via this button too. Annotation: A compressor reduces the amount of data needed to store video/audio data by using complex algorithms. For reviewing the compressed video files the computer has to know the compression algorithm, it needs a software decompressor. Therefore it is absolutely necessary that the decompressor for a compressed video file is available on all computers where this video EEG shall be reviewed. Framerate button: Sets the number of frames per second at which video data is recorded. A higher frame rate results in bigger file sizes. The default value is set to 25 frames per second. Framesize button: Sets the resolution of the recorded video file in pixel. The default resolution is set to 720x576. Annotation: When using some hardware compression device the settings of frame rate and frame size might be of no effect because the device doesn t support the parameters. Please check video file after first recording or stick with the default settings. 3.2.1 Audio Settings control group The audio settings can be set the same way than video settings. Audio Device: In case of some external hardware device (e.g. hardware compressor) the audio data might come together with the video stream. In that situation an audio device has not to be set (use none/with video). If you want to select another device you are free to do so. Audio Compressor: Sets the software audio compressor. 3.2.2 Time Settings control group Max Video File Duration: Video files result in very big file sizes. Even on modern computers a hard drive might be filled after some hours of video. For that reason and to minimize the risk of data loss, video recording data is split up into more than one data file. Here the maximum duration of one video file in minutes is set. Annotation: During recording ASA automatically closes a video file after the specified amount of minutes. It then creates a new video file with a by one higher count in its name (20060913_1226.000.avi, 20060913_1226.001.avi ). Closing and restarting takes some time and for that period in time there is no video data recorded (about 2 seconds). 1/19/2009 ANT Software BV, Enschede, Netherlands 22 of 38

3.2.3 Video Device Calibration control group This control group relates to synchronizing issues between some external hardware compressor and the recording computer. Device Calibration Factor: In case some tool is available which determines the discrepancy between the hardware device clock and the computer clock, a calibration value can be added here. Normally this value shall be set to 1. Annotation: By setting the value for Max Video File Duration to shorter values (<30 minutes) any discrepancy between device and computer clock is reduced because when a new video files is created it re-synchronizes with the computer system. EEG Video Offset: Because processing of video data takes some time there might be a slight offset between EEG and video data. When using software compressor this value can be set to a default value of 0. In case of using a hardware compressor a default value of -80ms can be set. Annotation: To precisely specify the offset value it might be useful to do a test recording with some VEP flashes. The flash is recorded with the camera and when reviewing EEG together with video data you can compare the point in time where the flash is seen in the video and the trigger event occurs in the EEG data. 1/19/2009 ANT Software BV, Enschede, Netherlands 23 of 38

Video drop-down menu EEG and Video Data Acquisition with ASA While in EEG recording mode some video device settings are accessible through the video drop-down menu. Figure 14: Video drop-down menu while in recording mode Video Sources: In case the connected video device has more than one video input source it can be selected here. A common case is for example an input connector for Composite and S-Video input signals. Audio Sources: The same as video sources only for audio inputs. Property Pages: Here some vendor specific property dialogs are located. Every vendor delivers its own dialogs and parameters to adjust. 1/19/2009 ANT Software BV, Enschede, Netherlands 24 of 38

3.3 Video toolbar Start and stop of video recording can be controlled via the video toolbar. This can be done without having any effect on the EEG recording. Figure 15: Video toolbar during recording Record: Starts a new video file and writes video data to it. (Only accessible if not already recording) Stop: Finishes a currently written video file. (Only accessible if not already recording) 1/19/2009 ANT Software BV, Enschede, Netherlands 25 of 38

3.4 Perform video data acquisition After stepping through previously described Preparation steps for additional video data recording the actual recording of video data can be performed. 1. Start a new acquisition session as described in Data Acquisition. 2. When switching to online mode the video view opens up automatically displaying a preview of the current video stream delivered by the connected camera. 2.1. In case of non-visible preview video data in video view recheck connection of video camera to acquisition computer. 2.2. If still no video data is visible reopen the Video Application Options Dialog and modify settings. Close dialog and check for video preview. 2.3. If still no video data is visible go to the Video menu entry and check for currently selected video source. 3. Start recording to file (EEG and video) exactly the same as described in Data Acquisition. A video marker will be set and in title bar of video view Video Recording plus the duration of the currently recorded video file is displayed. Annotation: The displayed duration is the duration of the current video file, not of the EEG recording duration. In case a new video file is started the duration in title bar is reset. 4. Stop acquisition as described in Data Acquisition. Video recording is stopped simultaneously. 5. Restarting EEG recording (new segment) will automatically restart video recording within a new video file. 6. Recording/Stop of recording of video files only can be done via the Video toolbar. 1/19/2009 ANT Software BV, Enschede, Netherlands 26 of 38

4 Video Review After you have recorded video data together with the EEG data, you can review it in synchrony with the EEG data. You can navigate either using navigation tools of the EEG or navigation or of the video (video toolbar). In each case EEG and video are always displayed synchronized with each other. 4.1 Perform Video-EEG review 1. Open an EEG data file which has video recorded with it. EEG data is displayed and video view opens. 2. Navigating through the data can be done on multiple ways: 2.1. All navigation elements used for review of EEG data can be used as for EEG data without video (change of current latency, paging, jump to event etc.). 2.2. Control of video through video toolbar. 2.3. Usage of video drop-down menu. Annotation: Video data will always be synchronous with EEG data, no matter if navigation is done via video controls or EEG data controls. 3. Within video view you can use zooming and shifting 1/19/2009 ANT Software BV, Enschede, Netherlands 27 of 38

4.2 Video toolbar A toolbar for controlling video navigation is available. Figure 16: Video toolbar during review Play: Starts playback of video file which is present at current point in time. During video file playback the current latency within ASA gets updated and therefore moves with the video latency. When playback reaches end of one video file it automatically jumps to the next video file continuing playback with that file. Stop: Stops playback of currently presented video file. When pressing the stop button the current EEG/video latency jumps back to the beginning of the current video file, not to the beginning of EEG. Pause: Pauses playback of currently running video file. Latency will remain at current position and pressing playback will continue from that position. Step Forward/Backward: Using these two buttons the user can step through the video and EEG data video frame by video frame. Annotation: For example the video file was recorded with 25 frames per second. Than pressing stepping button once moves the current latency by 1/25=40ms forward or backward in time. Speed: Changes playback speed to ¼, ½, 1, 2, 4 times of real time. 1/19/2009 ANT Software BV, Enschede, Netherlands 28 of 38

4.3 Video drop-down menu Some of the controls accessible through the video toolbar are also located in the video drop-down menu. Figure 17: Video drop-down menu during review 4.4 Zooming and shifting During reviewing and recording of video data it is possible to zoom into the video and shift the video picture. Zooming: Zooming into the picture is done by left-clicking with the mouse into the video view. To zoom out of the picture just right-click the mouse within the video view window. To reset the picture to its original size just press the middle mouse button (press mouse wheel). Shifting: To shift just keep the left mouse button pressed and move the mouse. The picture will move with the mouse cursor. 1/19/2009 ANT Software BV, Enschede, Netherlands 29 of 38

5 Video Editing After long-term recordings it often is useful to cut away those pieces of video which are of no interest, because video files need a lot of storage space. That means you mark interesting EEG/video parts with video clipping marks and then only keep those parts. 5.1 Video Editing Application Options Dialog Select the menu option Tools / Options... to open the application options dialog. Select the Video Editing tab. 1/19/2009 ANT Software BV, Enschede, Netherlands 30 of 38

Figure 18: Video editing application options dialog 1/19/2009 ANT Software BV, Enschede, Netherlands 31 of 38

5.1.1 Video Edit Output control group When using video editing, new video files will be created. You can use the origin of the original video files or select another output directory for those newly created files. Output Directory: Directory into which the cut off video files are stored. By setting this folder manually to.\, the video files get written into the same location than the EEG data files. 5.1.2 Compressor Settings control group The cut off pieces of video need to be recompressed. Video and audio compressor and its settings can be done here. Annotation: Usage of original video file format is not supported yet. 1/19/2009 ANT Software BV, Enschede, Netherlands 32 of 38

5.2 Video editing dialog To access the video editing dialog go to Video / Edit Video. A dialog pops up. Annotation: Settings from this dialog which also exist in Video Editing Application Options Dialog can be changed here too, but won t be persisted. Next time you open the video editing dialog the settings will be reset to application settings done in application options dialog. Figure 19: Video editing dialog 1/19/2009 ANT Software BV, Enschede, Netherlands 33 of 38

5.2.1 Video Cutting group You can determine that all set video cut marker shall be taken to cut off video sequences (Use all Video Cut Marker) or only those video cut marker which are selected are taken into account (Use Only Selected Video Cut Marker). Annotation: After one video cutting has been performed there are no more original video files existing. When only cutting for selected marker is done the other cutting markers won t have any usage as video cut markers any more, because there is no original data to be cut. Delete Original Video Files: By checking this option all original video files will be physically deleted and marker file will be overwritten. If unchecked the original will be kept but renamed to original file name plus.orig as extension. Those files can be accessed by renaming them manually to.avi extension. Annotation: Because linkage between EEG and video data is done through the video markers it is important to know that loosing or manipulating the EEG event file leads to a loss of possibility to review EEG synchronous with video data. That s why a copy of the event file is also kept when deletion of original video files is disabled. 5.2.2 Video Export Settings group The cut off pieces of video need to be recompressed. Video and audio compressor and its settings can be done here. Annotation: Usage of original video file format is not supported yet. 1/19/2009 ANT Software BV, Enschede, Netherlands 34 of 38

5.3 Video cut markers Video cut markers are used to determine intervals for which video data shall be kept. Basically a video cut marker is the same than event marker. That s why the handling and setting is the same. Only shifting after marker has been set is not allowed and the markers have other default names. Figure 20: Event bar with video cut marker 5.4 Perform video edit Video editing is done in a 3-step way: 1. Parts of interest within the video have to be determined by setting video cut markers in EEG view. 2. Cutting has to be started via the video editing dialog. 3. ASA generates new video files and modifies marker file (original video markers will be deleted). 1/19/2009 ANT Software BV, Enschede, Netherlands 35 of 38

5.4.1 Set video clip markers 1. Mark a time interval which should be kept as video clip, navigate to this latency in the EEG data view. Select the video cut marker from the events toolbar. 2. Mark a time interval in the EEG view. When asked to type in a description you can type one or leave it out. Than the default name Video Cut Marker plus a number will be set. Annotation: You can set multiple cutting markers, but don t set overlapping ones. For one point in time there can only exist one video file. If you set a marker over the border of one segment a warning will appear and the end of the marker will be set to the end of the first segment. Figure 21: Warning message 1/19/2009 ANT Software BV, Enschede, Netherlands 36 of 38

5.4.2 Start video cutting 1. Open video editing dialog. 2. Set options in dialog as described in video editing dialog. 3. Start cutting by pressing OK. 3.1. In case Delete Original Video Files is enabled a dialog will appear warning you that all original video files will be deleted. Figure 22: Warning message 4. If OK was pressed actual cutting starts and a progress dialog is displayed. Figure 23: Editing progress dialog 5. After processing is completed the view gets refreshed and all original video markers and files have vanished from the user interface (no matter if deletion was enabled or not). Only selected cutting sequences remain. 1/19/2009 ANT Software BV, Enschede, Netherlands 37 of 38

6 Adding a new experiment By installation, experiments AEP, VEP, and P300 can be used. In order to use other experiments, e.g. those you know from cognitrace, Simply copy the *.vbs (for example SEP.vbs for SEPs) file from cognitrace to the folder..\asa4\data\scripts\data Acquisition\Stimulation and rename with the script file eemagine to ASA. 1/19/2009 ANT Software BV, Enschede, Netherlands 38 of 38