Mobile Application Development (Part 1)-Program a Sound-Monitoring Prototype

Transcription

1 See discussions, stats, and author profiles for this publication at: Mobile Application Development (Part 1)-Program a Sound-Monitoring Prototype Article December 2010 CITATIONS 0 READS 7 3 authors, including: Vahid Garousi Wageningen University & Research 146 PUBLICATIONS 1,113 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: Software test automation View project IndAcSE: Meta-analysis of Industry-Academia Collaborations in Software Engineering View project All content following this page was uploaded by Vahid Garousi on 09 February The user has requested enhancement of the downloaded file.

2 F EATURE ARTICLE by Riley Kotchorek, Mike Smith, & Vahid Garousi (Canada) Mobile Application Development (Part 1) Program a Sound-Monitoring Prototype Mobile medical devices are poised to change how healthcare professionals service their patients need. This series details how to develop a prototype Windows Mobile application for such a purpose. In this article, you learn about programming a sound-monitoring prototype used to identify medical issues related to sleep apnea. Editor s note: Riley Kotchorek wrote this article, with contributions by Professors Mike Smith and Vahid Garousi. S park describes itself as a blog, radio show, podcast, and an ongoing conversation about technology and culture (cbc.ca/spark). The host, Nora Young, certainly does not come across as a techno geek, but she has interviewed some pretty tech-savvy people since 2007! It was Nora s interview with Tapani Salmi (Podcast 49) about the HappyWake- Up cell phone application that got us excited. The podcast explained how HappyWakeUp (happywakeup.com) used a cell phone to monitor the sounds of a sleeper s movement for the last 30 minutes before a planned wake-up alarm call. This time consists of alternating deep sleep and arousal periods. If you wake up during an arousal time, you are refreshed. If you wake up during a deep sleep period, you switch between happy and grumpy! When I joined Drs. Smith and Garousi s research group at the University of Calgary (Canada) in the summer of 2009, I was informed that, We reckon that HappyWake- Up might just be the surprise killer application of If people could automatically get up on the right side of the bed in the morning, then it might change the mood of the world. Perhaps there would be no more road rage! Photo 1 The first prototype requires that a simple message to be displayed in the application s window. However, I Dr. Garousi sighed wistfully, The application only runs on Nokia and iphones. That means I can t use it as a case study for my course Testing for New Technologies. We only have Windows Mobile phones in the teaching laboratories. You ll just have to write the code for an application we can use! Perhaps the conversation did not exactly happen that way. But you get the gist. It s why I ended up writing a prototype medical mobile application which I call The Snorecognizer to help determine how often people snore when taking a nap after eating supper (see Photo 1). This application may sound a bit crazy; however, it addresses a problem that is real and capable of causing considerable financial losses at both business and societal levels. Snoring is related to sleep apnea, which is a major health issue across the world. I developed a series of mobile applications to form the first practical part of a software engineering course about the reliable development of mobile medical devices. These applications are intended be futher developed by future graduate students when investigating smart phone aids for addressing sleep-deprivation issues. (If you are interesting in learning more about the impact of 16 CIRCUIT CELLAR

3 Photo 2 New Project Wizard settings in order to create a C++ project for a Windows Mobile device Listing 1 The implementation of a basic Hello World! operation inside the WM_PAINT case of the switch(wmessage) statement in the WndProc() callback function in our mobile application 1 LRESULT CALLBACK WndProc(HWND hwnd, UINT wmessage, 2 WPARAM wparam, LPARAM lparam) { 3 // WndProc variable declarations 4 PAINTSTRUCT ps; 5 HDC hdc; 6 TCHAR outputstring[ ] = L"Oy you you re snoring!"; 7 8 // switch statement to interpret the message from Windows OS 9 switch(wmessage) { // Code for handling wmessages sent the when application 12 // starts and finished 13 case WM_CREATE: etc; break; case WM_DESTROY: etc; break; // A WM_PAINT message occurs when the operating 18 // system wants to refresh or "paint" a window 19 case WM_PAINT: 20 // Prepare window for painting and get a handle to the DC 21 hdc = BeginPaint(hWnd, &ps); // Get info on application window's size 24 RECT windowsize; 25 GetClientRect(hWnd, &windowsize); // Immediately draw the text to the known screen area 28 DrawText(hdc, outputstring, 29-1, windowsize, DT_CENTER DT_WORDBREAK); // End the painting session for the window 32 EndPaint(hWnd, &ps); break; 35 } // End switch(wmessage) 36 return 0; 37 } // End WndProc( ) sleep deprivation, I recommend books such as Sleep Thieves by Stanley Coren, Free Press.) This article covers the first part of my work, building the initial Snorecognizer prototype at the Hello World level. I had to demonstrate how to write software to use the many basic features available on a PDA or smart phone. This meant the prototype would have to be able to accept input commands from the available keys and display text, as well as handle basic audio operations. By doing some basic DSP analysis on the monitored input sounds, the final cell phone Snorecognizer outputs graphs on the maximum number of snores per snooze and other useful pieces of information. CELL PHONE OS Before saying Hello to this new world, I had to choose a target platform and an environment in which to develop. There are many different smart phone and PDA platforms available, each with its own pros and cons. Most of the existing mobile operating system (OS) options have free or inexpensive development environments. Some platforms run Java-based applications, while others run C, C#, or C++. With all of the different choices available, I chose Windows Mobile because the University of Calgary had licenses for Microsoft s Visual Studio 2008 in place. The code could be first tested using the emulators that come packaged with the Windows Mobile software development kits (SDKs). Then the code could be moved to UTStarcom devices for use in the classroom. Another advantage was that Windows Mobile applications could be developed using the same C/C++ language that the undergraduates other embedded system courses used. A full version of Microsoft Visual Studio can be expensive for somebody who just wants to start playing around with mobile devices. However, a fully functional, 90-day trial version of Visual Studio 2008 Professional is available on Microsoft s website ( CIRCUIT CELLAR 17

4 Photo 3 This screenshot shows the options available in the drop down target device list. The only difference between the two emulator options is that one has a square display and the other has a standard rectangular display. The third option runs the code on a physical device connected via ActiveSync or Windows Mobile Device Center. A useful source of code ideas are available in the Microsoft Developer Network function descriptions/usage guides (msdn.microsoft.com). The Windows Mobile SDKs are also available on the same download site at no charge. Remember to get the SDK that matches your target device s version of Windows Mobile. For example, if you have an HP ipaq RX5910 GPS device running Windows Mobile 5.0 for Pocket PC, you you ll need the Windows Mobile 5.0 SDK for Pocket PC. WORKING PROTOTYPE Getting started can be the most difficult part of a new project. Here I ll detail how to make a basic mobile windowed application that displays a message Oy you you re snoring! on the mobile device s screen (see Photo 1). To begin, you create a new project in Visual Studio by selecting File > New > Project in the menu bar. Next, choose Visual C++ Smart Device as the project type (see Photo 2), and then select Win32 Smart Device Project as the template. After clicking Ok, select the software development kit that corresponds to the device s Windows Mobile version. My device uses Windows Mobile 5.0 for Pocket PC, so I used Windows Mobile 5.0 Pocket PC SDK. The application type should be set to Windows Application. Clicking the Finish button causes Visual Studio to generate a large portion of the code. The WinMain() function, the entry point into the application, continuously listens to the messages sent to it by the mobile operating system when events (interrupts) occur. These messages are then dispatched to the WndProc() callback function. This main user code entry point is where the messages are deciphered and so the appropriate actions can be taken. The default WndProc() procedure generated by Visual Studio is nothing more than a large switch statement (see Listing 1) based on which event message (wmessage, line 9) has been sent by the mobile operating system to the application. Standard cases are called when the application starts and finishes (WM_CREATE, line 13, and WM_DESTROY, line 14). For our code, I wanted to make the WndProc() function responsible for posting the greeting message in the application s main window (see Photo 1). When the OS receives a request to recreate this window, it sends a WM_PAINT message to the application. Deciphering of this incoming wmessage message is done in the WndProc() function by the WM_PAINT case in switch(wmessage) shown in Listing 1. The case statement starts by calling the BeginPaint() function (line 21). This function takes a handle to the window (hwnd) and a pointer to a PAINTSTRUCT structure. It returns a handle to information about the device context (DC). This DC defines the screen area that will be affected by the paint operation, which in this case is the client area of the mobile device s window. Next, you need details of the area on the screen into which you want to place text. For this particular case, call GetClienterct() (line 25) and pass in a handle to the window. The result is a rectangle variable, windowsize, filled with details about the client window. DrawText() is then called to display Oy you you re snoring! in the area defined by the rectangle and the device context (lines 28 and 29). The L leading the string passed to the DrawText() function is an indication that this string should be created with wide characters (TCHAR Unicode symbol) that take 2 bytes of storage instead of just one. Using TCHAR means that the application can be adapted to a wide range of languages. The third parameter, 1, indicates the use of a null-terminated string and avoids the need to specify the precise length of the string. The DT_CENTER and DT_WORDBREAK flags passed in the fifth argument to DrawText() tell the system to center the text horizontally and wrap the text to the next line if it goes outside of the rectangle specified in argument four. Each call to BeginPaint() must be followed by an EndPaint() call (line 32) to complete the painting processes. These functions can be called only inside the WM_PAINT case. DEBUGGING Once the basic Snorecognizer prototype was coded, I needed to test it. There are two approaches to debugging the code: running the application 18 CIRCUIT CELLAR

5 Listing 2 A new WM_KEYDOWN case in the switch(wmessage) statement inside the WndProc() function changes the message displayed based on user input via buttons 1 LRESULT CALLBACK WndProc(HWND hwnd, UINT wmessage, 2 WPARAM wparam, LPARAM lparam) { 3 // WndProc variable declarations 4 static TCHAR outputstring[100] = L"Welcome to the Snorecognizer"; 5 // etc 6 7 switch (wmessage) { 8 9 // VK_RIGHT and VK_LEFT are virtual-key codes for the 10 // right and left keys (set by the operating system) 11 case WM_KEYDOWN: 12 switch(wparam) { case VK_RIGHT: // Change global output string 15 wsprintf(outputstring, L"Recording the audio ); 16 break; case VK_LEFT: 19 wsprintf(outputstring, L"Playing back recorded audio"); 20 break; // Let all other keys retain their standard functionality 23 default: 24 return DefWindowProc(hWnd, message, wparam, lparam); 25 } // switch(wparam) // Tell the operating system that hwnd window needs updating 28 InvalidateRect(hWnd, NULL, TRUE); 29 break; // New WM_PAINT message displays text from a local static object. 32 case WM_PAINT: 33 hdc = BeginPaint(hWnd, &ps);. break; // Code for interpreting other wmessages case WM_CREATE: etc; break; } // End switch(wmessage) } // End WndProc( ) on an emulator or checking it later when a real device available to work with. Choosing whether to run an emulator or debug an application directly on a physical device can be done in the target device drop-down menu (see Photo 3). If you can t find the target device menu amongst your toolbars, it can be enabled by selecting View > Toolbars > Device from the menu bar. You can begin running the application in one of the Windows Mobile device emulators supplied as part of the Microsoft SDKs. You can do this by first selecting Build > Build Solution from the menu bar and fixing any errors that pop up. The second step is to choose Debug > Start Debugging from the same menus. The appropriate Windows Mobile emulator will activate and, after the program has been transferred to the emulator, it will display the message (see Photo 1). Click the Ok button (see Photo 1) to stop the application. Since this is a mobile device, clicking the X at the top of the screen just minimizes the application rather than stopping it as would happen on a desktop. Remember that minimized mobile applications still consume power. An emulator is all fine and dandy, CIRCUIT CELLAR 19

6 20 but on-device debugging is the proverbial meat and potatoes when testing mobile applications. To do this, you you need three more things: the free program Microsoft ActiveSync if running under XP or Windows Mobile Device Center if you re programming on Windows Vista; a cable to connect the device to the computer; and a mobile device itself. In our case, we first used the GPS-capable HP ipaq RX5910 loaded with Windows Mobile 5.0 for Pocket PC donated by a local company. To get started with on-device testing, first connect your device to the computer and wait until ActiveSync/Windows Mobile Device Center recognizes and syncs to it. Next, back in the Visual Studio development environment, select your project again. However, this time, choose the device option from the drop-down target-device selection menu (see Photo 3). Now simply follow the same build/debug procedure we used when running on an emulator. KEY STROKE RECOGNITION The final Snorecognizer project will need more than just the ability to print out a simple warning message such as Oy you you re snoring. The user will need to use the mobile device s keys to control its operation. We can demonstrate this additional functionality by having the application output different messages to the screen according to which button has been pressed. Unlike a desktop or laptop with its keyboard, mobile devices have relatively few buttons. Pressing any of these keys causes a WM_KEYDOWN message to be sent to the WndProc() function. Refer to Listing 2 (lines 11 to 29) for details on how the WM_KEYDOWN message is handled. Detailed information about each WM_KEYDOWN message is contained in the wparam and lparam arguments passed to WndProc() (line 1). The wparam variable contains a code corresponding to which key has been pressed. The lparam variable contains additional information about the key press, such as whether or not the key has been pressed and released Listing 3 Modifying the WM_KEYDOWN case of the switch(wmessage) statement inside WndProc() function starts and stops the audio devices based 1 // Mask used to check if a key is being held down 2 #define PREVIOUSLY_DOWN_MASK 0x // WndProc variable declarations 4 static int recording = 0, playing = 0; 5 static HWAVEIN hwavein; 6 static HWAVEOUT hwaveout; 7 static WAVEHDR waveoutbufferheader; 8 static WAVEHDR waveinbufferheader; 9 // etc switch (wmessage) { 12 case WM_KEYDOWN: 13 // Check the PREVIOUSLY_DOWN bit of lparam to ensure we do 14 // not repeat actions if the button is held down 15 if ((lparam & PREVIOUSLY_DOWN_MASK) == 0) { 16 switch(wparam) { 17 case VK_RIGHT: //The user has selected record 18 // Record if not already recording or playing 19 if (!playing &&!recording){ 20 recording = 1; 21 waveinaddbuffer(hwavein, &waveinbufferheader, 22 sizeof(wavehdr)); waveinstart(hwavein); wsprintf(outputstring, L"Recording Audio"); 27 InvalidateRect(hWnd, NULL, TRUE); 28 } 29 // else if we are currently recording and want to stop 30 else if (!playing && recording) { 31 recording = 0; 32 waveinreset(hwavein); 33 } 34 break; case VK_LEFT: //The user has selected play 37 // If we aren't already recording or playing 38 if (!recording &&!playing){ 39 playing = 1; 40 waveoutwrite(hwaveout, &waveoutbufferheader, 41 sizeof(wavehdr)); wsprintf(outputstring, 44 L"Playing back recorded audio"); 45 InvalidateRect(hWnd, NULL, TRUE); 46 } 47 //else if we are currently playing and want to stop 48 else if (!recording && playing) { 49 playing = 0; 50 waveoutreset(hwaveout); 51 } 52 break; default: 55 return DefWindowProc(hWnd, message, wparam, lparam); 56 } // End switch(wparam); 57 } // if ((lparam & PREVIOUSLY_DOWN_MASK) == 0) 58 else 59 return DefWindowProc(hWnd, message, wparam, lparam); break; // End case WM_KEYDOWN: case WM_PAINT: etc; break; } // End switch(wmessage) CIRCUIT CELLAR

7 Listing 4 New code inserted into the WM_CREATE case in the WndProc() function s switch(wmessage) statement to initialize the buffer and audio devices 100 case WM_CREATE: 101 // Setup the initial message and state variables 102 wsprintf(outputstring, 103 L"Welcome to the Snorecognizer"); 104 recording = 0; 105 playing = 0; // Zero the buffer header structures (will cause errors 108 // if not done) 109 memset(&waveinbufferheader, 0, sizeof(wavehdr)); 110 memset(&waveoutbufferheader, 0, sizeof(wavehdr)); // Allocate and zero the audio buffer 113 waveinbufferheader.lpdata = new char[audiobuffersize]; 114 waveoutbufferheader.lpdata = waveinbufferheader.lpdata; 115 waveinbufferheader.dwbufferlength = AUDIOBUFFERSIZE; 116 waveoutbufferheader.dwbufferlength = AUDIOBUFFERSIZE; memset(waveinbufferheader.lpdata, 0, AUDIOBUFFERSIZE); // Setup the audio format based on pre-defined values 121 WAVEFORMATEX waveformat; 122 waveformat.wformattag = CHOSEN_WAVE_FORMAT; 123 waveformat.nchannels = CHANNELS; 124 waveformat.nsamplespersec = SAMPLES_PER_SEC; 125 waveformat.navgbytespersec = AVG_BYTES_PER_SEC; 126 waveformat.nblockalign = BLOCK_ALIGN; 127 waveformat.wbitspersample = BITS_PER_SAMPLE; 128 waveformat.cbsize = CBSIZE; // Open connections to the audio devices 131 waveinopen(&hwavein, 0, &waveformat, (DWORD) hwnd, 0, 132 CALLBACK_WINDOW); 133 waveoutopen(&hwaveout, 0, &waveformat, (DWORD) hwnd, 0, 134 CALLBACK_WINDOW); // Prepare the headers 137 waveinprepareheader(hwavein, &waveinbufferheader, 138 sizeof(wavehdr)); 139 waveoutprepareheader(hwaveout, &waveoutbufferheader, 140 sizeof(wavehdr)); // Code already in WM_CREATE (generated by Visual Studio) 143 // to handle the application menus break; or is being held down. (We will need this information later on.) KEY USE To demonstrate using the keys, we modified certain aspects of the Wnd- Proc() function (see Listing 2). Let s review. To enable the WM_PAINT event to display different messages, we are able to change the TCHAR string array used by the DrawText() function inside the WM_PAINT case statement. As a result, we can modify what is displayed on the screen when a button is pressed. Now we can create a new case inside the switch(wmessage) statement of WndProc() to handle WM_KEYDOWN messages. This gives the application the ability to recognize a key press. Inside this new case statement, wparam was compared to the Windows constants VK_RIGHT (right arrow, line 14) and VK_LEFT (left arrow, line 18) in order to decipher which key had been pressed. Our outputstring (line 4) can be modified to display a different message using wsprintf(). The CIRCUIT CELLAR 21

8 wsprintf() function differs from the standard sprintf() write-to-astring-buffer function in that it handles the wide Unicode-enabled characters rather than standard 8-bit characters. The default case of the WM_KEY- DOWN message (line 23) activates the DefWindowProc() function. This informs the OS that it is to perform the default action corresponding to the current key press. This is a better programming style than just setting the default case code to the following: default: /* DO NOTHING */ break; This would not allow any other device keys to function properly. We can force the mobile OS to issue a WM_PAINT message to update the device s display window. This is done in line 28 by passing the InvalidateRect() function a handle, hwnd, to the window we want to invalidate (redraw). Setting the second parameter to NULL causes the window s entire client area to be updated, rather than just part of it. The last argument of InvalidateRect() is set to TRUE so as to erase the background before painting over it. Setting this parameter to FALSE will cause the new text to be written without erasing the previously displayed message first. Running the program with these changes will display the welcome message on the screen initially. Pressing the left or right arrow keys will cause a change between playback and recording messages, respectively. AUDIO I/O DEVICES The final stage in developing the prototype has to dow with using the mobile OS waveform audio input and output devices. In the case of Windows Mobile, the built-in microphone and speakers are the devices used. Listing 3 shows the modification of the WM_KEYDOWN events so that the application will record a sound and then play it back. Four functions Listing 5 Definitions used to set the audio format and audio buffer size placed inside Snorecognizer.h file, which is included in the main Snorecognizer.cpp file. // Pulse Code Modulation format #define CHOSEN_WAVE_FORMAT WAVE_FORMAT_PCM // Mono audio with 44.1-kHz sampling rate, 2 bytes per sample #define CHANNELS 1 #define SAMPLES_PER_SEC #define BITS_PER_SAMPLE 16 // 2 bytes per block #define BLOCK_ALIGN (((CHANNELS) * (BITS_PER_SAMPLE)) / 8) // 88,200 bytes per second #define AVG_BYTES_PER_SEC ((SAMPLES_PER_SEC) * (BLOCK_ALIGN)) // 5-second audio buffer size #define AUDIOBUFFERSIZE (5 * AVG_BYTES_PER_SEC) // Size in bytes of any extra space at the end of the // WAVEFORMATEX structure for additional user info #define CBSIZE 0 need to be controlled: start recording, stop recording, start playback, and stop playback. The user interface would be intuitive if we used one press of the right key to start recording, and a second press to stop the recording. Similarly, we use the left key to start and stop the playback. We ll also initially limit the audio operation to recording and playing back five seconds of sound into a specified buffer. The application stops recording/playback either when you tell it to (second key press) or when it reaches the end of the buffer, whichever comes first. The first lines in Listing 3 show static variables needed for the new WndProc() function. The state variables playing and recording will keep track of the application s current play/record state. Device handles to the audio in/out devices hwavein and hwaveout are used to identify each device (lines 5 and 6). The WAVE- HDR structures will hold information regarding the audio buffer, as well as a pointer to the buffer itself. We use one buffer to hold the audio information. This means that in the prototype, you can t play and record at the same time. Two WAVEHDR structures are required: one header used by the audio-in device and the other is used by the audio-out device (lines 7 and 8). Both headers will use a pointer to the same audio buffer. We can now make use of the lparam variable, which contains additional information about the WM_KEY- DOWN message. In Listing 2, we changed the display for each WM_KEYDOWN message. It does not really matter if you have been accidently holding down a key for long enough to activate its repeat Listing 6 New cases added to the WndProc() function s switch(wmessage) statement to handle messages sent when the audio devices finish reading/writing their buffers //case for when the waveout device has finished playback 60 case MM_WOM_DONE: 61 wsprintf(outputstring, L"Audio recording complete"); 62 InvalidateRect(hWnd, NULL, TRUE); 63 break; //case for when the wavein device has finished recording 66 case MM_WIM_DATA: 67 wsprintf(outputstring, L"Playback complete"); 68 InvalidateRect(hWnd, NULL, TRUE); 69 break; 22 CIRCUIT CELLAR

9 functionality. Now, in this new prototype, we don t want to recognize a second WM_KEYDOWN event until you have physically released the key. We can avoid this repeat operation gotcha by checking the PREVI- OUSLY_DOWN_MASK bit of the lparam (line 15). If this bit is not zero, we don t want to re-execute the case statement as the application is in the repeat keystroke mode. If the right key has been pressed for the first time, we want to start recording (line 17). This involves using waveinaddbuffer() to tell the OS to add a new buffer to the audio input (wave) queue (line 21). Then the audio device is activated (waveinstart(), line 24) and the user informed that recording has started. A second press of the right key causes the recording to stop (waveinreset(), line 32). In a similar way, the left key is used to control the wave output device. The waveoutwrite() function handles both queuing the buffer and starting the output audio device (line 40). AUDIO DEVICE INITIALIZATION When the window for the application is first created, a WM_CREATE message is sent to the WndProc() function by the OS. In the default code, WM_CREATE is used to create the OK and HELP menu at the bottom of the application window (see Photo 1). Additional code must be placed in this case statement to set up the audio devices (see Listing 4). The inserted code has a straightforward logical flow. The string and state variables are set to their initial values. Both input and output buffer headers are then zeroed to avoid causing errors down the road. The lpdata member of the WAVEHDR structures will point to the audio buffer, which we have to allocate. Although the audio buffer is made up of signed short integer (16-bit) values, the dwbufferlength value must be expressed in bytes another gotcha to be avoided in lines 115 and 116. The buffer itself needs to be zeroed to avoid producing painful sounds when the output device is first activated. Next, we fill a WAVEFORMATEX structure with audio format information defined in Listing 5. The format structure is then passed to the waveinopen() and waveoutopen() functions, which specifies our selected audio format to the audio devices (lines 131 and 133). Listing 7 New code inserted into WM_DESTROY case in the WndProc() function s switch(wmessage) statement to cleanup allocated memory and close audio device connections. 70 case WM_DESTROY: 71 // Force the devices to stop what they are doing 72 // and return the queued buffer(s) to the 73 // application 74 waveoutreset(hwaveout); 75 waveinreset(hwavein); // Unprepare both headers 78 waveoutunprepareheader(hwaveout, &waveoutbufferheader, 79 sizeof(wavehdr)); 80 waveinunprepareheader(hwavein, &waveinbufferheader, 81 sizeof(wavehdr)); // Close the device connections 84 waveoutclose(hwaveout); 85 waveinclose(hwavein); // Remove the audio buffer 88 delete [] waveinbufferheader.lpdata; // Code already in WM_DESTROY (generated By Visual Studio) // to destroy the basic application menu break; CIRCUIT CELLAR 23

10 The waveinopen() and waveoutopen() functions open handles to the microphone and speakers, respectively. We pass a pointer to the handle we wish to populate (hwavein or hwaveout) as the first argument to these functions. The second argument specifies which audioin/out device we would like to use, although there s a good chance your device only has one. The audio format is specified by the third argument a pointer to our WAVEFORMATEX structure. The audio devices themselves are capable of sending messages (much like WM_KEYDOWN) to a callback mechanism. The fourth and sixth arguments of waveinopen() and waveoutopen() tell the system to send any messages from the audio devices to our application s WndProc() function. The fifth argument is saved for any custom data and is not currently used. Now that we have opened the device handles, we must prepare the buffer headers. This is done using the waveinprepareheader() and waveoutprepareheader() functions (lines 137 and 138). We pass the device handles, hwavein and hwaveout, to the functions to identify which devices we are referring to. The second argument is a pointer to the header we want to prepare. In addition, we must specify the size of the header, which is passed in the third argument. COMPLETED BUFFERS Now that we are in the position to be able to start recording and playback, how can we tell if an audio device has completed filling or emptying its buffer? We need this information to be able to update the screen with a done message to the user, but how? The route to solving this problem is the parameter CALLBACK_WINDOW used in the waveinopen() and waveoutopen() functions in Listing 4 (lines 132 and 134). This specifies that our program window is to be the callback mechanism for any messages from the audio devices. Because of this, when the wave-out device finishes with a buffer, it will send a MM_WOM_DONE message to the WndProc() function. We can create a new case inside the main WndProc() switch statement (see Listing 6, Line 60) for this message in order to update outputstring. Equivalently, the MM_WIM_DATA (line 66) message is sent when the wave-in device has finished with its buffer. APPLICATION CLOSEDOWN With the current code, the mobile device will be left in an inoperable situation for other applications when the user clicks Ok to exit the Snorecognizer prototype. For example, we have allocated a buffer in WM_CREATE. If this is not released, then exiting the code will generate a memory leak. In addition, if we prepare a buffer header for a device and forget to unprepare it, it will remain with the device even if our program exits! Once the audio device has been filled with prepared headers, it will refuse additional buffer headers, requiring a hard reset of the system. In order to avoid these troublesome errors, we should ensure our program properly cleans up after itself. (It reminds me of what my great-great aunt Agatha always used to tell me about tidying my room.) We will have to release the audio buffer, unprepare the buffer headers, and close the audio device handles. All of this can be accomplished inside the WM_DESTROY case of the WndProc() function s main switch statement (see Listing 7). A WM_DESTROY message is sent to WndProc() whenever the window is destroyed (line 70). This mirrors the initialization code we wrote for the WM_CREATE message. The main gotcha to avoid is to release things in WM_DESTROY in the exact opposite order in which they were created in WM_CREATE. For example, the handles become useless after they have been closed. With clean code, the window can close without worry of a memory leak or other errors. WHERE TO NEXT? In this article, I described a simple prototype for the Snorecognizer Windows Mobile application. So far, the design handles basic key input/screen output and controls the on-board audio devices on a Windows Mobile device. I hope you enjoyed this article. In the next article, I ll delve further into the world of Windows Mobile and snore recognition. I will explain how to expand the prototype s functionality to include simple audio DSP, handling audio operations over a long period of time, and even showing our grasp of general graphing of the results! I Authors Note: We would like to thank DirectVoxx and Telus for arranging the donation of the mobile equipment used in this article. The Snorecognizer application was a spin-off from Riley s research on the development of testing tools for mobile devices. This work was supported through a Natural Sciences and Engineering Research Council (NSERC) of Canada undergraduate student research scholarship and by additional funding through Dr. Garousi s Award Alberta Ingenuity New Faculty Award no We also thank Lizie Dunling-Smith for her editorial suggestions in making the article more readable. Riley Kotchorek (riley.kash@gmail.com) is an. RILEY PLEASE UPDATE YOUR INFO HERE. When he s not playing the latest video game, he enjoys programming in C# and Java. He will be returning to the University of Calgary, Canada. in computer engineering. Mike Smith (smithmr@ucalgary.ca) has a strong interest in the fields of Biomedical and Computer Engineering and holds a firm belief that agile methodologies, such as test-driven development, are a route to lower numbers of defects in mobile systems. Mike s sense of humor often appears in his frequent contributions to Circuit Cellar. He is a professor at the University of Calgary, Canada and is director of the SMILE laboratory (Small Microsystems for Improving Life Expectancy). Vahid Garousi (vgarousi@ucalgary.ca) is an Assistant Professor of Software Engineering at the University of Calgary. He leads 24 CIRCUIT CELLAR

11 a team of research associates and students in the Software Quality Engineering Research Group (SoftQual). This group focuses on techniques to improve upon existing methodologies and develop new approaches in the production of industrial software. PROJECT FILES To download the code, go to ftp://ftp.circuitcellar.com/pub/ Circuit_Celar/2010/245. RESOURCES J. Buiting and L. Lemmens, Explorer-16 Value Pack and audio signal processing, Elektor, Microchip Technology, PIC24FJ64GA004 Family Data Sheet, DS39881B, Radiometrix, UHF Radio Telemetry Receiver Module, SILRX-UHF, 2001., UHF Radio Telemetry Transmit Module, TXM-433, R. Richey, Adaptive Differential Pulse Code Modulation using the PIC16/17, Microchip Technology, AN643, SOURCES LTC-1865 ADC Linear Technology PIC24FJ64GA002 Microcontroller Microchip Technology, Inc. Motorola MPXAZ6115 pressure sensors Freescale Semiconductor, Inc. SILRX Receiver and TXM 43-MHz transmitter Radiometrix PQ1X331M and PQ1X451M regulators Sharp Microelectronics CIRCUIT CELLAR 25 View publication stats