Hands-On Lab Multi-Touch WMTouch - Native Lab version: 1.0.0 Last updated: 12/3/2010
CONTENTS OVERVIEW... 3 EXERCISE 1: BUILD A MULTI-TOUCH APPLICATION... 5 Task 1 Create the Win32 Application... 5 Task 2 Test the Existence and Readiness of Multi-touch Hardware... 6 Task 3 Add the Stroke Source and Header Files to the Project, and Draw Lines with your Fingers.. 7 SUMMARY... 14 2
Overview Windows 7 gives users the ability to manage applications with the touch of their fingers, using no intermediate device. This expands the stylus-based capabilities of tablet PCs. Unlike other pointing devices, this new capability allows multiple input events at the same time from different pointing locations, and it enables complex scenarios, such as managing applications with ten fingers or with multiple simultaneous users.however, to pull this off, we have to adapt our application's user interface and behavior to support this new input model. Objectives In this Hands-On Lab, you will learn how to manage gesture events, including: Understanding the implications of manipulating multiple touch events simultaneously Checking for multi-touch hardware existence and readiness Extracting information from the WM_TOUCH Windows Message System Requirements You must have the following items to complete this lab: Microsoft Visual Studio 2008 SP1 Windows 7 The Windows 7 SDK A multi-touch hardware device Introduction To have a Multi-Touch driven application you can choose one of three approaches, the Good, The Better and the Best. The Good approach is the easiest between the three. You should design your application user interface with touch ability in mind. Use large and clean Win32 based controls that make a natural interface for better user experience. Touch abilities such as scrolling come from the Win32 controls. There is no need for extra work. For example, try to scroll the document that you are reading now with your fingers! This is the Good approach. The "Better" approach lets the system receives the various low level touch events and get the result of the heuristics that the system does with these events. For 3
example the user made a rotation movement on the screen; the system will issue a rotation gesture event with the rotation angle. Although the "Better" approach is easy to use, it has its limitations. Using gesture one cannot get Rotate, Translate and Scale simultaneously. Also you cannot handle many different touch based actions in the same time. For example two users that operate different areas of the Window. The Best approach is to read the low level touch events as the input to the application. Applications like Piano or complex controls like multiple-sliders that can be operate simultaneously are good examples. Run MS Paint, select a drawing tool from the gallery and draw with five of your fingers (if the hardware permits) In this Hands-On Lab you we will mimic the new MS Paint multi-touch painting feature. We will use the Best approach. We will read and use the raw touch events. Enter the Low-level WM_TOUCH Multitouch message decoding. About the Multi-touch Scratchpad Application The Multi-touch Scratchpad application presents a simple window that allows simultaneously drawing of continues lines with your fingers. In the HOL folder you can find a project for each task of the lab. The Starter folder contains files that you will need during the lab. A finished version of the lab is located in the Final folder. 4
Exercise 1: Build a Multi-touch Application Task 1 Create the Win32 Application 1. Start Visual Studio 2008 SP1 2. Select new based Win32 application project: 5
3. Compile and run! 4. We are going to use APIs and Macros that belong to Windows 7, change the WINVER and _WIN32_WINNT definitions in the targetver.h header file to 0x0601 #ifndef WINVER //Specifies that the minimum required platform is Windows 7 #define WINVER 0x0601 #endif #ifndef _WIN32_WINNT //Specifies that the minimum required platform is Win 7 #define _WIN32_WINNT 0x0601 #endif 5. Compile and run! Task 2 Test the Existence and Readiness of Multi-touch Hardware 6
1. The application that we are building requires touch-enable computer, add the following code, before the call to InitInstance() in the _twinmain(), to check the hardware touch ability and readiness: BYTE digitizerstatus = (BYTE)GetSystemMetrics(SM_DIGITIZER); if ((digitizerstatus & (0x80 + 0x40)) == 0) //Stack Ready + MultiTouch MessageBox(0, L"No touch support is currently availible", L"Error", MB_OK); return 1; BYTE ninputs = (BYTE)GetSystemMetrics(SM_MAXIMUMTOUCHES); wsprintf(sztitle, L"%s - %d touch inputs", sztitle, ninputs); 2. You can see that besides checking for touch availability and readiness we also find out the number of touch inputs that the hardware support. 3. Compile and run! Task 3 Add the Stroke Source and Header Files to the Project, and Draw Lines with your Fingers We would like to use our fingers as a multiple mouse device. We want to draw a line with each of our fingers that touches the screen. To do that we are going to use two stroke collections. One collection holds the finished strokes (lines) and another collection holds the on-going currently painting lines. Each finger that touches the screen adds points to a stroke in the g_strkcoldrawing collection. When we 7
raise the finger from the screen, we move the finger's stroke from the g_strkcoldrawing to the g_strkcolfinished collection. At WM_PAINT we draw both collections. 1. In the Starter folder you will find two files: Stroke.h and Stroke.cpp. Copy them to the project folder and use Add Existing item to add them to the project. 2. Add an #include "Stroke.h" line at the top of MTScratchpadWMTouch.cpp file #include "Stroke.h" 3. Add global variables definition at the //Global Variables: section at the top of mtgesture.cpp file: CStrokeCollection g_strkcolfinished; // The user finished entering strokes. // The user lifted his or her finger. CStrokeCollection g_strkcoldrawing; // The Strokes collection the user is // currently drawing. 4. Add the following lines in the WndProc(), note that WM_PAINT has been already created by the application wizard: case WM_PAINT: hdc = BeginPaint(hWnd, &ps); // Full redraw: draw complete collection of finished strokes and // also all the strokes that are currently in drawing. g_strkcolfinished.draw(hdc); g_strkcoldrawing.draw(hdc); EndPaint(hWnd, &ps); break; 5. Now it's time to enable WM_TOUCH messages. By default a Window receives WM_GESTURE messages. To switch to the low-level WM_TOUCH messages we need to call to the RegisterTouchWindow() API. Add the following code to the InitInstance() function just before the call to ShowWindow(): // Register the application window for receiving multi-touch input. if (!RegisterTouchWindow(hWnd, 0)) 8
MessageBox(hWnd, L"Cannot register application window for touch input", L"Error", MB_OK); return FALSE; 6. We asked Windows to send WM_TOUCH messages. WM_TOUCH message is a special message. Unless you asked the system not to gather multiple touch events in one message (see the TWF_FINETOUCH parameter) you get all of your touch points in one message. This is reasonable since the user touches the screen with many touch points simultaneously. Add the following lines to the WndProc() function: case WM_TOUCH: // A WM_TOUCH message can contain several messages from different contacts // packed together. unsigned int numinputs = (int) wparam; //Number of actual contact messages TOUCHINPUT* ti = new TOUCHINPUT[numInputs]; // Allocate the storage for //the parameters of the per- //contact messages // Unpack message parameters into the array of TOUCHINPUT structures, each // representing a message for one single contact. if (GetTouchInputInfo((HTOUCHINPUT)lParam, numinputs, ti, sizeof(touchinput))) // For each contact, dispatch the message to the appropriate message // handler. for(unsigned int i=0; i<numinputs; ++i) if (ti[i].dwflags & TOUCHEVENTF_DOWN) OnTouchDownHandler(hWnd, ti[i]); else if (ti[i].dwflags & TOUCHEVENTF_MOVE) OnTouchMoveHandler(hWnd, ti[i]); else if (ti[i].dwflags & TOUCHEVENTF_UP) OnTouchUpHandler(hWnd, ti[i]); 9
CloseTouchInputHandle((HTOUCHINPUT)lParam); delete [] ti; break; 7. The wparam holds the number of touch input that came with the WM_TOUCH message. The GetTouchInputInfo() API fills a TOUCHINPUT array with touch information for each touch point. After you finish to extract data from the TOUCHINPUT array, you need to call the CloseTouchInputHandle() to free system resources. Here is the definition of the TOUCHINPUT structure: (Extracted from WinUser.h) typedef struct tagtouchinput LONG x; LONG y; HANDLE hsource; DWORD dwid; DWORD dwflags; DWORD dwmask; DWORD dwtime; ULONG_PTR dwextrainfo; DWORD cxcontact; DWORD cycontact; TOUCHINPUT, *PTOUCHINPUT; typedef TOUCHINPUT const * PCTOUCHINPUT; /* * Conversion of touch input coordinates to pixels */ #define TOUCH_COORD_TO_PIXEL(l) ((l) / 100) /* * Touch input flag values (TOUCHINPUT.dwFlags) */ #define TOUCHEVENTF_MOVE 0x0001 #define TOUCHEVENTF_DOWN 0x0002 #define TOUCHEVENTF_UP 0x0004 Four parameters from the TOUCHINPUT structure are in our interest: The x and y are the touch location in screen coordination multiply by a hundred. This means that we need to divide each axis value by hundred (or use the TOUCH_COORD_TO_PIXEL() macro) and call ScreenToClient() to move to the Window coordination system. Be aware that if the screen is 10
set to High DPI (more than 96 DPI), you may also need to divide the values by 96 and multiply by the current DPI. For simplicity we skip this step in our application. The other two parameters are the dwid and dwflags. The dwflags tells us the type of the touch input: Down, Move or Up. In our application TOUCHEVENTF_DOWN starts new stroke, TOUCHEVENTF_MOVE adds another point to an existing stroke and TOUCHEVENTF_UP finishes a stroke and move it to the g_strkcolfinished collection. The last parameter is the dwid, this is the touch input identifier. When a finger touches the screen for the first time, a unique touch id is associated with the finger. All farther touch inputs that come from this finger get the same unique id until the last TOUCHEVENTF_UP input. When the finger leaves the screen the id is freed and may be reuse as a unique id for other finger that will touch the screen. It's time to handle the touch inputs, add the following functions before the WndProc() function: // Returns color for the newly started stroke. // in: // bprimarycontact flag, whether the contact is the primary contact // returns: // COLORREF, color of the stroke COLORREF GetTouchColor(bool bprimarycontact) static int s_icurrcolor = 0; // Rotating secondary color index static COLORREF s_arrcolor[] = // Secondary colors array RGB(255, 0, 0), // Red RGB(0, 255, 0), // Green RGB(0, 0, 255), // Blue RGB(0, 255, 255), // Cyan RGB(255, 0, 255), // Magenta RGB(255, 255, 0) // Yellow ; COLORREF color; if (bprimarycontact) // The application renders the primary contact in black. color = RGB(0,0,0); // Black else // Take the current secondary color. color = s_arrcolor[s_icurrcolor]; // Move to the next color in the array. s_icurrcolor = (s_icurrcolor + 1) % (sizeof(s_arrcolor)/sizeof(s_arrcolor[0])); 11
return color; // Extracts contact point in client area coordinates (pixels) from a // TOUCHINPUT structure. // in: // hwnd window handle // ti TOUCHINPUT structure (info about contact) // returns: // POINT with contact coordinates POINT GetTouchPoint(HWND hwnd, const TOUCHINPUT& ti) POINT pt; pt.x = TOUCH_COORD_TO_PIXEL(ti.x); pt.y = TOUCH_COORD_TO_PIXEL(ti.y); ScreenToClient(hWnd, &pt); return pt; // Handler for touch-down input. // in: // hwnd window handle // ti TOUCHINPUT structure (info about contact) void OnTouchDownHandler(HWND hwnd, const TOUCHINPUT& ti) // Create a new stroke, add a point, and assign a color to it. CStroke strknew; POINT p = GetTouchPoint(hWnd, ti); strknew.addpoint(p); strknew.setcolor(gettouchcolor((ti.dwflags & TOUCHEVENTF_PRIMARY)!= 0)); strknew.setid(ti.dwid); // Add the new stroke to the collection of strokes being drawn. g_strkcoldrawing.addstroke(strknew); // Handler for touch-move input. // in: // hwnd window handle // ti TOUCHINPUT structure (info about contact) void OnTouchMoveHandler(HWND hwnd, const TOUCHINPUT& ti) // Find the stroke in the collection of the strokes being drawn. int istrk = g_strkcoldrawing.findstrokebyid(ti.dwid); POINT p = GetTouchPoint(hWnd, ti); 12
// Add the contact point to the stroke. g_strkcoldrawing[istrk].addpoint(p); // Partial redraw: redraw only the last line segment. HDC hdc = GetDC(hWnd); g_strkcoldrawing[istrk].drawlast(hdc); ReleaseDC(hWnd, hdc); // Handler for touch-up message. // in: // hwnd window handle // ti TOUCHINPUT structure (info about contact) void OnTouchUpHandler(HWND hwnd, const TOUCHINPUT& ti) // Find the stroke in the collection of the strokes being drawn. int istrk = g_strkcoldrawing.findstrokebyid(ti.dwid); // Add the finished stroke to the collection of finished strokes. g_strkcolfinished.addstroke(g_strkcoldrawing[istrk]); // Remove finished stroke from the collection of strokes being drawn. g_strkcoldrawing.removestroke(istrk); // Redraw the window. InvalidateRect(hWnd, NULL, FALSE); 8. To make the drawing a little bit more interesting we pick a different color for each unique id. The primary touch is the first finger that touched the screen. It is special inout since it act as the mouse pointer. We chose to give it a black color. Compile the application and run! You can touch it now! 13
Summary In this lab, you have learned how to consume the low level WM_TOUCH messages. You ve seen how to test for the existence of multi-touch hardware. How to configure a Window to get the WM_TOUCH message, how to extract the inputs from the message and how the system correlate touch id to a touch input. Enjoy and keep in touch! 14