Designing Non-Rectangular Skin-able GUIs.

Transcription

1 Designing Non-Rectangular Skin-able GUIs. M.Tsegaye Department of Computer Science Rhodes University Grahamstown, 6140, South Africa Phone: +27 (46) Fax: +27 (46) Abstract The majority of GUI applications in use today on popular desktops use a simple rectangular window with components that paint themselves using unattractive solid colours. These user interfaces are not always suitable for specialized applications that are used for representing real world objects such as digital cameras and portable media players. This paper presents the process involved in creating non rectangular skin-able user interfaces that use realistic textures to paint themselves. Index Terms skin-able application, layered windows, window skinning, non rectangular windows, user interface design 1.0 Introduction Traditional applications on most GUI systems use rectangular windows as the primary user interface regardless of the type of application the user interface was designed for. Most of these applications try to represent real world three dimensional objects such as audio layers, digital cameras and telephones with rectangular windows that have menu bars, scrollbars and other common 2D GUI controls. This type of interface, although simple to use, is not always intuitive. It requires a long time to get used to. The use of GUIs that utilise mostly rectangular but skin-able windows has increased with the introduction of powerful PC hardware such as the GPUs from ATI and NVIDIA. The popular Microsoft Windows platform now has skin-able user interface components (introduced in Windows XP). Microsoft [1] claims that this new more attractive interface increases user productivity. Apple s Aqua interface [2], introduced much earlier than Windows XP s, uses a similar approach. It has GUI components that are very attractive and gentle on the eye. Wired news [3] gives an account of the skin-able user interface revolution. It lists Winamp, a popular skin-able audio player, as the application that started the skinning phenomenon. Most UNIX window managers such as Blackbox and Enlightenment have always offered skin-ability to users. Commercial products that allow the creation of non rectangular and rectangular skin-able user interfaces for the Microsoft Windows platform exist. One such product is WindowBlinds [5]. Window managers such as KDE on UNIX also make available an API usable under an open source license. This paper presents an approach that can be used for creating non-rectangular and rectangular skin-able user interfaces in much the same way as they are implemented in commercial systems. The following steps are followed: Selection of an appropriate skin script format to allow the definition of UI components. Creation of a library for dynamically loading the above skin script at runtime. Design and creation of UI controls which are aware of being skinned. Design and creation of skins for these controls. The results presented use the Microsoft Windows API but the ideas can be applied on other platforms using a platform specific API. Appendix A presents a summary of the Microsoft Windows API relevant to skin-able user interface development. 2.0 Selecting a Skin Definition Script Format A skin definition script enables users to define the UI for an application at runtime instead of at compile time as done in traditional applications. Any time during the life of the application the script file can be loaded by the application, parsed, interpreted and the user interface updated as defined by script. 2.1 File Formats Various file formats can be used for the script e.g. a simple text file with an application specific layout or a binary format. A data definition format such as XML would be the most logical format to use because it is an industry standard. Numerous libraries exist for parsing it and any XML editing application can be used for editing the script. XML also shows the hierarchical relationship of components clearly. UIs usually have a hierarchical relationship e.g. most applications have a parent frame with child components that can be frames themselves, and those child frames can have children. 2.2 UI Definition Format The script needs to define every component of the user interface. Since XML is selected as the file format for the script, the skin definition format can be as verbose as necessary to make it easier for human editors to understand without making it any more difficult to parse. Here is how a simple skin definition with no components would look like: <?xml version = 1.0?> <My_Interface> </My_Interface>

2 2.2.1 Texture Entries Skinning makes extensive use of textures. These textures can use up a lot of memory if not managed well. The script entry for defining textures should act as a pointer to an area where textures are located. Here is the texture definition entry that has been selected: <?xml version = 1.0?> <My_Interface> <bitmap name= skin.png id= 0 transparency= #ff00ff /> </My_Interface> The texture entry, named bitmap, defines the location of a texture resource skin.png, an id for it and a colour on the texture to be considered as transparent, The PNG file format supports transparency natively so the application can extract the value from the texture which makes the transparency entry redundant. It is specified here for clarity. In cases where texture formats such as the BMP format, which does not handle transparency well, are used it is mandatory. Multiple entries for textures, each with different ids, can be specified Frame entries The next set of entries in the skin definition script should be those that define the various controls on the UI. Just about every UI has a frame. The following frame definition is used: <?xml version = 1.0?> <My_Interface> <bitmap name= skin.png id= 0 transparency= #ff00ff /> <bitmap name= skin2.png id= 1 transparency= #ff00ff /> <window name= test1 main= yes > <settings> <background left= 10 top= 10 width= 450 height= 150 bitmapid= 0 /> </settings> <window name= child1 child= yes > <settings> <background left- 10 top= 100 width= 250 height= 350 bitmapid= 1 /> </settings> </window> </window> <window name= test2 > <settings> <background left= 10 top= 10 width= 450 height= 150 /> <color name= background value= #ff0000 /> </settings> </window> </My_Interface> The XML element window represents a frame. The script fragment above shows two top level windows and one child window. The first is designated as the main window. The texture region that will be used to paint this frame s background is defined by the background entry, which has a pointer to a texture, bitmap entry, whose id is 0. Although the background defined is rectangular, the parts of the texture in this region that have the transparency colour will be treated as such. The second top level window named test2 does not use a texture for its background. It is rectangular and has a red background. In this way the script is flexible enough to define frames of any shape. A child frame would be nested within another frame as shown for the window element named child1. This child is not a top level child window (popup window), because it has an entry that says child= yes Additional Controls Script entries for other controls can be defined depending on the look of the particular control. Here is what an entry for a button would look like: <button name="button1"> <skin name="normal" left="105" top="59" width="19" height="29" bitmapid="0"/> <skin name="hover" left="104" top="99" width="20" height="29" bitmapid="0"/> <skin name="mousedown" left="104"top="139" width="20" height="28" bitmapid="0"/> <skin name="disabled" left="70" top="41" width="19" height="29" bitmapid="0"/> </button> The button script entry defines texture regions to be used for painting the button during its various states with pointers to appropriate textures. Many more controls such as sliders, list boxes, tree controls and animations can be defined in this way. An entry such as this can be embedded in a frame script entry where that control will be contained. 3.0 Creating a Skin Script Loading Library The skin loading library should be able to parse the script file and create the various controls defined within it. It should also manage texture resources. 3.1 The Skin Loader In an object oriented development environment the skin loader can be represented as an object. A new object can be instantiated for each different skin definition script. In this way the process of loading/reloading a skin at runtime consists of deleting a skin loader object if it already exists and creating a new instance for a newly specified script. This approach separates the user interface from the main application. The various controls can be assigned names that are understood by the application which interacts with the skin loader object to access handles to them. When the controls generate events, these can be routed to the application by the skin loader object. The skin loader should scan a specified location for skin definition scripts and keep a list of them. It should also present some sort of interface to allow users to switch skins, The skin switching interface is necessary only for applications that utilise multiple skins suited to different user tastes. Another function of the skin loader is to keep a list of textures that are defined by the currently loaded skin definition script. It should make appropriate use of available texture loading APIs.

3 Proper parsing of skin files and error recovery is essential. Errors can occur in many places. The first is in the syntactic structure of the XML script. This is usually handled by the XML parser. Errors should also be detected by the application as values are extracted from the various XML elements. Appropriate feedback should be given to users during the process of loading a skin definition script. 4.0 Designing Skin-able UI Controls This step involves creating controls from scratch or extending the functionality of existing non skin-able GUI components to be skin-able. If a control is represented as a class in an object oriented environment then there needs to be some common ancestor control class from which other controls can be derived. In Microsoft Windows this could be the CWnd MFC class and in KDE it could be the QWidget class. Existing controls such as frames, buttons and text displays in these systems are already designed well and can be extend to be skin-able. 4.1 Painting Controls The paint methods from existing GUI classes are overridden to make them paint themselves using textures, if this functionality is not already present in that class. Rectangular texture coordinates and texture ids obtained from the skin loader are used in the painting process. A transparent blting operation is performed to copy only the opaque regions of the texture to the control s display area. Thus even if the control is rectangular users perceive it as non rectangular since only the opaque areas are seen by them. textures, selection of texture regions and editing of the various parts of the skin definition script. 5.0 Multiple Skin Switcher Demo The test skin switching application shown on Figure 5.0 demonstrates the skin loading library constructed following the steps outlined in the preceding sections. It shows the ease with which the application can change from one form to another. Figure 5.1 shows a skin script editing application that makes the editing of skin scripts easy. 6.0 Conclusion Real world three dimensional objects such as audio players and digital cameras need not be represented using rectangular windows that use solid colours to paint themselves. They can instead use realistic textures for skinning themselves and present an intuitive user interface to users. Applications that have a skin-able, non-rectangular or rectangular user interface can be created by following these steps: Selecting an appropriate skin definition format. Creating a library for dynamically loading skin definition scripts. Creating controls that understand the entries in the scripts and skin themselves. Creating textures and a skin definition scripts to describe the required user interfaces. Figure 5.0 Multiple Skin Changer Demo Application Double buffering should be used for controls that paint themselves very often, to avoid the undesirable effect of screen flicker. 4.2 Skin Awareness by Controls Controls are represented as objects therefore each object should know how to skin itself given an XML element that represents itself. At script loading time the skin loader parses the XML file and creates controls as they are found in the script definition file. It also passes a control s XML element definition to that control. If the XML element passed to a control contains another element representing a control then the former control becomes the parent and the above process is repeated for the latter control. Figure 5.1 Skin Script Editing Environment 4.3 Creating Skins for Controls Once the skin loader engine and the various controls have been created then user interfaces for just about any application can be created. This process requires the use of a software package capable of creating textures for use as skins. GIMP [9] is a powerful open source paint package that can be used for this process. Once a drawing for an interface and its various components has been made by an artist, it can be inspected by interested parties for approval. A skin definition script can then be created. To make the process of creating skin definition scripts easier than having to type it out by hand, it is advisable to construct a development environment that allows loading of

4 Appendix A 1.0 Introduction This appendix presents some of the facilities available on the Win32 platform that enable the development of skinable UI components. 2.0 Painting and Drawing using the Win32 GDI Microsoft Windows is a message based system. Whenever a window owned by an application needs painting due to some event e.g. a user resizing the window or the window being covered by another, it receives messages that tell it to redraw affected areas. 2.1 Device Contexts Traditionally if an application wanted to produce pretty graphics on the screen, it would do so by accessing a graphic device s display memory and modifying it. This would cause changes to be observed on the screen. Window s is a multitasking environment with multiple applications. For the sake of efficient use of the graphic device, the system manages all drawing [7]. Every window has associated with it a GDC (Graphic Device Context). All drawing is directed to this GDC. HDC hdc = GetDC(hWnd); SetPixel(hdc, 10,10,RGB(255,0,0)); ReleaseDC(hWnd,hdc); Figure 2.1: Setting pixel (10, 10) to red on a window referenced by handle hwnd. When a window is created the system will return a handle, hwnd on Figure 2.1, for that window to the application. The application uses this handle to make drawings on the window. For example to change a pixel s colour in a window requires getting a handle to the window s device context, hdc on Figure 2.1, and then drawing to that device context as shown Off Screen Device Contexts Off screen device contexts are device contexts that are represented in memory. These allow drawings to be made in memory rather than on screen as is the case for device contexts associated with windows. There are two main benefits in using off screen DCs. The first is that an application can draw complex drawings only once into a memory DC and whenever it wants to redraw parts of itself it can simply copy the pre-drawn image from an off screen DC to its display DC, speeding up the drawing process. The second is it that it eliminates flicker while drawing which is common when modifying contents of a screen directly Creating off screen Device Contexts An off screen DC is an array of bits. The bitmap it stores must be of the same format as the screen DC if the bitmap is to be displayed correctly on the screen when the off screen DC is copied to the screen DC. An off screen DC is created as follows: HDC screendc = GetDC(hWnd); HDC offscreendc = CreateCompatableDC(screenDC); HBITMAP hbitmap = CreateComaptableBitmap( HBITMAP holdbitmap = SelectObject(offscreenDC,hbitmap); ReleaseDC(hWnd,screenDC); screendc,width,height); The first step is to create an off screen DC compatible to the screen DC then a bitmap of the desired height and width. For the bitmap to be useful it must be selected into the new off screen DC. Once selected Win32 places the restriction that it may not be reselected into any other device context. When a new device is selected into a device context with a SelectObject call the old bitmap previously associated with the dc is returned. The old bitmap is a one by one monochrome bitmap that gets associated to a device context at the time of the CreateCompatableDC call Deleting off screen Device Contexts Device contexts use memory. When the off screen DC is no longer required it and the bitmap selected into it must be freed as shown on Figure The original monochrome bitmap must be re selected into the off screen DC before the DC is deleted. hbitmap = DeleteObject(hbitmap); DeleteDC(offscreenDC); SelectObject(offscreenDC,holdbitmap); Figure Deleting an off screen device context 2.2 Bit to Bit transfer of blocks of bitmaps The contents of a DC can be transferred from one DC to another by using one of the bit block transfer functions rather than doing it manually by using a for loop and using GetPixel and SetPixel which would be a very slow operation. The BitBlt, StretchBlt and TransparentBlt functions implement this functionality in hardware where available. TransparentBlt is especially useful for implementing non rectangular controls in a window. 2.3 Loading images that use the Windows BMP format Applications can use the Win32 LoadImage function for loading bitmaps from a file on disk as follows: HBITMAP bitmap = LoadImage(hInstance,filename, IMAGR_BITMAP, width,height,lr_fromfile); It returns a handle to a bitmap which can be selected into a device context. Once this is done the bitmap can be manipulated as desired. The latest version of the platform SDK contains a new GDI interface, GDIPlus, which allows the loading of different image file formats such as JPG and PNG.

5 3.0 Non Rectangular Windows In win32 there are two methods for creating non-rectangular windows. The first is by using window regions and the second is a more recent addition to the win32 API, using layered windows. 3.1 Window Regions Regions are areas of a window that are defined by an application. There are three types rectangular, elliptical and polygonal. Rectangular regions can be defined as ordinary rectangles or those with rounded edges. Polygonal regions are the most interesting as they allow windows to have any polygonal shape Using Regions A number of region creation functions are provided by the win32 API some of which are listed on Figure All of these return a region handle which can be used for manipulating the region e.g. hit testing to see if a user clicked inside a region or not. HRGN hregion = CreateRectRgn(left,top,right,bottom); Creates a rectangular region hregion = CreateRoundRectRgn(left,top,right,bottom, xedge,yedge); Creates a rectangular region with rounded corners hregion = CreateEllipticRgn(left,top,right,bottom); Creates an elliptical region hregion = CreatePolygonRgn(points,pointCount,fillmode); Creates a polygonal region Figure Some of the win32 region creation functions A large number of operations can be performed on regions such as combining regions to form more complex ones and filling regions with a colour. When no longer required a region must be freed. This is done using the DeleteObject function call which is used to release acquired GDI objects. 3.2 Layered Windows The layered window functionality was introduced into win32 with the release of windows NT5. It introduces two new ways for painting windows. In addition to allowing transparency the API allows windows to have translucency. For a window to make use of layered window functionality it must add the WS_EX_LAYERED style to its extended window styles. This can be done in two ways: at window creation time or at run time as shown on Figure 3.2a. At window creation time hwnd = CreateWindowEx( WS_EX_LAYERED, clasname,windowname, WS_POPUP WS_VISIBLE, left,top, width, height, NULL,NULL, hinstance, NULL); At runtime SetWindowLong(hwnd, GWL_EXSTYLE, GetWindowLong(hwnd, GWL_EXSTYLE) WS_EX_LAYERED); Figure 3.2a Adding the WS_EX_LAYERED style to a window s extended styles Once a window has the WS_EX_LAYERED style added to its extended styles the applications can decide to use the layered window API in two ways. The first will let the application perform all drawing in its window by itself. The system will disable sending of all window paint messages to it. The second method will let the application paint itself in response to the win32 paint messages in the normal way. The first method is more efficient if an application modifies its entire window contents frequently. Figure 3.2b illustrates the two methods. Method 1: Disables normal paint messages and the application is responsible for all drawing at appropriate times. BLENDFUNCTION blend; blend.blendop = AC_SRC_OVER; blend.blendflags = 0; blend.alphaformat = 0; blend.sourceconstantalpha = balpha; UpdateLayeredWindow(hwnd, hscreendc, newscreenpos, newscreensize, offscreendc, imageposonscreendc, colorkey,&blend,ulw_alpha ULW_COLORKEY); Method 2: The Application paints its windows in response to the normal system paint messages. SetLayeredWindowAttributes(hWnd, colorkey, balpha, LWA_COLORKEY LWA_ALPHA); Figure 3.2b the two ways for using the layered window API. The source constant alpha value ranges from 0 for 100% transparent to 255 for opaque. An application, for example, can use this value to implement translucent menus of varying translucency. The colour key value is the value used to implement transparency. All bits in the window with that colour value will be treated as transparent by the system. This enables windows to assume any shape they like.

6 3.3 Which to use, Layered Windows or Regions? The layered window API is the preferred way of implementing applications that make use of non-rectangular windows. It is much more efficient than the region API which has no support for translucency at all. With regions complex shapes have to be defined by supplying individual points. Using layered windows is much simpler, all that is required is a bitmap with transparent areas set using a colour key value usually RGB(255,0,255). To make use of layered windows a more recent version of the platform SDK is required. The region API works on older windows 9x and NT systems and as well as the newer NT5. Regions can be used in both parent and child windows but the layered window API supports only parent windows. This is not a problem since non rectangular child windows can be simulated using the TransparentBlt function. References [1] Microsoft Windows XP Technical Overview, [2] Apple Aqua, [3] Wired GUIs Just Want to Have Fun, [4] BetaNews Skinning the Windows GUI, [5] StarDock WindowBlinds, [6] Microsoft Microsoft User Interface Design and Development, [7] Microsoft Windows GDI, [8] Microsoft Layered Windows, [9] GIMP Gnu Image Manipulation Program,