Xwindows Windows Drawing Events Advanced Topics Xwindows: Windows Window Models Creating Windows in X X11 Architecture Before Windowing Systems 3
What's Under the Hood of a GUI Toolkit? Up until now, we ve been assuming: - Low-level events get to us, somehow (e.g., information about the keyboard, mouse, the need to paint, etc.) - We can create a window, somehow - We can paint on that window, somehow What is this mechanism that allows us to get this information, and to create and paint windows? And just as importantly, what is it that this mechanism is doing? That is, what is it abstracting away for our convenience? 4 Images: LG monitor CC BY-SA 2.0 by florisla; HP s rotatable display by HP; Vaio CC by Yoggy from Yokohama, Kanagawa, Japan; X61 public domain by Evan-Amos; mouse CC 5 BY-SA 3.0 by Darkone; TrackPoint CC by Inklein; keyboard public domain by the US Navy Abstracting Away the Hardware Our output displays can vary in dimensions, pixel density, number of colors that can be displayed We can get pointer input from a wide range of devices - Mouse - Stylus - Touch input We can get character input in a wide range of ways - Hardware keyboards - Soft keyboards As application developers, we d like to deal with all these different pieces of hardware as if they re the same thing 7
Base Window System The Base Window System (BWS) provides this layer of abstraction between the application programmer and the lowlevel hardware: - Display device(s) - Graphics cards - Pointing devices - Keyboard input 8 9 Base Window System Lowest level abstraction for windowing system Routines for creating, destroying, managing, and painting in windows Routes input to correct window Manages painting, so windows not painting in other windows area Ensures only one application changing frame buffer (video memory) at a time - one reason why single-threaded / non-thread-safe GUI architectures are popular 10
Base Window System Creates canvas abstraction for applications - Applications shielded from details of frame buffer, visibility of window, other application windows Provides basic graphics routines for drawing Each window has its own coordinate system - Each window always assumes its top-left is (0,0) - Simplifies painting/drawing - BWS transforms between coordinate systems 11 X (X11) Window System A base window system first developed in 1984 A standard for low-level graphical output and user input - A protocol to create windows, handle input, draw graphics, - Not a window manager (more on that later) - Does not specify style of user interface Free and cross-platform (os-, processor-, form factor-agnostic) Separate from operating system 12 Window Manager Layered on top of Base Window System Creates the look and feel of each window - Appearance (e.g., chrome ) and interaction for common window operations Provides interactive components for windows (menus, close box, resize capabilities) Manages things like window behavior, such as where windows appear when first created 14
Motif (Stacking) 15 DWM (tiling) 16 KDE (simple compositing) Screenshot 17 by CS Nev 349 Delap, - X Windows CC BY-SA 3.0
KWin (compositing) 18 Metisse (compositing) 19 Chapuis and Roussel, UIST 2003. http://wiki.mandriva.com/en/projects/metisse Window Manager Application Window vs. Application Canvas - the window manager owns the window (including its controls) - the application owns the canvas window controls 20
BWS vs. Window Managers X separates Base Window System from Window Manager - Enables many alternative look and feels for windowing system (e.g., KDE, GNOME, fvwm ) - One of the keys to its lasting power: Can continue to grow by changing the Window Manager layer BWS and Window Manager are separate processes 21 BWS vs. Window Managers OSX, Microsoft Windows combine BWS and Window Manager Trade-offs in approaches? - Look and feel - Window management possibilities - Input possibilities 22 More on X11 23
X Windows Design Criteria (~1986) 1. implementable on a variety of displays 2. applications must be device independent 3. must be network transparent 4. support multiple, concurrent application displays 5. support many different applications 6. support output to overlapping windows ( even when partially obscured) 7. support a hierarchy of resizable windows ( an application can use many windows at once) 8. high-performance, high-quality text, 2-D graphics, imaging 9. system should be extensible 24 (from Scheifler & Gettys, 1986) X Client-Server Architecture Separate user interface and application: - the X Client handles all application logic - the X Server handles all display output and user input A server handles requests from multiple clients, processes data as requested, and returns the results to the clients X inverts conventional www server and client relationship (in www, web browser is the client, web site is the server ) 25 Why Client-Server? Goal was flexibility and economy - Many clients (perhaps on multiple machines), one display 26
Displays, Screens, Windows In X, a display may have multiple screens A display may have multiple windows A window may cross multiple screens 27 Using X Across the Network hostname:displaynumber.screennumber e.g. :0, :0.1, machine.cs.uwaterloo.ca:0.0, DISPLAY environment variable should be set to local display - printenv DISPLAY ssh X - X11 forwarding: when you run an X Client on a server, it uses your local machine as the X Server! 28 Xlib library to wrap low level X Window protocol - to avoid implementing message passing for every new program uses buffered input and output queues - need to flush them: XSync, XFlush Xlib functions: - connection operations: e.g. XOpenDisplay, XCloseDisplay, - connection operation requests: e.g. XCreateWindow, XCreateGC, - connection information requests: e.g. XGetWindowProperty, - local event queue operations: e.g. XNextEvent, XPeekEvent, - local data operations: e.g. XLookupKeysym, XParseGeometry, XSetRegion, XCreateImage, XSaveContext, Xlib data types: - e.g. Display, Window, GC, XSizeHints, XWhitePixel, XBlackPixel, etc. Xlib is not a window manager Xlib does not specify style of user interface or provide widgets 29
Structure of a Typical X Program 1. perform client initialization 2. connect to the X server 3. perform X related initialization 4. event loop: get next event from the X server handle the event: if the event was a quit message, exit the loop do any client-initiated work send drawing requests to the X server 5. close down the connection to the X server 6. perform client cleanup 30 null.min.cpp: creates and destroys a display #include <cstdlib> #include <iostream> #include <X11/Xlib.h> // main Xlib header Display* display; int main() { display = XOpenDisplay(""); // open using DISPLAY env var if (display == NULL) { std::cout << "error\n"; exit ( 1); else { std::cout << "success!\n"; XCloseDisplay(display); // close display 31 Makefiles You need makefiles for all assignments - (you can re-use the one in the code demos by setting NAME) - http://mrbook.org/tutorials/make/ # super simple makefile # call it using 'make NAME=name_of_code_file_without_extension' # (assumes a.cpp extension) NAME = "null.min" all: @echo "Compiling..." g++ o $(NAME) $(NAME).cpp L/usr/X11R6/lib lx11 lstdc++ run: all @echo "Running..."./$(NAME) 32
openwindow.min.cpp: Display a Window Display* display; Window window; // save the window id int main( int argc, char* argv[] ) { display = XOpenDisplay(""); // open display if (!display) exit ( 1); // couldn't open, so bail int screen = XDefaultScreen(display);// info about the display window = XCreateSimpleWindow( display, XDefaultRootWindow(display), // window's parent 10, 10, // location: x,y 400, 300, // size: width, height 2, // width of border XBlackPixel(display, screen), // foreground colour XWhitePixel(display, screen)); // background colour XMapRaised(display, window); // put window on screen XFlush(display); // flush the output buffer std::cout << "ENTER2exit"; std::cin.get(); // wait for input XCloseDisplay(display); 33 Code Review: openwindow.cpp Same Functions/Macros and procedures as min verson: - XOpenDisplay - XDefaultScreen - XWhitePixel, XBlackPixel - XCreateSimpleWindow - XSetStandardProperties - XMapRaised - XFlush Difference is cleaner coding practice, but longer code 34 X Windows: Drawing Drawing Models Drawing in X
Drawing X Windows manages multiple windows - where window is located, is it covered by another window, etc... - enables drawing using local coordinate system for window 37 Drawing Models Three different conceptual drawing models: Pixel SetPixel(x, y, colour) DrawImage(x, y, w, h, img) Stroke DrawLine(x1, y1, x2, y2, colour) DrawRect(x, y, w, h, colour) Region DrawLine(x1, y1, x2, y2, color, thick) DrawRect(x, y, w, h, colour, thick, fill) DrawText( A, x, y, colour) 38 Drawing Options for Stroke and Region Models e.g. drawline(x1,y1,x2,y2) - what colour? - how thick? - dashed or solid? - where are the end points and how should the ends overlap? - How to communicate all the options? Observation: most choices are the same for multiple calls to drawline 39
Graphics Context Gather all drawing options into a single structure and pass it to the drawing routines - In X, the GC structure All graphics environments use variation on this approach - Java: Graphics Object - OpenGL: Attribute State In X, the graphics context is stored on server - Switch between multiple saved contexts to reduce network traffic - But limited memory on server - There is a default context - The context is global to the application: need a policy 40 XGCValues (Xlib Graphics Context) typedef struct { int function; // how the source and destination are combined unsigned long plane_mask; // plane mask unsigned long foreground; // foreground pixel unsigned long background; // background pixel... int line_width; // line width (in pixels) int line_style; // LineSolid, LineDoubleDash, LineOnOffDash int cap_style; // CapButt, CapRound, CapProjecting int join_style; // JoinMiter, JoinRound, JoinBevel int fill_style; // FillSolid, FillTiled, FillStippled, int fill_rule; // EvenOddRule, WindingRule int arc_mode; // ArcChord, ArcPieSlice... Font font; // default font... XGCValues; 41 drawing.min.cpp int w = 300; int h = 300; XFlush(display); sleep(1); // let server get set up before sending // drawing demo with graphics context here... GC gc = XCreateGC(display, window, 0, 0); // graphics context XSetForeground(display, gc, XBlackPixel(display, screen)); XSetBackground(display, gc, XWhitePixel(display, screen)); XSetFillStyle(display, gc, FillSolid); XSetLineAttributes(display, gc, 3, // 3 is line width LineSolid, CapButt, JoinRound); // other line options // draw some things XDrawLine(display, window, gc, 10, 10, w 10, h 10); XFillRectangle(display, window, gc, 50, 50, w (2*50), h (2*50)); XSetForeground(display, gc, XWhitePixel(display, screen)); XDrawLine(display, window, gc, w 10, 10, 10, h 10); XFlush(display); 42
Code Review: drawing.cpp initx initializes three graphics contexts main changed to call several procedures to draw drawrectanglesincorners - get window attributes (eg width and height) - use of XDrawRectangle drawstuff - parameters say which GC and where to draw - use of XDrawLine, XDrawArc, XDrawRectangle, XFillRectangle Note: Minimize window and it vanishes - Need to redraw (need event to know when) 43 Painter s Algorithm The basic graphics primitives are primitive. To draw more complex shapes: - Draw back-to-front, layering the image - Called Painter s Algorithm 44 Painters Algorithm Analogy fast and with music: http://youtu.be/ghhxtjxanm4 45
Painting Advice Keep it simple - Clear the window and redraw everything each frame - Get fancier (eg. clipping, double buffering) only if you really need to for performance reasons Repaint when necessary -- but no oftener. Flush the buffer often enough -- but no oftener. - Unless you re debugging! 46 X Windows: Events Events and the Event Loop 47 Human vs. System User Interactive System perceive present seconds milliseconds or faster express translate 48
Events Defined 1. An observable occurrence, phenomenon, or an extraordinary occurrence. 2. A message to notify an application that something happened. Examples: Keyboard (key press, key release) Pointer Events (button press, button release, motion) Window crossing (mouse enters, leaves) Input focus (gained, lost) Window events (exposure, destroy, minimize) Timer events 49 Role of the X Server 1. Collect event information 2. Put relevant information in a known structure 3. Order the events by time 4. Decide which application/window should get event 5. Deliver the event Some events come from the user via the underlying hardware; some from the window manager. 50 Receiving Events In X, applications get the next event using: XNextEvent(Display* display, XEvent* evt) Gets and removes the next event in the queue. If empty, it blocks until another event arrives. Can avoid blocking by checking if events available using: XPending(Display* display) Query number of events in queue, never blocks. 51
Selecting Input Events to listen to Don t always need all of the events. (Why?) // Tell the window manager what input events you want. XSelectInput( xinfo.display, xinfo.window, ButtonPressMask KeyPressMask ExposureMask ButtonMotionMask ); Defined masks: NoEventMask, KeyPressMask, KeyReleaseMask, ButtonPressMask, ButtonReleaseMask, EnterWindowMask, LeaveWindowMask, PointerMotionMask, PointerMotionHintMask, Button1MotionMask, Button2MotionMask,..., ButtonMotionMask, KeymapStateMask, ExposureMask, VisibilityChangeMask,... See - http://www.tronche.com/gui/x/xlib/events/types.html - http://www.tronche.com/gui/x/xlib/events/mask.html 52 Event Structure: Union X uses a C union typedef union { int type; XKeyEvent xkey; XButtonEvent xbutton; XMotionEvent xmotion; // etc.... Each structure contains at least the following typedef struct { int type; unsigned long serial; // sequential # Bool send_event; // from SendEvent request? Display* display; // display event was read from Window window; // window which event is relative to X Event 53 Java Event Structure: Inheritance Java uses an inheritance hierarchy Each subclass contains additional information, as required (not shown) 54
eventloop.min.cpp window = XCreateSimpleWindow(display, XDefaultRootWindow(display), 10, 10, 300, 200, 2, foreground, background); XSelectInput(display, window, PointerMotionMask KeyPressMask); // select events XMapRaised(display, window); XFlush(display); XEvent event; // save the event here while( true ) { // event loop until 'exit' XNextEvent( display, &event ); // wait for next event switch( event.type ) { case MotionNotify: // mouse movement cout << event.xmotion.x << "," << event.xmotion.y << endl; break; case KeyPress: // any keypress exit(0); break; XCloseDisplay(display); 55 Responding to Events (blocking) while( true ) { XNextEvent(display, &event); // wait for next event switch(event.type) { case Expose: //... handle expose event... cout << event.xexpose.count << endl; break; case ButtonPress: //... handle button press event... cout << event.xbutton.x << endl; break; case MotionNotify: //... handle event... cout << event.xmotion.x << endl; break; repaint(... ); // call my repaint function 56 Code Review: eventloop.cpp XSelectInput XNextEvent eventloop KeyPress and XLookupString character vs. scan codes (Displayable, next slide) 57
Display List displayable base class /* * An abstract class representing displayable things. */ class Displayable { public: virtual void paint(xinfo &xinfo) = 0; ; 58 Displayable Text /* * Display some text */ class Text : public Displayable { public: virtual void paint(xinfo &xinfo) { XDrawImageString( xinfo.display, xinfo.window, xinfo.gc, this >x, this >y, this >s.c_str(), this >s.length() ); // constructor Text(int x, int y, string s):x(x), y(y), s(s) { ; 59 private: int x; int y; string s; (see also Displayable Polyline) Displaying the Display List of Displayables list<displayable*> dlist; // list of Displayables dlist.push_front(new Text(event.xbutton.x, event.xbutton.y, "x")); /* * Function to repaint a display list */ void repaint( list<displayable*> dlist, XInfo& xinfo) { list<displayable*>::const_iterator begin = dlist.begin(); list<displayable*>::const_iterator end = dlist.end(); 60 XClearWindow( xinfo.display, xinfo.window ); while( begin!= end ) { Displayable* d = *begin; d >paint(xinfo); begin++; XFlush( xinfo.display );
X Windows: Advanced Techiques Animation Double Buffering Clipping 61 Animation A simulation of movement created by displaying a series of pictures, or frames. Goals: - Move things around on the screen - Repaint 24-60 times per second (frames-per-second, frame rate, or FPS ) - Make sure events are handled on a timely basis - Don t use more CPU than necessary - Easily understood code 62 Responding to Events (non-blocking) while( true ) { if (XPending(display) > 0) { // any events pending? XNextEvent(display, &event ); // yes, process them switch( event.type ) { // handle event cases here... handleanimation(xinfo); // update animation objects repaint(xinfo); // my repaint This doesn t block, but it doesn t work very well either 63
Keys to animation: managing time #include <sys/time.h> // get microseconds unsigned long now() { timeval tv; gettimeofday(&tv, NULL); return tv.tv_sec * 1000000 + tv.tv_usec; const int FPS = 30; // sleep for about 1/30 second usleep(1000000/fps); 64 Code Review: animation.cpp New events used - EnterNotify (EnterWindowMask) - LeaveNotify (LeaveWindowMask) - ConfigureNotify (StructureNotifyMask)?!? // update width and height when window is resized void handleresize(xinfo &xinfo, XEvent &event) { XConfigureEvent xce = event.xconfigure; if (xce.width!= xinfo.width xce.height!= xinfo.height) { xinfo.width = xce.width; xinfo.height = xce.height; 65 Double Buffering Flickering: when an intermediate image is on the display - e.g.: Clear, then redraw strategies Solution: - Create an off screen image buffer - Draw to the buffer - Copy the buffer to the screen as quickly as possible (hopefully between refreshes) 66
Double Buffering // create off screen buffer xinfo.pixmap = XCreatePixmap(xinfo.display, xinfo.window, width, height, depth); // size and *depth* of pixmap // draw into the buffer // note that a window and a pixmap are drawables XFillRectangle(xinfo.display, xinfo.pixmap, xinfo.gc[0], 0, 0, width, height); // copy buffer to window XCopyArea(xinfo.display, xinfo.pixmap, xinfo.window, xinfo.gc[0], 0, 0, width, height, // pixmap region to copy 0, 0); // top left corner of pixmap in window XFlush( xinfo.display ); 67 Code Review: doublebuffer.cpp XFreePixmap to free buffer when size changes drawable types: Window and Pixmap Displayables need to be updated Resizing a window generates lots of flicker, even on some apps in the default X distribution. - http://www.ruska.it/michal/flicker.html has good suggestions, including disabling the background pixmap if it isn t needed (ie: you fill the entire window anyway) - XSetWindowBackgroundPixmap(xinfo.display, xinfo.window, None); 68 Clipping 69
Code Demo: clipping.cpp XSetClipMask XSetClipRectangles if (!is_clipping) XSetClipMask(display, gc, None); else XSetClipRectangles(display, gc, 0, 0, &clip_rect, 1, Unsorted); Also: - Pattern class to init and paint an object - Random helper function 70 Summary Basic X architecture (client, server, network) Windows: opening, disposing Drawing - Models (pixel, stroke, region) - graphics contexts - Painter s Algorithm; Display lists Events (structure, selecting, event loop, etc) Animation Double Buffering Clipping 71