Inventions on auto-configurable GUI-A TRIZ based analysis

Transcription

1 From the SelectedWorks of Umakant Mishra September, 2007 Inventions on auto-configurable GUI-A TRIZ based analysis Umakant Mishra Available at:

2 Inventions on Auto-Configurable GUI A TRIZ based analysis By- Umakant Mishra, Bangalore, India Table of Contents 1. Introduction User configurable GUI Ideal Final Result and auto-configuration Inventions on adaptive and self configurable graphic user interfaces Predictive and adaptive computer interface Method and system for efficient organization of selectable elements on a graphical user interface Adaptive menus Web document based Graphical User Interface Method for dynamically displaying controls in a toolbar display based on control usage Menu management mechanism that displays menu items based on multiple heuristic factors Adjusting a click time threshold for a graphical user interface Method and system for automatically resizing and repositioning windows in response to changes in display Summary...13 Reference to patents:...14 Other references: Introduction The graphic user interface is a revolutionary development over the prior art technique of the command line interface. The graphic user interface displays various program elements (such as, commands, functions, internal operations etc.) through friendly graphic elements (such as, buttons, icons etc). This helps the user to do even complex operations through simple means like clicking buttons or dragging various graphic elements.

3 Although a GUI is generally considered to be convenient, a poorly designed GUI may not be found convenient to a user. Hence the developer has to consider various aspects while developing a graphic user interface. However, there is a typical issue involved in designing a GUI. The GUI is intended for the end user but designed by the developer. Even the experienced developer cannot realistically anticipate what options will be required by the users at a future point of time. Besides, even a well-designed GUI may not be felt convenient to all categories of users as the skill, expertise and requirement of the users vary from each other. For example, an advanced user may require some advanced options to be displayed on the toolbar, but a new user may get confused to see those advanced options. Hence it is necessary to provide some option to customize the graphical user interface. 1.1 User configurable GUI As the user interface is meant for the user, it is the user who can best decide what features he is comfortable with. Secondly, the requirements are different from user to user. Hence, no other person can determine the user requirement as accurately as the user himself. Thirdly, the same user may prefer different configurations at different types of jobs and at different levels of expertise. All the above considerations necessitate the GUI to have some options to be configured by the end user. There are various aspects of a GUI, such as, inclusion and exclusion of GUI elements, location and placement of GUI elements, altering the color, appearance, size, sequence and other aspects of GUI elements, which are often allowed to be customized by the user. The details of user configurability are discussed in the previous article Inventions on User Configurable GUI- A TRIZ based analysis by the same author. However, there are some problems involved in implementing user configurability. The user needs to have certain level of competence to configure the GUI. The user needs to spend some time to configure the GUI. The user has to change the configuration again and again depending on changes in his working environment. The user has to alter configurations according to increasing familiarity. The customization done by one user may not suite to another user. In such cases the users of a shared computer will face difficulties.

4 This problem can be described in the form of a contradiction. The GUI needs to be customized to meet the changing requirements of the user, but the user should not take the trouble of customizing the GUI (Contradiction). This problem can be addressed by automatic-configuration of the GUI (Principle- 25: Self service). Although it is difficult to automate every aspect of a GUI, there are many aspects, which can be adjusted automatically to cope with the changing requirements of the user. 1.2 Ideal Final Result and auto-configuration Ideally the GUI should contain all those buttons (or menu items or GUI elements) and only those buttons (or menu items or GUI elements) that the user needs. Besides the GUI elements should obscure no display area, should take no time for searching, and should cause no confusion to any level of user (IFR). Although the IFR is difficult to achieve, it helps to draw a set of desired results in the line of IFR, which may be listed as follows. The GUI should contain all the options required by the user. The GUI should not contain unnecessary items that may cause confusion. The (meaning and functionality of) GUI elements should be selfexplanatory, creating no confusion even to the novice user. The GUI elements should be automatically organized to display the most desirable options on the top for ease of selection. The GUI elements should occupy minimum screen space thereby leaving the valuable screen space to be used for other purposes. The buttons for advanced features should appear or disappear depending on the requirement of the user. The items in the menubar, toolbar etc. should change according to user requirement. The GUI should be capable of displaying any number of items that may be required by a user without obscuring other information on the screen. The GUI elements should be self-exploring or the user should do minimum effort to locate a desired function. The GUI should be easy for selection and navigation.

5 Similar GUI elements (like toolbar buttons) should be automatically grouped for easy identification. The GUI elements should be automatically added, deleted or altered to cope with the changing familiarity of the user. All the above desirables require a lot of automation in the graphical user interface. There are many inventions on automating various features of GUI. Some of them (selected from US Patent database) are illustrated below. 2. Inventions on adaptive and self configurable graphic user interfaces 2.1 Predictive and adaptive computer interface (US Patent ) Background problem Sometimes the user repeatedly goes though a particular route to select a menu. The user has to navigate the same route again and again wasting a lot of time. For example in CAD application, the user may first select Draw menu, then select Circle option, then select Center and Radius option. There is a need to access these remote items in a faster way. Solution provided by the invention Siefert et al. disclosed a method of predictive and adaptive interface (Patent , Mar 98), which studies the behavior of a user in selecting options in a computer program and determines which options the user prefers. Subsequently when the options are presented to the user, the invention emphasizes the preferred options and de-emphasizes the others. For example, if a particular user repeatedly selects one option from a given menu, the invention detects this repeated selection, and adapts to the user s selection by eliminating other options from the menu.

6 The invention analyses the interaction of the user with the menu items and presents the desired remote options directly to the user. For example, the user can select the desired option Center and Radius directly instead of going through Draw and Circle options. In this case, the process could save 2/3 of the time by avoiding the two intermediate options. TRIZ based analysis The menu should have sub-menus and sub-sub-menus to accommodate more number of menu items. But the user should not click again and again to go through menus and sub-menus to explore the desired option (Contradiction). The invention analyses the interaction of the user presents the desired remote options directly to the user (Principle-10: Prior Action, Principle-21: Skipping). 2.2 Method and system for efficient organization of selectable elements on a graphical user interface (US Patent ) Background problem Organizing the selectable elements on a GUI in a logical manner can help user to select the desired option quickly and conveniently. Typically the most frequently used elements are placed in most accessible screen region for fast access. But there are difficulties to achieve the above efficiency. Firstly, the conventional GUI is static so they do not change according to the usage pattern of the user. Secondly, the usage pattern of users at different skill levels is different. There is a need to automatically organizing the selectable elements on a GUI as the user needs changes and the application receives new features. Solution provided by the invention Patent (invented by Nielsen, assigned by Sun Microsystems, issued in Dec 1999) discloses a method of automatically organizing selectable elements on a GUI in an efficient manner.

7 The invention keeps a statistics of access frequency to the elements, and organizes the elements depending on the access frequency. The elements of higher access frequency are placed in a first area and the elements of a lower access frequency are placed in a second area. (This patent was continued further in patent issued later in May 2000). TRIZ based analysis The invention counts the frequency of user access to elements and differentiates between the elements with higher access frequency and those with lower access frequency (Principle-1: Segmentation). The method puts the more frequently used elements in a location that is easily accessible so that the user takes less time to search and select those elements (Principle-10: Prior action). The invention moves the elements dynamically from first area to second area (or vice versa) according to their frequency of access (Principle-15: Dynamize). 2.3 Adaptive menus (US Patent ) Background problem: The designer of computer applications provides more and more commands in the menus for easy access to the user. However, it suffers from its negative side as the increasing number of menu-items clutters the screen and confuses the users. Example Invention: US Patent (Invented by Arcuri et al, assigned by Microsoft, Sep 2000) discloses a method of dynamically changing the available commands in a given short menu based upon the usage pattern of the user. Initially the menu opens in a short menu mode. The short menu can be dynamically expanded to long menu mode to display the complete set of commands.

8 The items in the short menu change according to the usage pattern. The most frequently used commands are added to the menu and less frequently used items are removed. This adaptive menu solves most of the earlier problems. TRIZ based analysis Ideally the menu should be able to accommodate any number of items. The menu should consume less (or no) space on the screen and the user should have no problem in navigating the menu (Ideal Final Result). When the menu contains more number of items it requires more space (or needs scrollbar) and looks confusing to the user. If the menu is displayed in less space then it cannot accommodate sufficient number of items (Contradiction). The menu should contain maximum number of options to offer all desired options to the user. At the same time, it should contain minimum number of items to consume less space and facilitate easy navigation (Contradiction). The items in the menu change dynamically according to the usage pattern of the user (Principle- Dynamize). The invention allows the menu to be short or long dynamically according to the number of elements to be displayed (Principle- Dynamize). More frequently accessed items are added and least frequently accessed items are removed from the menu dynamically (Principle- Discard and recover). 2.4 Web document based Graphical User Interface (US Patent ) Background problem Generally the GUI of standard packages contains predetermined operations. The user is not able to customize or edit the features of GUI. Every time the GUI is modified, the user must install or upgrade the program on his computer or network. There is a need for a GUI that is editable and easily upgradeable without the user installing the programs again and again. Solution provided by the invention Van Hoff invented a method of web based GUI (Patent , assigned by Sun Microsystems, May 2001). According to the invention the GUI of the web document is located at the server computer, which is downloaded to client computer through HTTP. The web browser on client computer runs the GUI application. The web browser comprises of an editor that edits the features of the web documents on the client computer by adding/ removing links for GUI web documents. In this way, the GUI can be customized without reinstalling the application.

9 According to the invention, the applets are written in a platform independent language that is preferably Java. The web browser includes a Java virtual machine module that verifies the integrity of applets, then interprets and executes the applets on the client computer. TRIZ based analysis The invention uses GUI, which are based on web documents. The man-made changes are done at the server and not at the client (Principle-17: Another dimension). The web documents are located at the server and loaded to the client computer by HTTP. An editor module of the browser automatically edits and customizes the web documents at the client computer (Principle-25: Self service). The invention uses HTML based web documents and Java based applets, which can run on any client machine through a browser (Principle-6: Universality). 2.5 Method for dynamically displaying controls in a toolbar display based on control usage (US Patent ) Background problem The toolbar displays the icons and controls for a single click access. As a toolbar can accommodate a limited number of items generally the most frequently used options are displayed in the toolbar. Usually the developer has to determine the importance of the controls in anticipation with the users needs. Sometimes the developer has to do a guesswork on which controls are more or less important for the user. This method creates dilemma for the developer and restriction for the users. It is necessary to display the controls on the toolbar based on the need and usage pattern of the user. In other words, the toolbar should display the most important controls to the user, which may change throughout the session.

10 Solution provided by the invention Patent (Invented by Arcuri, et al., assigned by Microsoft, in May 01) provides a method of displaying the controls on the toolbar based on the usage of controls. The invention tracks the usage of the controls to determine which controls are more important and which controls are less important. It places the more important (or more likely to be used) components on the toolbar and removes the less important (or less likely to be used). When the toolbar is loaded for the first time, it loads the controls based on a predefined order. Subsequently the display of controls changes according to the usage pattern. TRIZ based analysis Ideally the toolbar should be able to accommodate all the controls required by the user and it should occupy minimum (or no) screen space (Ideal Final Result). The invention divides to controls in the toolbar according to their most recently used status. The most recently used controls are displayed in the toolbar and others are displayed in a drop-off well (Principle-1: Segmentation). When the user selects a control from the drop-off well display, the status of selected control is changed to most recently used (Principle-23: Feedback, Principle-35: Change parameter). If the space in the toolbar does not allow displaying all the recently used controls, it displays the most recently used controls depending on the available space (Principle-16: Partial or excessive action). 2.6 Menu management mechanism that displays menu items based on multiple heuristic factors (US Patent ) Background problem Menu control is one of the important controls used in Graphical User Interfaces. There have been several evolutions to make the menu more and more dynamic and adaptive. The prior art menu management systems consider only one or two

11 factors (such as recency and frequency of selections) to organize menu items. Besides it does not allow a user to choose which heuristic factors are considered to tailor the menu items. There is a need to consider multiple heuristic factors to control an improved menu management mechanism. Solution provided by the invention Steven Roth invented a method (Patent , assigned by IBM, July 01) of using multiple heuristic factors to manage the dynamic changes in the menu options. The term heuristic is used to generally describe information about past use. The invention provides several discrete yet complementary features for menu management. One feature is automatic menu arrangement based on a combination of frequency and recency of selection. Another feature is the consideration of time of the day for menu arrangement. This allows the user to order that certain menu items are presented during certain hours of the day. The invention was further expanded in patent by the same inventor Roth, Assigned by IBM, issued in June 03. TRIZ based analysis The arrangement of the menu should change automatically to suite the user s need (Ideal Final Result). If the menu is programmed to organize itself based on user interaction, the menu organization will not suit to another user working on the same computer. Besides, a person may work on different utilities at different times, If the menu is adjusted to one utility, it will not suit to the other (Problem). The invention uses multiple heuristic factors such as frequency of selection, recency of selection, time of selection etc. to arrange the items accordingly (Principle-40: Composite).

12 2.7 Adjusting a click time threshold for a graphical user interface (US Patent ) Background problem A graphical user interface is mostly operated by a pointing device like mouse. The same mouse button can be used for different purposes by using in different ways, such as single clicking, double clicking and even multiple clicking. For example, a single-click may simply highlight a GUI element and a double-click may execute a function for the GUI element. Generally a timing threshold is used to differentiate between two single-clicks and a double-click. Fixed timing thresholds sometimes tend to frustrate some users especially who are too slow or too fast in clicking operations. That is why some operating systems provide an option to set the threshold timing through a control panel. These adjustments also have some drawbacks. For example, a parent may start a computer system and then allow a child to play a game. In such a case the threshold is adjusted for the parent and the child may find difficult to adjust with that speed. Even the same user may change his interaction with the GUI over a period of time because of reasons like fatigue. Solution provided by the invention Patent (invented by Bates, et al., assigned by IBM, issued in Sep 2002) provides a method of monitoring the user-click events and dynamically changing the threshold level according to changes in the operator s behavior. According to the invention a click event is accepted for execution only if the click time threshold of that click event matches with that of the previous clicks. Besides the current click event is used to determine a new click time threshold for the subsequent click events. Thus the adjustment of click event recognition is dynamically made while the user interacts with an application. In addition, the button monitoring may initialize

13 its default click event based on the history of the usage by a user or for an application. For example, the game program s default click time threshold is different from the business software application. TRIZ based analysis The invention allows to set different click time thresholds for different users (such as, parent, child etc.) and different type of applications (such as, game application, business application etc.) (Principle-3: Local quality). The invention continuously monitors the actuation of the mouse button (Principle- 20: Continuous action) and adjusts the click time threshold automatically based on the duration of current click events (Principle-15: Dynamize, Principle-25: Self service). According to the invention a click event is accepted for execution only if the click time threshold of that click event matches with that of the previous clicks (Principle-9: Prior counteraction). 2.8 Method and system for automatically resizing and repositioning windows in response to changes in display (US Patent ) Background problem Some display systems offer the user the ability to change the resolution of the display device. But increasing or decreasing the resolution results in decreasing and increasing the size of display items correspondingly, and often shifts the location of components from their original position. But there are some components, such as tool palettes and utility windows, which are not desirable to be shifted from their original positions. Similarly, while switching the display from a full-sized monitor to an LCD screen, the positions and sizes of the objects on the display can change significantly, because of the operating parameters of the two display devices. This may even make some items unviewable because they are positioned outside the display area. It is necessary to provide a mechanism that is capable of maintaining windows and similar objects in an accessible condition even if there are changes in the display environment. Solution provided by the invention Rodden et. al, disclosed a method of automatically resizing and repositioning windows (Patent , assignee Apple Computers, Oct 2002) in response to changes in display environments or display parameters. According to the invention, the window position is recalculated and redrawn within the newly available display area. A minimum size is retained to display at least the minimum required information. For example, if the window is a utility window containing buttons, the minimum size of the window requires that at least one button be visible.

14 If the minimum size is not present in the available display area, the window is moved into a free area to attain the minimum size. Conversely, if there is more space after displaying the minimum information, the system decides whether to display additional integral components. After the window is displayed, the control elements, such as scroll buttons etc. are redrawn at the new size and position. TRIZ based analysis The invention resizes and relocates the windows to be displayed in an altered display environment (Principle-15: Dynamize). Although the components are resized, the positions of the windows remain in tact regardless of changes in the size or resolution of the display device (Principle-39: Calm). If the space is not sufficient to display the window, the window shrinks to display the minimum required information. Conversely, if there is space to display more than the minimum required information, the system will determine whether to display additional integral components (Principle-16: Partial or excessive action). 3. Summary It is necessary to customize a GUI, as the developer cannot anticipate the exact requirements of users at the time of development. Although the user can best describe his requirement and therefore best configure the GUI, it is not advisable for the user to spend time and effort to configure the GUI. This problem is solved by the concept of automatic configuration. There are many problems involved in the process of automatic configuration. The article illustrates and analyzes various inventions on automatic configuration. Some of the interesting solutions are as below. The frequency of the user interaction is tracked and analyzed to derive the user behavior and user requirement. Frequently accessed items are displayed directly to the user instead of displaying in remote menu options.

15 The elements with higher access frequency are automatically placed in an area that is easier and faster to access. The number of items in the menu increases or decreases based on the number of frequently used items. The size of the menubar or toolbar increases or decreases in order to accommodate the desired number of items. The GUI may automatically classify the functions into basic or advanced according to the pattern of their usage. The most recently used controls are automatically included in the toolbar. If there is no space the lease recently used item is removed. Using multiple heuristic factors like frequency of selection, recency of selection, time of selection etc. to arrange the menu items. Adjusting the click-time threshold automatically based on click-time of previous interactions. Adjusting different click-time threshold for different users and different applications based on user clicks. Automatically resizing the windows and the contents of windows according to changes in resolution and other display parameters. Automatically moving windows to new locations when there is not enough space to display the minimum number of controls in the window. Thus there are many aspects of a GUI, which can be automated to make the GUI adaptable to the changing requirements of the user. Automation is one of the most desirable feature in every system and so in a graphical user interface. There are many more inventors working to automate more and more aspects of a GUI. It will not be surprising to see a completely adaptable GUI in future, which can change all its functions and behavior according to the varied expectations of different types of users. Reference to patents: 1. US Patent , Predictive, adaptive computer interface, invented by Siefert et al., assignee NCR Corporation, issued Mar US Patent , Method and system for efficient organization of selectable elements on a graphical user interface, invented by Nielsen, assignee Sun Microsystems, Dec 1999.

16 3. US Patent , Adaptive Menus, Invented by Arcuri et al, assigned by Microsoft Corporation, issued Sep US Patent , Method for dynamically displaying controls in a toolbar display based on control usage, Invented by Arcuri, et al., assigned by Microsoft, issued in May US Patent , Web document based Graphical User Interface, invented by Van Hoff, assigned by Sun Microsystems, May US Patent , System and method for displaying data using graphical user interface control elements, invented by Dodson et al., assigned by IBM, May US Patent , Menu management mechanism that displays menu items based on multiple heuristic factors, Invented by Roth, Assigned by IBM, Jul US Patent , Adjusting a click time threshold for a graphical user interface, invented by Bates, et al., assigned by IBM, issued Sep US Patent , Method and system for automatically resizing and repositioning windows in response to changes in display, assigned by Apple Computers, Oct US Patent , Menu management mechanism that displays menu items based on multiple heuristic factors, Invented by Roth, Assigned by IBM, June Other references: 11. Umakant Mishra, The Revised 40 Principles for Software Inventions, July 2006, trizsite journal, available at: 12. Umakant Mishra, Inventions on user-configurable GUI- a TRIZ based analysis, trizsite journal, Oct 2006, available at Umakant Mishra, Inventions on Adaptable Menu- a TRIZ based analysis, trizsite journal, July 2005, available at Umakant Mishra, Inventions on Menu Interfacing for GUI Applications- a TRIZ based Analysis, Part-1 and 2, trizsite journal Feb/Mar 2006, available at US Patent and Trademark Office (USPTO) site,