Patterns of coordination in Improvise visualizations

Transcription

1 Patterns of coordination in Improvise visualizations Chris Weaver The GeoVISTA Center and Department of Geography, The Pennsylvania State University ABSTRACT Visualization systems have evolved into live-design environments in which users explore information by constructing coordinated multiview visualizations rapidly and interactively. Although these systems provide built-in support for wellknown coordinations, they do not allow invention of novel coordinations or customization of existing ones. This paper presents a categorization of 29 coordination patterns that have proven to be broadly useful for visual data analysis. By coupling coordination with visual abstraction, it has been possible to realize 27 of these patterns in Improvise, demonstrated here with five example visualizations. Keywords: Coordination, design patterns, information visualization, linked views, multiple views 1. INTRODUCTION Visualization systems are increasingly adopting a user-centered development model in which domain expects work closely with visualization designers to create and coordinate multiple views rapidly and iteratively as needed during data analysis. Research on these systems has revealed numerous coordination techniques that combine navigation and selection. 1 However, most systems implement only a few of these techniques. Visualization designers coordinate views using a small set of predefined primitives that follow known patterns of coordination. Like other visualization systems, Improvise 2 enables users to specify combinations of data, views, and coordinations interactively. Unlike other systems, Improvise enables users to invent new combinations and refine existing ones using Coordinated Queries, a flexible, yet relatively high-level visualization query language for coordinating data access, data processing, and data rendering across multiple views. Visual abstraction takes place through expressions that specify how to map data attributes into graphical attributes in views. Multiple data sets can be loaded, indexed, grouped, filtered, sorted, and projected as a function of interactive navigation and selection in and between multiple views. This approach significantly extends the practical design space of exploratory visualization. Coordination patterns are inspired by design patterns. 3, 4 Just as design patterns are recipes for building software, coordination patterns are recipes for building visualizations. The term pattern is used here concretely, meaning a coherent composition of visual building blocks that produces useful visual behavior in the form of interactive dependencies between views. Numerous coordination patterns have been described in the visualization literature. This paper organizes 29 coordination patterns, both well-known and exotic, into six categories Navigation, Selection, Ordering, Containment, Mutation, and Compound and describes each pattern using examples from five Improvise visualizations. 2. NAVIGATION PATTERNS Navigation is interactive manipulation of points, ranges, and regions of space in views. In Improvise, navigation occurs by mapping mouse and keyboard inputs into points and ranges in a multi-dimensional data space displayed by views and other controls. Decoupling data from the space in which it is displayed allows for flexible display of multiple data sets in high-dimensional spaces across many views. Common coordination patterns involving navigation include: Plot+Axes. Range sliders control orthogonal cartesian dimensions. Synchronized Scrolling. Views show the same region of space. Plot Matrix. Two-dimensional views show an N-dimensional space. Overview+Detail. One view shows all data; the other, a portion. Further author information: cweaver@psu.edu, Telephone:

2 Figure 1. Visualization of simulated ion motion. (A) Axes label a plot and provide a way to change X and time independently. (B) Horizontal synchronized scrolling coordinates three time series plots showing X, Y, and Z ion positions. (C) A plot matrix shows ion motion as seen from three orthogonal sides of a cube. (D) An overview uses a portal (circled) to select the extent of a detail view. (E) A perceptual slider enables users to select a visible range of time using a translucent color gradient instead of numeric values. (F) Motion outside the cubic detail region is filtered out in the detail plot matrix but drawn in gray in a 3-D view. (G) Available data sets are shown as nested views that are rotationally coupled with a stereoscopic pair of 3-D views. Perceptual Slider. Users navigate in non-spatial dimensions such as color. Navigation-Dependent Encoding. Spatial relationships affect item appearance. Sliders and other controls are often useful for manipulating individual parameters of a visualization. In Dynamic Queries,5 non-spatial data attributes can be manipulated using range sliders. LinkWinds6 provides controls that can be coordinated with views for dynamic filtering. In Improvise, users browse visualizations by interacting with views and non-data controls such as checkboxes, text fields, and sliders. In the plot+axes pattern, axis controls are independent of plots, but perform the usual roles of marking, labeling, and handling interaction in one dimension. In figure 1A, horizontal and vertical axes are coordinated with a plot. Panning or zooming in the T and X axes causes the plot to translate or stretch horizontally and vertically, respectively. Manipulating the plot causes both axes to update appropriately. Synchronized scrolling is a common form of coordination in which two views are constrained to show the same data items or the same region of a coordinate space. For instance, plots in DEVise7 can be coordinated through visual links to show the same range of X and/or Y. In Snap-Together Visualization,8 synchronous scrolling between lists of items is achieved by coordinating their scroll actions. Improvise plots can be coordinated with each other in the same way that they coordinate with axis controls: through their range properties. In figure 1B, scrolling in three plots is horizontally synchronized but vertically independent. Variations of synchronized scrolling include two-dimensional, horizontal-only, vertical-only, and dimensionally-crossed. Plot matrices9 show an N-dimensional space as a stairstep arrangement of 2-D plots. Synchronized scrolling in this case is complicated by the need to invert the coordinates of some plots in order to produce the expected navigation behavior. In figure 1C, inverting the coordinates of a plot is a simple matter of swapping the ranges bound to its two range properties.

3 The shared Z range synchronizes vertical navigation in the XZ plot with horizontal navigation in the ZY plot. (Building plot matrices is straightforward but tedious; Improvise provides shortcuts for creating common multiview constructions.) Coordination using the overview+detail 10 technique differs from synchronized scrolling in that the entire area shown in a detail view is synchronized with a subarea of an overview. DEVise cursors are an example of this technique in which a selection box in a plot has the same X and Y ranges as some other plot. In Improvise, portals (not to be confused with portals in DataSplash 11 ) are draggable controls for selecting a rectangular region. In figure 1D, the X and Y ranges of a detail plot are coordinated with the ranges of a portal inside an overview plot. As a result, the portal covers the region in the overview (its context) that corresponds to the full region visible in the detail view. This construction can be chained to create multiple levels of detail (as in 12 ). Omitting the X or Y range produces one-dimensional overview+detail. Another use of one-dimensional portals is in perceptual sliders, which allow users to select data by thinking visually while acting spatially. In figure 1E, a plot is coordinated with a portal in a gradient view to create a perceptual slider based on color. The projection expression used by the plot visually encodes ion motion by mapping time into the same color gradient shown in the gradient view. The filter used by the plot elides points that fall outside the color range selected by the portal. Although the user perceives portal position as a selection on color, selection is actually on a range of time. (Perceptual sliders are similar to visualization sliders, 13 but present possible perceptual values instead of a distribution of actually occurring data values.) Perceptual sliders are a special case of navigation-dependent encoding, in which visual encoding of items in views is a function of spatial parameters of other views. DEVise uses this technique in the form of record links that cause a plot to render only those tuples that are visible in some other plot. In figure 1F, a 3-D view coordinates with the three detail plots so as to visually differentiate between items inside and outside the rectangular region visible in each plot. In the case of navigation-dependent filtering, the 3-D view elides motion outside the cube defined by three ranges, two of which are bound to the XY plot in the detail matrix. In the case of navigation-dependent projection, the 3-D view draws the same points in gray. Navigation-dependent filtering also occurs between the three plots. The bounds of each plot naturally filters items in two dimensions; in the third dimension, an expression defined in terms of the corresponding third range elides items. The result is true 3-D overview+detail. 3. SELECTION PATTERNS Selection consists of user interactions that distinguish arbitrary data items. Views can visually differentiate selected items from unselected items in many ways, the most common being highlighting with color. In Improvise, a selection indicates selected records by the integer identifiers assigned to them when data is accessed during visualization. Decoupling selections from data in this way separates coordination of views on data from coordination of views on selections. This approach makes it possible to coordinate multiple views using multiple independent selections of the same data set in a single visualization. Useful coordination patterns involving selection include: Shared Selection. Views highlight the same data items. Selection-Dependent Data. Selected items represent whole data sets. Selection-Dependent Encoding. Selection affects item visibility and appearance. Selection-Dependent Aggregation. Aggregates exclude unselected data items. Split Selection. Multiple views split up selectable items into disjoint subsets. Magic Selection. Data items jump between views when selected. Shared selection is a form of brushing that allows users to select items in views, and see the corresponding items in other views. In XGobi, 14 users can brush items in multiple plots of high-dimensional data. Brushing-and-linking in Snap-Together Visualization uses select actions to coordinate selections in two views of the same data. In figure 2A, two Improvise views coordinate to share a selection over data that describes the 83 counties in Michigan. The map draws counties as polygons read from ArcView Shapefiles; the table draws each county as a row of text with a nested bar plot.

4 Figure 2. Visualization of Michigan election results. (A) Shared selection of counties between a table and a map. (B) Selecting a race causes results for that race to be loaded and shown throughout the visualization. (C) A pie chart uses a filter to compare results for selected candidates only. (D) A plot highlights selected counties with gray bars. (E) The diagonal of a matrix view shows vote percentages considering only selected candidates. (F) Four plots break down election results and winning candidate party color in decreasing order of total county votes, for all counties and for selected counties only. (G) An inset view summarizes county winners for the entire state. (H) A four-layer plot colors counties by winning candidate party. (I) An additional layer displays inline detail for results at the current mouse location in a plot. (J) Semantic zoom labels counties with nested bar plots at sufficient zoom. (K) Five views show the same election results in different forms. Selecting items in either view (by clicking shapes or rows) changes the selection, causing both views to redraw with the selected items highlighted. Selection-dependent data allows users to select from multiple related data sets (or subsets of one large data set) in a single visualization, such as during analysis of a sequence of experiments. Selecting a data set in one view to show in other views is a form of drill-down. For instance, Snap-Together Visualization supports drill-down by coordinating a select action in one view with a load action in another view. Selection-dependent loading of data in Improvise is performed using an expression defined in terms of (1) data that lists the names of (or otherwise identifies) loadable data sets, and (2) a selection on that data. In figure 2B, the election results for each office are stored in separate files. The expression constructs the name of a file to load using the selected office name. Whenever the user selects an office, the visualization loads data from the corresponding file. Using expressions, the user can specify a file, URL, or database as the source of data to visualize. Selection-dependent filtering is an asymmetric version of shared selection in which the filtered view differentiates between selected and unselected items by not drawing unselected items instead of highlighting selected ones. Whereas selectiondependent filtering determines the visibility of items, selection-dependent projection determines the appearance of items. Most visualization systems can coordinate two views so as to highlight the items in one view that correspond to items selected in the other view. Highlighting is usually a fixed function of the type of view, typically implemented as a special background or outline color. In XGobi, points and lines in plots can be brushed using glyphs as well as color. By using expressions to determine the entire visual encoding of items in views, highlighting in Improvise is a user-customizable visual differentiation of selected and unselected items. Highlighting of items can appear as a special background color, reverse video, a special font, or just about any variation on visual attributes the user can dream up. Customizable highlighting can also be used to avoid conflict with normal visual encoding of items. In figure 2C, the pie chart shows vote shares for candidates selected in the Candidates table. Although both views display the same data, the filtered view elides unselected

5 Figure 3. Visualization of music playlists. (A) Split selection of the master (library) and other playlists across two table views. (B) Magic selection shifts genres from an unselected list (bottom) to a selected list (top) and back again. candidates using an expression defined in terms of the selection. The result is a kind of multi-item details-on-demand that allows comparison of details for selected subsets of items. Similarly, in figure 2D the Votes v. County plot highlights counties based on whether they are selected in the Counties table. Many visualization systems enable users to explore large data sets in limited screen space by manipulating aggregates.15 Visage16 supports aggregation of records into categories of raw and derived attributes. Improvise supports categorical aggregation through aggregate operators and group queries. Because both of these mechanisms are based on expressions, users can categorize records on arbitrary functions of attributes. One way to think of selection is as a way for users to impose an arbitrary binary categorization scheme on data. Aggregates can be calculated over either category; that is, over selected items or unselected items. Aggregates can also be calculated over the subset of items for which an attribute maps into a selected item in another data set. In figure 2E, for example, a matrix view displays nested bar plots for comparing candidates pairwise. The diagonal of the matrix displays the percentage of votes for each selected candidate relative to the total votes for all selected candidates. The aggregate is a sum of votes over records for which the candidate attribute maps to a selected candidate. Selection-dependent aggregation is essentially the application of selection-dependent filtering to data prior to aggregation. In this case, filtering is done on selected counties as well as selected candidates. Split selection is an extension of shared selection in which records are divided into non-overlapping subsets, each displayed in its own view. Division can be performed using a group query or by applying mutually exclusive filters. Selection of items occurs as if they are all contained in a single view. Figure 3A shows selection of music playlists split between two tables, one containing the full music library, the other containing all other playlists. Because both views are in single selection mode, selection of any item in either table causes all other items in both tables to be deselected. Splitting in Improvise is not limited to showing different regions of a single display space. The views that display each subset need not have the same visual encoding, or even be of the same type. For example, the second view could be a simple list showing the distribution of genres in non-library playlists using nested bar plots. Magic selection is a combination of shared selection and selection-dependent filtering in which the items in a data set are split between two complementary views. One view shows selected items; the other shows unselected items. Clicking an

6 item in either view toggles the corresponding item in the selection, causing the item to magically switch views. The magic selection pattern underscores how important it is to distinguish between selection interaction by users to divide items into arbitrary, complementary equivalence classes from highlighting visual encoding to indicate each item s equivalence class. In figure 3B, clicking a genre in either list causes it to magically reappear in the other list. Selection gestures in the top list view actually cause musical genres to be deselected; invisibility is the encoding used to highlight unselected items. 4. ORDERING PATTERNS Ordering patterns involve the visual prioritization of data items in views. The prioritization is usually spatial, determining the vertical, horizontal, or z-order placement of data items. There are many possible ways of coordinating views through data ordering, three of which are: Shared Priorities. Views display the same data items in the same order. Top Picks. Selected items appear before unselected items. Proximity Sort. Items closest to the mouse pointer come first. Shared priorities has the same purpose as shared selection (brushing), namely visual accentuation of certain records over others across multiple views. Whereas brushing accentuates items arbitrarily selected by the user, shared prioritization accentuates items as a numeric function of their attributes. In Improvise, views prioritize records using sort expressions. By sharing a sort expression, multiple views can prioritize data in a common way. In figure 2F, four plots of the same data share an expression that visually orders counties from left to right on decreasing total number of votes. When views process data for rendering, logical ordering using sort expressions occurs prior to visual ordering by visual encoding. In some views, visual ordering is a function of view type (such as top-to-bottom organization of rows in lists). In other views, visual ordering is determined by the visual encoding (such as glyph coordinates in plots). Prioritization of data in Improvise is flexible because logical ordering and visual encoding are both based on user-defined expressions. Like projections and filters, sorts can be defined in terms of selections. As a result, it is possible to create a variety of ways to coordinate selection and ordering across views. For instance, views can sort selected items before unselected ones, optionally subsorting on a derived attribute. In lists and other naturally ordered views, this approach is useful for visually grouping items of particular interest. In plots and other spatial views, this approach is useful for assuring that selected items are not hidden underneath unselected items. The top picks pattern is an example of selection-ordering coordination in a single view. Selecting items moves them in front of unselected items. In figure 4A, selected musical genres appear at the top of a list. Deselecting a genre returns it to its former position amongst unselected items. The result is similar to magic selection, but confined to a single view. Ordering can also be defined in terms of points, ranges, angles, and other navigational parameters of views. In the proximity sort pattern, items in a view are sorted in order of increasing distance from the current mouse position. Proximity sort is related to similarity sort, in which items in one view are ordered on a measure of their similarity to an item selected in another view. In figure 4B, the order of musical albums in a table is coupled with mouse movement inside the plots of a 3-D plot matrix. Moving the mouse over a view in the plot matrix causes albums to be sorted in order of increasing root mean square distance (decreasing proximity) from the current (year,time,track) location of the mouse. Album order is not affected when the values of these coordinates are all undefined, i.e. when the mouse is in none of the plots. 5. CONTAINMENT PATTERNS Containment patterns involve the presentation of additional information in existing views. Using containment patterns, it is possible to display and summarize multiple data sets with multiple visual encodings in a single context. Useful containment patterns include: Layered Views. Stacked plots with synchronized scrolling. Sliding Layers. Stacked plots with independent scrolling.

7 Figure 4. Visualization of music albums. (A) A table of musical genres shows top picks by moving selected rows to the top. (B) Albums are sorted on increasing distance from the mouse in the (year,time,track) plot matrix. (C) Compound brushing of albums; names are drawn in bold if selected in at least one plot, and in larger italics if selected in all three. (D) Small multiples summarize albums of each genre across three columns. (E) Navigation and selection in multiple views filter albums on decade, genre, duration and track count. Inset Views. Miniature overviews on top of detail views. Inline Detail. Additional layers that summarize data items. Nested Views. Glyphs that visually encode entire data sets. Nested Lenses. Transparent views in draggable, stretchable frames. Popup Detail. Transient popup windows that summarize data items. Layered views (such as piles in DEVise) enable users to visualize multiple data sets using different visual encodings in a single context. A common use of layering is to visualize a single data set using a layer to highlight items selected in a lower layer. In Improvise, plots have an adjustable number of layers each defined by its own data, projection, filter, and selection properties. In figure 2H, a four layer plot draws a map using four different projections of two data sets. The bottom layer draws all counties independent of voting results. The top three layers fill, highlight, and label only the counties which are involved in the selected race. Drawing labels in the highest layer keeps them from being obscured by shapes in underlying layers. The combination of layering and compound glyphs provides extensive control over the z-order of items drawn in plots. All four layers invoke expressions to load and downsample county polygons from Shapefiles for drawing.

8 Figure 5. Geovisualization of county-level census data. Nested lenses draw county names (A), color-coded roads (B), and urban areas (C) on top of a map. (D) Overlapping lenses implicitly reveal multiple layers of detail. (E) An additional map layer explicitly labels roads inside the lens intersection. Sliding layers is a variation of layered plots in which the ranges of different plot layers are decoupled. Users pan and zoom layers independently, allowing them to compare regions of spatially-encoded data by visual overlap. To add sliding layers to Improvise plots, it would be necessary to: (1) decouple the range properties of each layer; (2) allow users to switch layers quickly during navigation. Like layered plots, sliding layers could be coordinated with each other, allowing arbitrary sets of layers to be navigationally coupled on X and/or Y. Sliding layers could also be implemented as independent single-layer plots with transparent backgrounds, arranged in a stack. Although this layout would preclude direct navigation in all but the topmost plot, indirect navigation would be possible by coordinating layers with views and controls outside the stack. The inset views pattern is a variation of the overview+detail pattern in which the overview is placed on top of the detail view. Positioning the inset in a corner of the detail view mimics the appearance of insets in geographic maps. In figure 2G, an inset version of the state map fills each county with the winning party color; the main map draws detail as well, in the form of a nested bar chart summarizing relative candidate vote totals above the name of each county. The X and Y ranges of the inset view have been locked to prevent panning and zooming. In many visualization systems, moving the mouse cursor over a view raises popups containing details of data items at that point. For instance, clicking a shape in a DEVise plot creates a short-lived window that lists the raw attributes of the data item represented by the shape. In GeoVISTA Studio,17 clicking a glyph that represents an entire data set creates a long-lived window containing a regular view of that data. In Improvise, inline detail takes the form of an additional view layer that provides details about data items at a particular point in the view. In figure 2I, the top layer of a plot summarizes vote results for glyphs beneath the mouse. The layer encodes details for every data item into the same spot in the top right corner of the plot, but filters out results not located at the current navigational point. As a result, inline detail of multiple results occurs only when they overlap, i.e. when they are from the same county or involve similar number of votes. Nested views enable exploration of a group of related data sets by displaying each data set in its own view, all contained in a larger view. In DataSplash, portals are clickable hyperlink windows into other data spaces. In Improvise, nested views are special glyphs in which the value being visually encoded is an entire data set. As a result, all Improvise views can contain nested views. In figure 1G, a list visually encodes data files as a filename beside an icon that shows the data as 3-D points. The visual encoding of each list item is a nested 3-D view glyph that applies a second visual encoding to the data from the corresponding file. These plots are navigationally coordinated with the main 3-D stereogram through shared camera position and orientation parameters. See-through tools18 are draggable windows that modify the appearance of the user interface beneath them. In visualization, lenses are transparent controls that modify the visual encoding (including visibility) of data. In Improvise, a nested lens is a portal that acts like a transparent plot, translating and scaling its contents relative to its parent plot. In figure 5A-5C, nested lenses draw county names, roads, and urban areas on top of a county-level map of census data. Because lenses are an extension of regular portals, they navigationally coordinate with their parent views in the same way. Users can specify

9 Figure 6. Item distortion in the album visualization. Rectangle size is a function of distance from the mouse. Moving the mouse from (A) to (B) shifts the distortion center. The other plots distort in one dimension. the data, filter, and visual encoding of nested lenses just like any other view. In this case, the lens replaces the road layer in the map, and can be dragged and stretched to reveal roads in particular areas of the map. Popup detail is a variation of overview+detail and inline detail in which clicking an item creates a window containing a new view. The window can be temporary or sticky. A popup view can visually encode a single record or an entire data set, depending on what the clicked item represents. For example, clicking a region in a map view in GeoVISTA Studio creates a sticky window containing a plot of the region s abstract data. Popup detail views in Improvise would be similar to nested views. The contents of popup views could be defined in terms of attributes that specify background color, ranges, data, etc. of the popup view. Mouse clicks in glyphs in the primary view would generate popup views from these attributes. 6. MUTATION PATTERNS Mutation patterns involve radical alteration of visual encoding in response to navigation. When users indicate interest in a particular region of a displayed data space (by pointing, panning, zooming, etc.), mutation patterns differentiate data items in that region from those in other regions. As special cases of navigation-dependent encoding, mutation patterns can add detail or transform spatial attributes (such as location, size, orientation, etc.). Two common mutation patterns are: Semantic Zoom. Visual scale affects item appearance. Distortion Encoding. Navigational location affects item size and shape. Zoomable user interfaces 19 allows users to explore large data sets in a compact space. Semantic zoom 20 is a form of details on demand that lets users see different amounts of detail in a view by zooming in and out. For instance, the layer manager in DataSplash allows users to select the amount of detail by changing a view s altitude. The view draws data using the visual encodings visible at the chosen altitude. Semantic zoom in Improvise involves expressions that calculate glyphs as a function of a plot s own navigation. In figure 2J, visual encoding in the county map depends on its own two ranges, both directly and indirectly. At sufficient zoom, the top layer draws a centered label and a scaled, nested bar plot for all counties. To make the top layer easier to read, the fill layer reduces the saturation of the winning candidate s party color at the same zoom level. Although this example demonstrates synchronized zoom between plot layers, different layers can change detail at different zoom levels. One-dimensional zooming and multiple levels of detail are also straightforward. Focus+context approaches are another means to explore large data sets in limited space. Distortion techniques 21 such as fisheye lenses 22 transform the visual space in which data is drawn. In Improvise, views can apply location-specific

10 distortion encodings to data. In figure 6, the width of rectangle glyphs that represent albums in the Year v. Time plot is a gaussian function centered on the year at the current mouse location; the height is a five-tier step function centered on the album duration at the same location. Glyphs in the other two plots are scaled similarly. 7. COMPOUND PATTERNS Coordination in Improvise is not limited to binary relationships between views. Multiple views may simultaneously couple with a common view through a shared coordination. Moreover, any pair of views may be interactively coupled through multiple coordinations. Compound coordination patterns involve intercoupling of multiple views of multiple data sets through multiple coordinations to create complex interface behaviors, including: Compound Lenses. Overlapping lenses that modify data appearance in concert. Compound Brushing. Multiple independent selections of the same data set. Small Multiples. Views that display slices of a data set in parallel. Multiview+Detail. High-dimensional overviews of multiple lower-dimensional detail views. Multiform. Multiple dissimilar but reinforcing presentations of the same data set. Compound lenses consist of nested lenses acting in concert to modify the appearance of data in views, much like movable filters. 23 Composition of multiple lenses in a single view can be implicit or explicit. Implicit composition occurs when independent lenses overlap, coincidentally producing an additive graphical effect in their intersection. Explicit composition occurs when visual encoding of data in the view depends on the lens position. Figure 5D shows implicit composition of two lenses. One nested lens displays roads in selected states. A second lens displays urban areas. Roads are drawn over urban areas inside the spatial intersection of the two lenses. Although the lenses draw different data and move independently within the same context, they could share data, visual encodings, filters, or ranges. Sharing ranges would couple the lenses navigationally, much like synchronized scrolling. Figure 5E shows explicit composition of lenses. The map labels only those roads that are completely contained in the rectangular intersection of the lenses, as calculated from their ranges. The coordination is similar to navigation-dependent filtering, except that navigation occurs in the immediate context of the filtered data, i.e. in the lens directly above. Other focus+context effects can be created by customizing the coordination in terms of filtering, visual encoding, lenses, layering, or navigational and selectional dependencies. In the selection-dependent encoding example, the filter and visual encoding depend on selections on either candidates or counties. It is straightforward to extend them to depend on conjunctions or disjunctions of selections, much like additive encoding of selection highlighting in interactive externalizations. 24 As a result, compound brushing in Improvise is conjunctive or disjunctive highlighting across independent selections of the same items in different views. In figure 4C, the visual encoding of album names in a table depends both conjunctively and disjunctively on independent selection of items in three plots. Names of albums selected in at least one plot are drawn darker; those selected in all three are drawn in a larger font. Albums can also be selected in the table, independent of their selection in the plots. As this example shows, compound brushing is not merely visually conjunctive meaning that each selection affects a different visual attribute but also logically conjunctive. Small multiples 25 display slices of a single data set in parallel using the same visual encoding. An advantage of small multiples is that users can focus on differences in the data between slices rather than differences in presentation. In Improvise, small multiples can be created using individual top-level views (laid out vertically or horizontally in a frame, for example), or displayed in a single top-level view using nested views. Unlike the previous application of nested views, which displayed an arbitrary number of parallel data sets, small multiples involves splitting a single data set into a small number of categories or value ranges. In the visualization in figure 4, the small multiples technique is used twice to show how the distribution of album track count plotted against total album time varies from decade to decade and from genre to genre. Figure 4D shows small multiples as an application of nested views to a data set that has been grouped on an attribute with a small number of discrete categories, musical genre in this case. To create the other instance of small multiples, the albums data set is split into ten-year intervals. Each table serves simultaneously as a key (a name-color association), an overview (album track and time information), and a summary (album count) of each decade and genre.

11 The number of data dimensions that can be displayed in one view is limited by the number of parallel perceptual channels available for simultaneous visual encoding. As a result, a single view is often insufficient to provide a complete overview of high-dimensional data. The multiview+detail pattern addresses this problem by extending the overview+detail pattern to multiple overviews. The overviews visually encode overlapping subsets of data dimensions, acting together as a single multiview. Figure 4E shows an example of the multiview+detail pattern in which the multiview is made up of three plots and two tables. The album table view shows only albums that are visible in all three views of the plot matrix and whose genre and decade are selected in the other two table views. Each overview coordinates with the detail view through navigation- or selection-dependent filtering. Together, the five overviews filter the data items shown in the detail view. In multiform visualization, 26 multiple views present the same data set in different ways simultaneously. By providing alternative avenues for navigation and selection, users can adopt their own exploration strategies. Variation in the display of a data set can be achieved using different view types or different visual encodings. Figure 2K shows an example of multiform visualization in which a data set of per-candidate county vote totals is shown differently in five views as bar charts, a pie chart, colored text in table cells, and vertically aligned sets of rectangles. Multiform visualization in Improvise is a simple matter of sharing data sets between views. 8. SUMMARY A major goal in building Improvise has been to radically increase the flexibility of coordination and visual abstraction, as compared to other interactive visualization development environments. By coupling coordination, visual abstraction, and data querying into a single coordinated query language, multiple views can be indirectly connected through navigational parameters, selections, data sets, and visual encodings that are constructed interactively by visualization designers or even by skilled visualization users during exploration and analysis. Although the current set of patterns has grown and evolved in tandem with the design and operation of numerous Improvise visualizations, it is neither rigid nor complete. Because filtering, ordering, and visual encoding of information can be specified flexibly and precisely in terms of navigation and selection anywhere in a visualization, all of the patterns can be customized individually and in combination in the course of designing visualizations for particular analysis tasks. Moreover, many of the patterns including perceptual sliders, split selection, magic selection, top picks, proximity sort, sliding layers, and distortion encoding are specialized for particular tasks in a way that suggests they may each have numerous distinct, useful variations. Future work might focus on adapting existing coordination architectures to work on top of a common coordination abstraction layer built around Improvise. An abstracted coordination architecture would offer users a choice of several connect and operate styles, analogous to user-selectable look and feel styles in current graphic user interface environments. Users might start with a simple coordination model involving a basic set of patterns, working up to more flexible coordination models involving richer sets of patterns as their experience increases and analysis requirements evolve. Expert users might choose to work in the Improvise language directly most of the time, yet be able to prototype and refine visualizations rapidly when needed. The coordination patterns described here provide many useful avenues for visual analysis. Although the effective analytic design space in Improvise is limited in practice by its library of implemented views, visual attributes, and data processing algorithms, this library can be extended in a modular fashion to encompass the wide variety of techniques coming out of the visualization community every year. Realizing new patterns like sliding layers and popup detail in Improvise is a matter of combining modules, implementing new ones when necessary. As such, it is likely that many more useful coordination patterns have yet to be discovered. ACKNOWLEDGMENTS The author would like to thank Alan MacEachren, Mark Gahegan, and the affiliates and students of the GeoVISTA Center. REFERENCES 1. C. North and B. Shneiderman, A taxonomy of multiple window coordinations, Tech. Rep. CS-TR-3854, University of Maryland Department of Computer Science, 1997.

12 2. C. Weaver, Building highly-coordinated visualizations in Improvise, in Proceedings of the IEEE Symposium on Information Visualization, pp , IEEE, (Austin, TX), October E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software, Addison-Wesley Professional Computing Series, Addison Wesley, 1st ed., October J. O. Coplien and D. C. Schmidt, eds., Pattern Languages of Program Design, (New York, NY, USA), ACM Press/Addison-Wesley Publishing Co., C. Ahlberg and B. Shneiderman, Visual information seeking: Tight coupling of dynamic query filters with starfield displays, in Proceedings of CHI Conference: Human Factors in Computing Systems, pp , , ACM, (Boston, MA), April A. S. Jacobson, A. L. Berkin, and M. N. Orton, LinkWinds: Interactive scientific data analysis and visualization, Communications of the ACM 37, pp , April M. Livny, R. Ramakrishnan, K. Beyer, G. Chen, D. Donjerkovic, S. Lawande, J. Myllymaki, and K. Wenger, DEVise: Integrated querying and visualization of large datasets, in Proceedings of SIGMOD, pp , ACM, (Tucson, AZ), C. L. North, A User Interface for Coordinating Visualization Based On Relational Schemata: Snap-Together Visualization. PhD thesis, University of Maryland, R. A. Becker and W. S. Cleveland, Brushing scatterplots, Technometrics 29(2), pp , B. Shneiderman, The eyes have it: A task by data type taxonomy for information visualizations, in Proceedings of the IEEE Symposium on Visual Languages, pp , IEEE, (Boulder, CO), September C. Olston, A. Woodruff, A. Aiken, M. Chu, V. Ercegovac, M. Lin, M. Spalding, and M. Stonebraker, DataSplash, in Proceedings of SIGMOD, pp , ACM, (Seattle, WA), June C. Plaisant, D. Carr, and B. Shneiderman, Image browser taxonomy and guidelines for designers, IEEE Software 12, pp , March S. G. Eick, Data visualization sliders, in Proceedings of User Interface Software Technology (UIST), pp , ACM Press, (Monterey, CA), November A. Buja, D. Cook, and D. F. Swayne, Interactive high-dimensional data visualization, Journal of Computational and Graphical Statistics 5(1), pp , J. Goldstein and S. F. Roth, Using aggregation and dynamic queries for exploring large data sets, in CHI 94: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp , ACM Press, (Boston, MA), April S. F. Roth, P. Lucas, J. A. Senn, C. C. Gomberg, M. B. Burks, P. J. Stroffolino, J. A. Kolojejchick, and C. Dunmire, Visage: A user interface environment for exploring information, in Proceedings of the IEEE Symposium on Information Visualization, pp. 3 12, IEEE, M. Takatsuka and M. Gahegan, GeoVISTA Studio: A codeless visual programming environment for geoscientific data analysis and visualization, Computational Geoscience 28(10), pp , E. A. Bier, M. C. Stone, T. Baudel, W. Buxton,, and K. Fishkin, A taxonomy of see-through tools, in Proceedings of CHI, pp , ACM, (Boston, MA), April B. B. Bederson, J. Meyer, and L. Good, Jazz: An extensible zoomable user interface graphics toolkit in Java, in Proceedings of User Interface Software Technology (UIST), pp , (San Diego, CA), K. Perlin and D. Fox, Pad: An alternative approach to the computer interface, in Proceedings of SIGGRAPH, pp , (Anaheim, CA), August M. S. T. Carpendale, D. J. Cowperthwaite, and F. D. Fracchia, Making distortions comprehensible, Visual Languages, pp , G. W. Furnas, Generalized fisheye views, in Proceedings of the Conference on Human Factors in Computing Systems (CHI), pp , K. Fishkin and M. C. Stone, Enhanced dynamic queries via movable filters, in Proceedings of the ACM CHI Conference: Human Factors in Computing Systems, pp , ACM, (Denver, CO), May L. Tweedie, Characterizing interactive externalizations, in Proceedings of CHI, pp , ACM, (Atlanta, GA), March E. Tufte, The Visual Display of Quantitiative Information, Graphics Press, Cheshire, CT, J. C. Roberts, Multiple-view and multiform visualization, in Proceedings of SPIE (Visual Data Exploration and Analysis VII), R. Erbacher, A. Pang, C. Wittenbrink, and J. Roberts, eds., 3960, pp , SPIE, January 2000.