Voronoi Diagrams, Vectors and the Visually Impaired

Voronoi Diagrams, Vectors and the Visually Impaired Christopher Power Computer Science University of Western Ontario London, ON N6A 5B7 Canada power1@csd.uwo.ca Dawn Gill Epidemiology and Biostatistics University of Western Ontario London, ON N6A 5C1 Canada dawn.gill@schulich.uwo.ca Mark Daley Computer Science and Biology University of Western Ontario London, ON N6A 5B7 Canada daley@csd.uwo.ca Copyright is held by the author/owner(s). CHI 2006, April 22 27, 2006, Montréal, Québec, Canada. ACM 1-59593-298-4/06/0004. Abstract We describe an algorithm for the detection of targets which will be encountered by a visually impaired user while exploring a two dimensional diagram. A user test examining the success of this algorithm during a targeted search task is described. We discuss the implications of this work on interface design for the visually impaired, including the planned inclusion of this algorithm in a multi-modal document browser. Keywords Visually Impaired, Targeted Search, Voronoi Diagram, Guidance, Diagram Exploration ACM Classification Keywords H.5.2. [Information interfaces and presentation (e.g., HCI)]: User Interfaces --- Theory and Methods Introduction Recent advances in technology for the visually impaired has resulted in a collection of tools providing real-time information regarding the finger positions of blind users during exploration of tactile scenes. While technologies such as pressure sensitive touch pads [3] and refreshing pin displays [7] provide us with information regarding the actions of a user over time, how to use 1247

such information to enhance exploration tasks is still an open problem. For this contribution, we discuss an algorithm intended to provide assistance to the user during a search for a specific target in a tactile scene. This target could be any of a number of tactile features, consisting of polygons, lines or Braille characters. This algorithm, based on Voronoi Diagrams, predicts those objects the user will encounter, or pass near, during continued exploration along the path extending from her or his current finger movement direction. In this paper, we first discuss the user exploration strategies and finger movements that must be addressed by such a prediction algorithm. Then, we discuss the intuition of our algorithm. This algorithm is evaluated through a set of user experiments to determine its performance in predicting the target a user is attempting to locate in a tactile scene. We conclude with a discussion on how such an algorithm can be incorporated into a multi-modal document browser currently in development. Exploration Strategies During the interaction of a blind user with a tactile scene, such as a diagram or a map, she or he will use a variety of strategies to determine the spatial relationships of the tactile objects. These strategies include: perimeter search, grid search, object to object, perimeter to object, home location to object and cyclic search [2,5]. In many cases users will combine various strategies to improve object location encodings. Each of these strategies involves the movement of the finger to point at a specific object. The ideal movement for a user to follow would be a straight-line approach vector from the start position to target; however, such a movement is unlikely to occur during exploration. Instead, many small sub-movements are likely to change the movement vector, which indicates the current trajectory of the finger. These sub-movements include [4]: Direction change of the finger away from, or back towards the approach vector Passing through the intended target and having the finger re-enter the target. Crossing the axis of the approach vector repeatedly during the pointing task. Due to these types of sub-movements, the naïve approach of detecting which objects intersect the line extension of movement vector is unlikely to provide accurate prediction results. For this reason, we propose an algorithm that detects the intersection of the movement vector with the Voronoi regions around objects along that vector. Voronoi Diagrams We define a set of points S={s 1,s 2,,s n} with s x in two dimensional space which we refer to as Voronoi sites. For any two points p, q S such that p is not equal to q, the dominance of p over q is the subset of the plane being at least as close to p as to q. The Voronoi region of any given site p S, denoted reg(p), is the portion of the plane lying inside all of the dominances of p over the sites q S\{p}. Each of these polygonal regions is a convex polygon. This partition, V(S), of a space is referred to as a Voronoi Diagram [1]. An extensive 1248

survey of previous projects using Voronoi diagrams is available in the work by Aurenhammer [1]. Using this definition of a Voronoi diagram, we describe a method of partitioning tactile diagrams. Consider the group of objects O representing all objects contained within the tactile scene. For each object o O we define a bounding box B o as the minimal square which contains all points of object o. By calculating the center C o of each bounding box, we obtain a set of points S={C o for all o O}, which will serve as our site points for the Voronoi diagram for the tactile picture. Figure 1: (a) An embossed tactile scene consisting of 9 objects. (b) The corresponding Voronoi Diagram consisting of 9 Voronoi regions, 1 bounded and 8 unbounded. As an example, consider the diagram of boxes displayed in Figure 1 (a). The set S for this diagram, represented by 9 dots, and the corresponding Voronoi diagram is shown in Figure 1 (b). Each region contains one or more tactile features. Note that the regions reaching the perimeter of the diagram are unbounded, meaning the polygon is incomplete. We consider the perimeter of the diagram to be the closing edge(s) of these regions. Prediction Algorithm Due to the space constraints, we omit the formal algorithm. We instead discuss the intuition behind the algorithm in how it predicts the objects that are likely to be encountered by the user. Given an ordered set of finger positions P={p 1 p n}, we locate the region corresponding to position p n in which the finger of the user rests. The features contained within that region are added to the feature list F. We draw a line extending from her or his last position p n-1, through her or his current position p n, to the edge of the display area. This line, L, intersects several Voronoi regions, each containing a minimum of one tactile object. Due to the fact that the Voronoi regions crossed by the line are convex, we can guarantee that the line intersects with at most one edge of each region. If this edge is found, and it is not co-linear to the line indicating the path, we obtain the opposing region and assign it as the new current region. We recursively repeat this process, ordering the features in F by the region in which they are encountered. If the line is co-linear with the edge we end the recursion. An intersecting edge is not found in the case where the path line crosses into an unbounded region, indicating that the recursion should stop and return the feature list. This algorithm is computationally inexpensive to perform, making it ideal for a real-time application. 1249

Optimal algorithms for the construction of a Voronoi diagram require O(n logn) time, where n is the number of sites being used in the diagram [1]. Given a static diagram, this need only be computed once and stored for future reference. The prediction of the features can be completed in a worst case time complexity of O(e) where e is the number of edges in the Voronoi diagram. This worst case would only occur in a situation where the extension of the user's path crosses all Voronoi regions in the diagram. Experimental Methodology The experiment described herein was to determine the success rate of our prediction algorithm in terms of how well it predicts the target for which a user is searching. Apparatus Our apparatus consists of the Metec Dot Matrix Display (DMD) 120060, containing a surface of 7200 pins, 120 pins wide and 60 pins in height. These pins are small solenoids that are raised independently from one another. Each pin is approximately the size of a Braille point, resulting in a relatively low-resolution image (approximately 10 points per inch). For this experiment, we use one finger sensor, with the subject's index finger placed within the detection ring of the sensor. Contained within this ring is an electrical coil that detects the position of the sensor on the pin board. We set the DMD to scan for the position values for 20µs that are then transmitted to the attached computer, with a 20µs pause in between each scan. This position data was recorded on a Windows-based computer with a Pentium 4 1.2 GHz processor with 512 MB of RAM. The application recording data was implemented in the Java 5 programming language. Subjects There were 14 volunteer subjects for our experimental trials, seven of whom were female, ranging in age from 20 to 60. Each subject had no previous experience with tactile graphics, and had no significant motor impairments to affect the results. All subjects were right handed and used their dominant hand during the test. Each subject was fully sighted; however, due a plastic cover which hides the pins on the display, she or he was unable to see the picture, and thus she or he had to rely on their sense of touch for identifying features. Procedure The subjects were each presented with a collection of diagrams in a pseudo-random order as determined by a Latin square design. The diagrams were similar to the diagram shown in Figure 1 (a), with each diagram varying by the distance of the squares from the center point of the diagram. The varying distances of the squares at the cardinal points of the diagram were 4, 8, 12, 16 and 20 pins. When a diagram was first presented to the subjects, they were given the opportunity to familiarize themselves with the tactile picture through finger exploration without the finger sensor. This familiarization phase was intended to moderate training effects that may come from the subjects becoming more familiar with a particular diagram as the test progressed. 1250

The users were instructed to place their index finger in the sensor ring and to move that finger to the start point on the diagram, which was the center square. For each trial on a diagram the users were instructed to: Listen for a musical tone. On hearing the tone, move the finger to the target indicated on the screen through a textual description (e.g. North, South, Center etc.). When the designated target is arrived at, stop movement over the target. confident that the true unknown proportion of successful predictions lies between 82 to 88%. These values are reasonable as there will be some situations, such as a movement by the user that is orthogonal to the approach vector, where the algorithm will not detect the feature for which the user is searching. These, and similar, situations are ones where an application using our algorithm could correct the user s movements through, for example, audio feedback. Table 1 Sample proportions and associated 95% confidence intervals Diagram Successful Predictions Vectors Sample Proportion 95% Conf. Intervals The 20 repetitions of the trial were done to reduce the effects that training may have on the data recorded. 1 452 529 85.4 82.4 88.4 Results During each trial, the sub-movements of the user were recorded. Given two sequential position readings on the display, we generated a movement vector. Following this, we used our prediction algorithm to generate the feature list for that vector. If the feature for which the user was searching was contained within the feature list, we consider that to be a successful prediction, and unsuccessful otherwise. 2 439 510 86.1 83.1 89.1 3 448 519 86.3 83.4 89.3 4 496 572 86.7 83.9 89.5 5 516 597 86.4 83.7 89.2 In Table 1 we see that the algorithm is robust across all distances of travel as the percent of successful predictions ranged between 85 to 87% with narrow 95% confidence intervals. Using the first diagram as an example we can interpret the results as follows: the predictions in our sample were successful approximately 85% of the time. Further, we are 95% Discussion These results provide strong evidence that such an algorithm can be used in an assistive technology to provide passive guidance to the user. By this we mean that the system can provide unprompted feedback to 1251

the user simply by detecting her or his movement on the tactile display. There are examples of active guidance systems, such as the Talking Tactile Tablet (TTT) [3] or the IVEO [6] which require input from the user to provide feedback. In the case of the TTT, to determine if she or he is moving in the correct direction, the user must stop exploration and apply constant pressure to a location in order to obtain landmark information in a diagram or map. In a comparable passive guidance system we obtain information such landmarks during the exploration process. For example, if a user moves in a direction that is orthogonal to the target of interest for an extended period of time, she or he can be prompted with this information without having to stop and identify landmarks in the area. Such a system, a multi-modal document browser, is currently under development. This prototype will include the prediction algorithm presented in this paper for purposes of providing passive guidance to the user while searching for targets in technical diagrams. We also intend to use the prediction algorithm to provide a preview feature of what a user will encounter during the traversal between two points. This will provide the user with information regarding what she or he should expect to find during an active guidance exploration. This application will represent the first of its kind, and we expect it to lead to further insights regarding how relative spatial information can be provided through audio and tactile cues to assist the blind user in their diagram and map browsing tasks. Acknowledgements This research was funded in part by the Natural Sciences and Engineering Research Council of Canada through grant OGP0000243. References [1] Aurenhammer, Franz. Voronoi Diagrams-A Survey of a Fundamental Geometric Data Structure. ACM Computer Surveys 23, 3 (1991) 345-405. [2] Hill, E. W., Rieser, J. J., Hill, M. M., Hill, M., Halpin, J., and Halpin, R. How persons with visual impairments explore novel spaces - strategies of good and poor performers. Journal of Visual Impairment and Blindness 87, (1993), 295 301. [3] Landau, S. and Gourgey, K. Development of a talking tactile tablet. Information Technology and Disabilities 7, 4 (2001). http://www.rit.edu/~easi/itd/itdv07.htm [4] MacKenzie, I. S., Kauppinen, Tatu, and Silfverberg, Miika. Accuracy Measures for Evaluating Computer Pointing Devices. In Proc. of SIGCHI 2001, ACM Press (2001) 9 16. [5] Thinus-Blanc, C. and Gaunet, F. Representation of space in blind persons: vision as a spatial sense? Psychological Bulletin 121, (1997), 20 42. [6] ViewPlus Technologies. Online Product Web Site http://www.viewplustech.com/ [7] Weber, G. Reading and Pointing---Modes of Interaction for Blind Users In Ritter, G. (ed.) Information Processing 89, Elsevier Science Publishing (1989)) 535-540 1252