VisAssist Web Navigator Design Document

Transcription

1 VisAssist Web Navigator Design Document

2 Contents 1. Document Information Document Details Revision History Introduction Purpose Scope Design Goals Visually Impaired User Interface Familiar Content Designed for System and Application Evolution Ease of Use for the Visually Impaired Design Overview Description of Problem Technologies Used Context Diagram Component Diagram VisAssist UI Library...10 UI.1 Core Interaction...10 UI.2 Content Cursor implements GestureListener...11 UI.3 View Hierarchy...12 UI.4 Content Hierarchy...13 UI.5 Gestures...14 UI Library Traceability Matrix VisAssist MediaWiki Navigator...20 VisAssist MediaWiki Navigator Context Diagram...20 VisAssist MediaWiki Navigator UML...21 Code...21 org.visassist.mediawiki...22 org.visassist.mediawiki.article...23 org.visassist.mediawiki.article.articlecontents...24 org.visassist.mediawiki.contextmenus...25 org.visassist.mediawiki.mediawikiapi...26

3 org.visassist.mediawiki.parser...27 org.visassist.mediawiki.tasks...27 org.visassist.mediawiki.viewcontents...28 MediaWiki Traceability Matrix VisAssist Facebook Navigator...30 VisAssist Facebook Navigator Context Diagram...30 VisAssist Facebook Navigator UML...31 Login...31 Code...31 com.facebook.android...32 org.visassist.facebook...33 org.visassist.facebook.comments...34 org.visassist.facebook.events...34 org.visassist.facebook.feed...34 org.visassist.facebook.inbox...34 org.visassist.facebook.likes...35 org.visassist.facebook.logout...35 org.visassist.facebook.models...36 org.visassist.facebook.photos...36 org.visassist.facebook.profile...37 org.visassist.facebook.search...37 org.visassist.facebook.status...38 org.visassist.facebook.userlist...38 org.visassist.facebook.util...38 org.visassist.facebook.wall...39 Facebook Traceability Matrix VisAssist Twitter Navigator...66 VisAssist Twitter Navigator Context Diagram...66 VisAssist Twitter Navigator UML...67 org.visassist.twitter...68 org.visassist.twitter.activity...70 org.visassist.twitter.builder...71 org.vissassist.content...71

4 org.visassist.contextmenu...71 org.visassist.contextmenu.item...72 org.visassist.twitter.task...72 org.visassist.twitter.util...73 Twitter Traceability Matrix...73

5 1. Document Information 1.1 Document Details Title: Author(s): VisAssist Web Navigator Trevor Adams, Nate Bomberger, Tom Burdak, Shawn Busolits, Andrew Scott, Matt Staniewicz, Nate Vecchiarelli Reviewers: Tom Burdak, Andrew Scott 1.2 Revision History Version Date Editor Description v1.0 2/21/2012 AS Initial v2.0 5/1/2012 NV Updated for final Submission

6 2. Introduction 2.1 Purpose The purpose of this document is to provide a detailed implementation guide for developing and implementing the VisAssist Web Navigator as prescribed by the VisAssist Web Navigator System Requirements Specification. A concise overview of this document presents the VisAssist Web Navigator as a mobile web navigation application that provides targeted support for the visually impaired user. The applications provided with the VisAssist Web Navigator support for the popular web sites Facebook, MediaWiki, Twitter, and Amazon. A detailed framework is outlined for providing a unified user interface library (UIL) to be used by all applications to provide the user with audio and haptic feedback, coupled with a multi-touch, swipe gesture based application navigation. Additionally, the individual structure of the activities within each website application is outlined with references back to the over-arching UIL. 2.2 Scope The scope of this document is to provide a detailed layout for the implementation of the VisAssist Web Navigator user interface library (UIL) and the suite of applications that utilize the library. This document will not provide any specifications of the external API s that are utilized for obtaining information from the various websites. The structure of this document will provide low level details for implementing the UIL and the suite of web applications through UML class diagrams and a full decomposition for all methodology associated with the class diagrams. 2.3 Design Goals Visually Impaired User Interface The core system for this application suite is the UIL which provides an interactive experience for the visually impaired users in navigating through the application menus and content. The UIL is the central interaction to navigate the menu system within the individual applications as well provide auditory feedback to the user for each registered component within the applications content Familiar Content The web applications will utilize the UIL to provide interaction with the user for the functionality provided by the specific websites for each application. This will be accomplished through implementing the application specific functionality and binding it to the UIL service to allow for easy menu navigation as well as content presentation. This is accomplished through menu navigation provided through swipe gesturing through the menu trees followed by categorically organized content that is registered with the UIL. The content is provided by a unified user interface with all of the familiar website content presented in a convenient fashion for the visually impaired.

7 2.3.3 Designed for System and Application Evolution The modular design of the UIL provides a complete framework that is implemented by specific web applications. This design modularity allows for additionally web applications for additional sites to be built to follow the unified user interface provided by the UIL. Additional gestures can be added to the UIL for more navigational or content presentation actions for current and future VisAssist Web Navigator web applications Ease of Use for the Visually Impaired The entire VisAssist Web Navigator experience is designed to provide a simple and intuitive user experience for navigating and retrieving content from popular websites through touch gestures, and with auditory and haptic feedback. This user participation is a fun interactive experience that will allow the user to experience websites through a much more user friendly environment than current web navigation technology. 3. Design Overview 3.1 Description of Problem Web navigation has been traditionally a visual experience with interactive elements presented through dynamic content on the screen. This presents an unfortunate challenge for the visually impaired as they cannot interact on a primarily visual level with websites. This leaves many visually impaired users to a painful and difficult experience when interacting with many different highly interactive websites such as Facebook, MediaWiki, Twitter, and Amazon. The VisAssist Web Navigator provides a solution to these specific interactive websites by providing a unified user experience which is focused on ease of interaction with these websites for navigation and content retrieval. 3.2 Technologies Used The VisAssist Web Navigator utilizes the Android level 8 API for use on Android multi-touch devices running Android 2.2 and newer. Additionally, the individual web applications utilize the specific API that is associated with the website being targeted. These individual applications use their specific technologies including the Graph API for Facebook that is provided in the Android Facebook SDK, the REST-like API for MediaWiki, the Java SDK for Twitter, and the REST based API for Amazon.

8 3.3 Context Diagram The VisAssist Web Navigator is a suite of applications that provides a custom user experience for four popular websites. This system is built around a central user interface module that provides auditory and haptic feedback, as well as a simple gesture based touch system. Each individual application within the suite works with the external API for the specific website to gather information across the internet and form it into data structures presented to the user interface for actions based on user input and content presentation. The user interaction is accomplished with the Android view and activity user interface technologies in addition to the Android text to speech engine. Each application stores and reads user specific settings to and from the Android file system by the application configuration module.

9 3.4 Component Diagram The VisAssist Web Navigator is a system that implements a modified version of the model-view-controller paradigm. The VisAssist Gesture Handler component provides the user with a simple touch based gesture system to navigate through the various components of the specific website being targeted while providing auditory and haptic feedback with the content being presented. The System Controller is responsible for handling view presentation to the device display for the associated content. It is the central system for maintaining consistent and cohesive state for the entire application. The Content Models encapsulate the data obtained through API tasks to be presented by the associated view for the content. The API Task Handling component performs the various API calls for the application to obtain the application data and insert it into the appropriate content model and notifies the navigation system of the change for updating the display. The View component maintains a visual container for the System Controller to draw to the device screen.

10 4 VisAssist UI Library The UI library (UIL) encapsulates the commonalities between the applications, primarily related to gestures and menus. It also contains the cursor which is a UI element created to aide in the navigation of content by the visually impaired. The UIL is composed of five main class hierarchies that compose the complete UI system. To see the class descriptions and method decompositions please visit For a complete UML of the UI Library please see the included file VisAssistNavigatorUILibraryUML.PNG UI.1 Core Interaction

11 The UI Core is composed of a few different systems and hierarchies. At the very center of the system is the VisAssistActivity which acts as a broker and initializer. The Activity sets up the ContentCursor which is used to drive the speech engine when content is on the screen, the EventOverlay which generates gesture events, and the ContextMenuOverlay which overlays the ContextMenus. Main hooks into the system are locate at the MenuWrapper class and the MenuModifier interface. The wrapper is used to contain the state of the menu, the modifier is used to update the screen with a new view/item that was not in the menu originally. UI.2 Content Cursor implements GestureListener VisAssistItems form the menu hierarchy, and as such are an integral part in setting things up.

12 To make things as generic as possible, the item contains the view (VisAssistView) to be displayed. The view handles all of the presentation logic whereas the item handles all of menu logic. Contents (VisAssistContent) fit into the system by providing a mechanism to deal with common reappearing items. Any item to be presented on the screen is a content, things like tweets and search results are individual contents. A single tweet maps to a single content etc. UI.3 View Hierarchy VisAssistView has two implementing classes a MenuView and a ContentListView. MenuView is, for all intents and purposes, the default implementation of the view. It has a single content and is always populated. The ContentListView is a special view designed to support lazy loading of contents (from web requests) and lists of contents.

13 UI.4 Content Hierarchy Contents are an integral part of the system, as such, there are a few default implementions that the navigators can use and extend. TextContent was built to display a single block of text to the screen, where as ImageContent was designed for images. Of special note, ErrorContent is the content that will be displayed any non-fatal error occurs in the system. These help enforce the one item on the screen paradigm

14 UI.5 Gestures The gesture subsystem was designed to be as compartmentalized as possible. Events get generated by a series of recognizers and when one is generated it fires off a call to the listener with the associated event. The main two listeners are the VisAssistActivity, which brokers most requests to the various subsystems and the ContextMenuOverlay which overrides some of the default events.

15 UI Library Traceability Matrix UI.1.1 UI.1.2 UI.1.3 UI.1.4 UI.1.5 UI.1.6 UI.1.7 org.visassist.ui.androidview ViewRelocator X X X X X TwoScrollView DelayedImageView org.visassist.ui.cursor ContentCursor org.visassist.ui.task PeriodicUpdateTask X X X VisAssistTask X X X org.visassist.ui.view ContentListView X X X X MenuView X X X X VisAssistView X X X X PopulateCallback org.visassist.ui.content ImageContent X X X X WebContent X X X X TextContent X X X X MenuContent X X X X ErrorContent X X X X VisAssistContent X X X X org.visassist.ui.context ContextItem ContextMenuOverlay ContextMenu org.visassist.ui.overlay.gesture BaseGesture X X X X X X SingleSwipeGesture X X X X X X TwoFingerSwipeGesture SwipeGesture X X X X X X org.visassist.ui.overlay GestureEvent X X X X X X GestureListener X X X X X X EventOverlay X X X X X X Igesture X X X X X X org.visassist.ui.menu MenuWrapper X X X X X X X MenuModifier org.visassist.ui VisAssistActivity X X X X X X X VisAssistApplication X org.visassist.ui.speech ThreadUtil X X X X X X X ParseUtil X X X X X X X State X X X X X X X

16 EarconSupport X X ContinuousSpeaker X X X X X X X org.visassist.ui.item VisAssistItem X X X X X X UI.1.8 UI.1.9 UI.1.10 UI.2.1 UI.2.2 UI.2.3 UI.2.4 org.visassist.ui.androidview ViewRelocator TwoScrollView DelayedImageView org.visassist.ui.cursor ContentCursor X org.visassist.ui.task PeriodicUpdateTask VisAssistTask org.visassist.ui.view ContentListView X MenuView X VisAssistView X PopulateCallback org.visassist.ui.content ImageContent X WebContent X TextContent X MenuContent X ErrorContent X VisAssistContent X org.visassist.ui.context ContextItem X X X X X ContextMenuOverlay X X X X X ContextMenu X X X X X org.visassist.ui.overlay.gesture BaseGesture X SingleSwipeGesture X TwoFingerSwipeGesture SwipeGesture X org.visassist.ui.overlay GestureEvent X X X X GestureListener X X X X EventOverlay X X X X Igesture X org.visassist.ui.menu MenuWrapper MenuModifier X org.visassist.ui VisAssistActivity X X X VisAssistApplication org.visassist.ui.speech

17 ThreadUtil X X X X X X ParseUtil X X X X X X State X X X X X X EarconSupport X ContinuousSpeaker X X X X X X org.visassist.ui.item VisAssistItem X X X UI.2.5 UI.2.6 UI.2.7 UI.3.1 UI.3.2 UI.3.3 UI.3.4 org.visassist.ui.androidview ViewRelocator TwoScrollView DelayedImageView org.visassist.ui.cursor ContentCursor X X X X org.visassist.ui.task PeriodicUpdateTask VisAssistTask org.visassist.ui.view ContentListView MenuView VisAssistView PopulateCallback org.visassist.ui.content ImageContent WebContent TextContent MenuContent ErrorContent VisAssistContent org.visassist.ui.context ContextItem X X X ContextMenuOverlay X X X ContextMenu X X X org.visassist.ui.overlay.gesture BaseGesture X X X X SingleSwipeGesture X X X TwoFingerSwipeGesture X SwipeGesture X X X org.visassist.ui.overlay GestureEvent X X X X X X X GestureListener X X X X X X X EventOverlay X X X X X X X Igesture X X X X org.visassist.ui.menu MenuWrapper MenuModifier org.visassist.ui

18 VisAssistActivity VisAssistApplication org.visassist.ui.speech ThreadUtil X X X X X ParseUtil X X X X X State X X X X X EarconSupport X ContinuousSpeaker X X X X X org.visassist.ui.item VisAssistItem UI.3.5 UI.3.6 UI.3.7 UI.4.1 UI.4.2 org.visassist.ui.androidview X ViewRelocator TwoScrollView DelayedImageView org.visassist.ui.cursor ContentCursor X X X org.visassist.ui.task PeriodicUpdateTask VisAssistTask org.visassist.ui.view ContentListView X MenuView X VisAssistView X PopulateCallback org.visassist.ui.content ImageContent WebContent TextContent MenuContent ErrorContent VisAssistContent org.visassist.ui.context ContextItem ContextMenuOverlay ContextMenu org.visassist.ui.overlay.gesture BaseGesture X X SingleSwipeGesture TwoFingerSwipeGesture X X X SwipeGesture org.visassist.ui.overlay GestureEvent X X X GestureListener X X X EventOverlay X X X Igesture X X org.visassist.ui.menu

19 MenuWrapper X MenuModifier org.visassist.ui VisAssistActivity X X VisAssistApplication X org.visassist.ui.speech ThreadUtil X X ParseUtil X X State X X EarconSupport ContinuousSpeaker X X org.visassist.ui.item VisAssistItem X X

20 5 VisAssist MediaWiki Navigator VisAssist MediaWiki Navigator Context Diagram The VisAssist MediaWiki Navigator is an application that provides a custom user experience for multiple MediaWiki websites. This system is built around a central user interface module that provides auditory and haptic feedback, as well as a simple gesture based touch system. It interacts with the MediaWiki API to gather information and form it into data structures presented to the user interface for actions based on user input and content presentation. To see the class descriptions and method decompositions please visit For a complete UML of MediaWiki please see the included file VisAssistNavigatorMediaWikiUML.PNG

21 VisAssist MediaWiki Navigator UML Code The MediaWiki code base is divided into packages based on the functionality of the code within.

22 org.visassist.mediawiki The code in this package is the top-level code for MediaWiki. It is responsible for initializing the application as well as maintaining it s life-cycle.

23 org.visassist.mediawiki.article The code in this package maintains the state of a MediaWiki article after it has been loaded from the internet.

24 org.visassist.mediawiki.article.articlecontents

25 The code in this package maintains the state of a section of a MediaWiki article after it has been loaded from the internet. org.visassist.mediawiki.contextmenus The code in this package maintains the state of the Context Menus in the application. The UI subsystem will call methods in these classes to generate and display the context menus to the user.

26 org.visassist.mediawiki.mediawikiapi The code in this package handles all the interaction between a MediaWiki server and the MediaWiki Navigator.

27 org.visassist.mediawiki.parser The code in this package handles parsing the HTML returned from a MediaWiki server to a displayable view in the MediaWiki Navigator. org.visassist.mediawiki.tasks The code in this package handles creating the asynchronous call to gather data as well as create the views for the application. These calls normally interface with the MediaWikiAPI classes to retrieve data from a MediaWiki server.

28 org.visassist.mediawiki.viewcontents This package contains all of the classes required to generate the different views for the MediaWiki Navigator.

29 MW.6.3 MW MW MW.6.2 MW.6.1 MW MW MW.5.3 MW.5.2 MW MW MW.5.1 MW.4.2 MW.4.1 MW.3.2 MW.3.1 MW MW MW.2.1 MW MW.1.1 MediaWiki Traceability Matrix org.visassist.mediawiki MediaWikiActivity X X X X X X X X X X X X X X X X X MenuBuilder X X X X X X X X X X X X X X X X X org.visassist.mediawiki.article Article X X ArticleCache X X org.visassist.mediawiki.article.articlecontents MWContent X X X ImageContent X X X X Section X X X StringContent X X X X org.visassist.mediawiki.contextmenus DeleteSavedArticleContextMenu X X X SaveArticleContextMenu X X X org.visassist.mediawiki.mediawikiapi IMediaWikiAPI X X X MediaWikiAPI X X X MediaWikiAPIRequest X X X MediaWikiAPIResult X X X org.visassist.mediawiki.parser Parser X X org.visassist.mediawiki.tasks MediaWikiFetchArticleTask X MediaWikiSavedArticleSearchTask X MediaWikiSearchTask X X SectionLoadTask X X X X X org.visassist.mediawiki.viewcontents ArticleContent X X X X SearchContent X X X X SectionContent X X X X X

30 6 VisAssist Facebook Navigator VisAssist Facebook Navigator Context Diagram The VisAssist Facebook Navigator is an application that provides a custom user experience for the Facebook. This system is built around a central user interface module that provides auditory and haptic feedback, as well as a simple gesture based touch system. It interacts with the Facebook Graph API to gather information and form it into data structures presented to the user interface for actions based on user input and content presentation. To see the class descriptions and method decompositions please visit For a complete UML of Facebook please see the included file VisAssistNavigatorFacebookUML.PNG

31 VisAssist Facebook Navigator UML Login Facebook uses OAuth 1.0 for authentication. The end user is required to visit the facebook website in order to acquire the necessary tokens to authenticate the application and verify credentials. After the initial login process, the Application stores the access tokens which can be used until the user deletes the application, or revokes the Application authentication at facebook.com. Code The Facebook code base is divided into packages based on the functionality of the code within.

32 com.facebook.android The code in this package is provided by google as a Facebook SDK.

33 org.visassist.facebook The code in this package is the top-level code for Facebook. It includes the Application MainActivity classes.

34 org.visassist.facebook.comments The code in this package handles acquiring and presenting comments for any Facebook object supported by the application. org.visassist.facebook.events The code in this package handles acquiring and presenting the event list for a user. It also allows the user to reply to RSVP to an event, and to see who has RSVP'd to events they are invited to. org.visassist.facebook.feed The code in this package handles acquiring and presenting news feeds. This includes the main news feed, as well as walls for events and user profiles. org.visassist.facebook.inbox The code in this package handles acquiring and presenting private messages sent to the user.

35 org.visassist.facebook.likes The code in this package handles acquiring and presenting likes for a Facebook item. It also contains the code that allows a user to like an item. org.visassist.facebook.logout The code in this package handles logging the user out of the app.

36 org.visassist.facebook.models This package contains all objects used to store Facebook data. All data acquired from Facebook, aside from response codes presented directly to the user, is stored in an object in this class. org.visassist.facebook.photos

37 The code in this package handles acquiring and presenting the photo albums and photos for a Facebook user. org.visassist.facebook.profile The code in this package handles acquiring and presenting a user's name, profile picture, and their feed, as accessed from the Friends List, Search, or My Profile branches. org.visassist.facebook.search The code in this package handles searching for users. Users can either search through their Facebook friends or all registered Facebook users.

38 org.visassist.facebook.status The code in this package allows the user to update their status. org.visassist.facebook.userlist The code in this package handles generating lists of users. This is used for the Friends List and Search results. org.visassist.facebook.util This package contains code used widely across the application, including AsyncLogin, LinkedString (used to extract URLs from strings), and FacebookTask, which overrides PeriodicUpdateTask to set a default timeout of 30 seconds.

39 org.visassist.facebook.wall This package contains the code used to post a message to a friend's wall and give the user feedback. Facebook Traceability Matrix

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66 7. VisAssist Twitter Navigator VisAssist Twitter Navigator Context Diagram The VisAssist Twitter Navigator (VTN) is an application that enables a visually impaired individual to easily navigate Twitter. The system is built around a central user interface module that provides auditory and haptic feedback, as well as a simple gesture based touch system. VTN interacts with the open source Java library Twitter4j to gather information and form it into data structures presented to the user interface for actions based on user input and content presentation. Twitter s developer framework is a RESTful API and it uses OAuth 1.0 for authentication. Twitter4j provides an easy to use abstraction that allows the developer to focus on the application UI independent of debugging and building REST queries and responses. To see the class descriptions and method decompositions please visit

67 For a complete UML of Twitter please see the included file VisAssistNavigatorTwitterUML.PNG VisAssist Twitter Navigator UML The Twitter API is simple, yet provides many different ways the data can be retrieved and ultimately presented to the user. The decision to make the Activities self-contained units allows the developer to easily abstract the Activity away from the customizable XML view scheme used by Android. There are not a lot of different types of objects in the framework, however this design will facilitate custom views by allowing each activity to containerize custom views for the different ways a user might want to see list data.

68 org.visassist.twitter Twitter uses OAuth 1.0 for authentication. The end user is required to visit the twitter.com in order to acquire the necessary tokens to authenticate the application and verify credentials. While it is possible to circumvent the 3-way handshake programmatically, it is against Twitter s terms of use. XAuth is an alternative authentication mechanism which is available to approved production ready applications. All methods were made private, since nothing needs to be exposed to any other interface. If VisAssist is able to gain approval for XAuth in future releases, exposing the login() to other activities might be appropriate. TwitterLoginActivity is a concrete class that handles the OAuth login processs. The following sequence diagram summarizes Twitter s authentication process.

69 After the initial login process, the Application stores the access tokens which can be used until the user deletes the application, or revokes the Application authentication at twitter.com.

70 org.visassist.twitter.activity This package contains the Activities used by the application. This includes the main activity and the activity used for OAuth.

71 org.visassist.twitter.builder This package contains classes used by the main activity to build the main menu. org.vissassist.content This package defines the classes used to present content to the screen. org.visassist.contextmenu This package contains classes that defines the context menus for Twitter contents.

72 org.visassist.contextmenu.item This package contains classes that define the context menu items that populate the above context menus. org.visassist.twitter.task This package contains classes that define the tasks used to populate the contents.

73 org.visassist.twitter.util This package contains classes used across the entire application. Included in this package are classes that validate network connection and download images asynchronously. Twitter Traceability Matrix

74

75

76

77

78

79

80

81

82

83