Human Interaction Container Paradigm

Transcription

1 Human Interaction Container Paradigm HIT Lab. Sébastien PRAUD July 20th, 2004 THALES Research & Technology

2 Motivations Human Machine Interfaces (HMI) were designed to render applications or systems tracktable by the user, and it can be observed an increasing need on HMI that could provide assistance to the user in business execution. Such need leads to new approaches to HMI, that is model-based Human System Interaction that support Objective: investigate, develop, experiment and transfer the technology required for the next generation of intelligent user interfaces offering dynamic adaptation to user on one hand, and support model-driven approach for interaction engineering. Solution: Render the interaction logic explicit and independent from other logics (application/business logic and interface/presentation logic) Main issues: Implement the separation between the various logics Build models for the representation of the interaction context: User (profile, preferences, role, task or mission, ) Domain (application services, application state, business rules, ) Devices (workstation PC, laptop PC, PDA, mobile phone, tablet PC, ) Modalities (graphical, speech-based, gesture-based, ) Propose generic interaction processes that can be easily instantiated upon specific business domains 2 THALES Research & Technology

3 The Human Interaction Container (HIC) Paradigm Human Interaction Container Paradigm: Dig-out Interaction from MMI, by splitting Interaction on one hand, and interaction rendering for human senses (vision, touch, etc.) on the other. Current Systems HIC-based Systems Interface Processes Presentation Logic (Business Interface) Interaction Logic: HIC (Generic Interaction) Application Logic (Business Processes) Human Interaction Container Paradigm: Embodying human as sub-system in the system 3 THALES Research & Technology

4 Benefits and impacts A new paradigm of interaction Services User Benefits Context Task Device Behavior User-System Dialog UI Business centered user interface Redundancy of information Adaptation to user profiles The right information to the right person Persistency of business performance Plug and play devices Team awareness CCIS C4I Interface-Application Dialog Application Dynamic interfaces Selection of UI templates Static & dynamic GUI Voice browsing Adaptation to context Domain, task, user and device History of interaction Application state Rendering by Web browser VoiceXML browser GUI generator Mobility over networks and devices Collaborative interaction: support to Team communication Team coordination 4 THALES Research & Technology

5 Exemplification of the HIC Paradigm Current Architectures HIC-based Architectures From three tier architectures Tier One Client Tier Two: Application Server Tier Three Database Mobile Make Interaction services independent of Terminal specific use Make Application services independent of Organization specific use Tier One Client Devices Tier Two Business Interaction Services Tier Three Application Server Tier Four Database Mobile to four tier architectures 5 THALES Research & Technology

6 From background to Scientific Approach Generalization of the context-based approach from vocal natural language interaction to any type of interaction Acquisition tools Vocal natural language interaction Conceptual modeling Grammar derivation Domain Fill Models History Allow Task Cooperative interaction Vocal natural language interaction TVS approach Generalization Acquisition tools Any type of interaction Model acquisition/derivation Experts knowledge modeling Business Task User Context Models Thales business knowledge User task/mission Collaborative work User situation awareness & workload User profile, knowledge & beliefs K for intelligent interaction management Dialog context & history Allow Fill Learning & Revision Tools Cooperative interaction Any type of interaction HIC Approach User behavior learning User community learning Incremental model revision Acquired model revision 6 THALES Research & Technology

7 HIC Architecture Application 1 Application 2 Application N Application 1 Communication Task Management User Adaptation Dialogue State & History Business Interaction Patterns Application 2 Communication Dialogue Processing Task Planning Resource Management Application N Communication Input Interpretation Output Generation Business Rules Multi-modal Fusion Multimedia Generation Business User Interfaces Event Broking Event Dispatching HIC Audio Input ASR+Parsing WS Input Parsing PDA Input Parsing MP Input Parsing Audio Output Synthesis WS Output Display PDA Output Display MP Output Display Signal Pointing + Typing Pointing + Typing Selection + Typing Signal Graphics Graphics Graphics 7 THALES Research & Technology

8 First Prototype results Enhanced User Profile Management Only the information the user needs (filtering & push) Adaptive user model Application Server Database Database + Server Tiers HIC Framework Business Interaction Tier Audio HMI Client Tier Auto- Adaptation to User Terminal Automatic HMI configuration Automatic Data Filtering Workgroup Messenger Team Work modeling rule Automatic Acronyms Translator voice recognition/synthesis XML Based Communication model Platform-independent architecture IP network compatibility 8 THALES Research & Technology

9 Architecture of the HIC Framework Realization SOAP/HTTP (BlueTooth) Mobile Phone HMI (Java Midlet) Application Server (Java, C++ / CORBA) SQL Database Database + Server Tiers Java HIC Framework (Java, Perl, C++ / iros) JNI Audio Framework (C++ / OAA) Business Interaction Tier SOAP/HTTP (WiFi) Java RMI, JNI C++ PDA HMI (Java Swing) Workstation HMI (Java Swing, C++) Audio HMI (C++) Client Tier 9 THALES Research & Technology

10 Questions? Contact: THALES Research & Technology