Automotive demonstrator hmi proposal

Size: px

Start display at page:

Download "Automotive demonstrator hmi proposal"

Charlotte Montgomery
5 years ago
Views:

1 Automotive demonstrator hmi proposal

2 Outline T5.2B Demonstrator Architecture Distributed HMI View 1 (7 touch screen) View 2 (HUD) Multimodality and Proactiveness 2

3 Architecture GPS Mic FC LSC RSC St VISODO Polimi Reference DB Polimi/TD Visual search Video Audio/Video Audio Aggregator Akhela/Polimi Car OBD Proactive HMI Akhela 3

4 Architecture HMI GUI (GRAPHIC USER INTERFACE) Adaptive HMI Engine Proactive Decision Engine Context Engine Aggregator Phonon Remote DB User state sensors Visual Search Visual Odometry GPS OBD OpenGLES Tcp/Ip DBus QT 4.7 Phonon-GStreamer UBUNTU NATTY HARDWARE 4

5 DBUS DBus (Desktop Bus) is a simple inter-process communication (IPC) open-source system or software applications to communicate with one another. The design of DBus addresses two specific cases: 1. communication between desktop applications in the same desktop session. 2. communication between the desktop session and the operating system, where the operating system would typically include the kernel and any system daemons or processes. 5

6 Sensors daemons GPSD: This daemon monitors one or more GPS receivers attached to a host computer through serial or USB ports (Bluetooth), making all data on the location/course/velocity of the sensors available to the upper layers (Aggregator). OBDD (On-Board Diagnostics): is a daemon for monitoring vehicle's self-diagnostic and reporting capability through an usb/bluetooth adapter. OBD systems give the vehicle access to state of health information for various vehicle sub-systems. 6

7 Sensors daemons User State: This daemon reads all user parameters throught external sensors connected to the system via USB/Bluetooth (ECC, EEG, Blood Oxygen Level). Visual Odometry: This daemon constantly processes the video stream coming from the front camera and determines the current camera position. This information is used by the GUI to correctly align the AR Bubble. Visual Search: This daemon constantly processes the video stream coming from the side cameras and determines complex information about buildings and monuments. 7

8 OBD PIDS Detail Alfa Brera Protocol AUTO, ISO (CAN 29/500) Supported pids: "0101" Monitor status since DTCs cleared. (Includes malfunction indicator lamp (MIL) status and number of DTCs. "0104" Calculated engine load value. "0105" Engine coolant temperature. "010B" Intake manifold absolute pressure. "010C" Engine RPM "010D" Vehicle speed "010F" Intake air temperature "0110" MAF air flow rate "011C" OBD standards this vehicle conforms to "0121" Distance traveled with malfunction indicator lamp (MIL) on "0123" Fuel Rail Pressure (diesel, or gasoline direct inject) "012C" Commanded EGR "012F" Fuel Level Input "0133" Barometric pressure "0141" Monitor status this drive cycle "0149" Accelerator pedal position D "014A" Accelerator pedal position E 8

9 Distributed HMI - Concept Headup Display Dashboard Touch Screen Real/Simulated Sensors Distributed HMI Controller 9

10 Distributed HMI - Concept Complex information and Setup must be controlled from the touch screen display (View 1) Secondary information can be shown on HUD (View 2): current car speed/fuel level warnings road signs weather allarms distance to target 10

11 Distributed HMI - Concept HUD related display Navigation setup and overview 11

12 Distributed HMI - VIEW 1 View 1 is touchable and provides the main interaction with the user (setup and control) 12

13 Distributed HMI - VIEW 1 View 1 is touchable and provides the main interaction with the user (setup and control) gps Current driver: Luca Speed: 0 Km/h Torino 8 C 13

14 Distributed HMI - VIEW 1 Context 1: Driver starts driving without selecting any option. The Free Navigation panel is shown automatically when Speed > 20 Km/h. 14

15 Distributed HMI - VIEW 1 Context 2: Driver selects Navigation Mode from main Menu, and touches the map. Keyboard a text box are shown to enter Start and Destination: 15

16 Distributed HMI - VIEW 1 Context 2: after entering the Destination, the Route is generated and shown 16

17 Distributed HMI - VIEW 1 Context 3: When in AR mode, a message for an available AR Bubble is shown Augmented reality is available for the current position. Would you like to see it? YES NO 17

18 Distributed HMI - VIEW 1 Context 3: When the Bubble is downloaded, the VO algorithm provides the correct information to align the bubble with the current car orientation 18

19 3D Bubble Concept Visual Search + Visual Odometry + DB = 3D bubble with AR info 19

20 3D Bubble Concept Visual Search + Visual Odometry + DB = 3D bubble with AR info 20

21 3D Bubble Concept Visual Search + Visual Odometry + DB = 3D bubble with AR info 3D bubble 21

22 3D Bubble Concept Visual Search + Visual Odometry + DB = 3D bubble with AR info 22

23 Distributed HMI - VIEW 1 Context 4: System startup or request from user or low fuel level Warning: low fuel level Closest gas station: Via Scarampo, 12 - Milano Would you like to get there? YES NO 23

24 Distributed HMI View 2 Torino 5 C sunny 24

25 Distributed HMI - VIEW 2 Area 1: priority 1 (highest) - 6 slots Area 2: priority 2-6 slots Area 3: priority 3 (lowest) - 4 slots Icons move from one area to another one according to priorities Weather Area Torino 5 C sunny Area 3 Area 1 Area 2 25

26 Distributed HMI - VIEW 2 Pininfarina suggestions: keep the area in front of the driver clear put the same icon always in the same place limit to 4/5 the total number of icons projected on the HUD Weather Area Torino 5 C sunny Area 1 26

27 Distributed HMI HUD 27

28 Distributed HMI Touch Screen 28

29 Proactivity user input (touch) GUI outlook (panels, buttons, allarms, warnings, etc...) Astute Layers user state input (context) environment input (context) 29

30 Multimodality Text to speech for some info: AR (TOURISTIC) infos text messages current playing music Different sound allarms: info/warnings Touch screen for complex operations when not driving HUD visualization when possible 30

31 THANK YOU 31

ON-BOARD DIAGNOSTICS SCANNER

ON-BOARD DIAGNOSTICS SCANNER Supervisor: Dr. Samer Arandi Prepared by: Ahmad Saqfalhait & Morad Abushamma Hardware graduation project Outline 2 Background High Level View Protocols User Interface Storage