Gesture Recognition In Automotive using Simulink. Submitted by: Priyanka Shrivastava Shilpa Kauthekar

Transcription

1 Gesture Recognition In Automotive using Simulink Submitted by: Priyanka Shrivastava Shilpa Kauthekar

2 Contents of Presentation Gesture Recognition system What? Overview Algorithm Simulink Simulink Support Package for Raspberry Pi Results achieved Enhancements & Developments Challenges

3 Gesture Recognition system What is Gesture Recognition system? Gesture recognition refers to the mathematical interpretation of human motions using a computing device (1) Wiper Speed Control Gesture Recognition in Automotive Infotainment Locking/ Unlocking Calling and many more 1.

4 Gesture Recognition system Distraction: Why Gesture Recognition? Distractions With in-vehicle audio/visual options growing immensely in there breadth and capability, too often drivers become distracted Creative Solutions Ease of operating various devices such as Wiper, Infotainment systems etc. Reduction of driving errors Lower visual load 4

5 Hand written C code Complexity Comprehend and Maintenance problem. Handling of graphics is challenging C-Level Code Writing and debugging is time consuming and prone to errors Long Design Time The focus shifts from algorithm behavior to low-level programming details lengthening the design time Iterations Blurring effect of images and noise hampers the results, so several iterations are required to be done

6 Simulink Algorithm Understanding of the algorithm became easy Time Less time consuming and cost effective Maintenance Enhances maintainability Support for hardware connectivity Simulink Support Package for Raspberry pi hardware Outputs Outputs are more accurate How Simulink Helped Us? Errors Less prone to errors Ease of Use User Friendly Noise Helped in noise reduction Debugging Debugging becomes simple

7 Input Input Gestures (in particular, Hands in this 01case) acts as an input for the system Capture Capture Camera is the source for capturing the 02gestures which serves as Inputs. Process Process Raspberry pi board dumped 03with the logic developed in Simulink Output Output The output will control motor s speed. Results can be seen in MIL simulation 04 also.

8 Algorithm RGB Image Input RGB to HSV Conversion HSV Image Intensity to Binary Conversion Detect Hand Binary Image Blob Analysis Operation for generating Minor axis Output having the finger count Blob Analysis Operation for counting fingers Image having only the fingers Subtracting the two images Structuring element creation Assumptions Testing is done on still-images, which were captured and stored in database prior to the testing Images have static background Image having only the Palm Detect Palm Morphological Process: Erosion & Dilation

9 Simulink 9

10 Simulink 10

11 Simulink 11

12 How Simulink Support Package for Raspberry Pi helped us Configuring and accessing hardware s i/o peripherals and communication interfaces Monitor and tune parameters from Simulink model while algorithms run on hardware Image source: 12

13 How Simulink Support Package for Raspberry Pi helped us 13

14 User Interface developed in MATLAB Input Display This panel shows the hand gesture images either randomly fetched or selected from database Pre-Stored Inputs This section allows user to give different wiper modes. It automatically selects an image from the database and process it Manual Feeding of Images This section allows the user to input various set of inputs which is stored in the Database Huge set of inputs were given to the to check the robustness. It was done through the usage of this section. Output This section shows the output i.e, the action taken by the Wiper Washer System.

15 Results Input Intermediate Outputs Actual Output (Simulated) The speed of the wiper changes according to the output of the Simulink. It is set to High, Medium or Low speed according to 3, 2 or 1 input respectively. Wiper will be set to Off when fist is shown. 15

16 Enhancements and Developments Intensity independent Algorithm should be able to process different intensity images Inclusion of Video Algorithm should be robust to handle fast transitions from one gesture to another Enhancements Enhancing gesture recognition to other modules like infotainment, Locking/Unlocking, Calling etc. Challenges Robustness To handle Blurring effect of images and noise Fast Computation Real time fast computation Video Effects Self-Occlusion of Images, Bulging Images, sudden transition of gestures etc. Illumination Algorithm should handle global illumination changes and shadows

17 17