Embedded Algorithms for Motion Detection and Processing

Transcription

1 Embedded Algorithms for Motion Detection and Processing 28 th February 2018 Marco Castellano System On Sensor Team Leader Analog, Mems & Sensors

2 Embedding Algorithms on Sensors 2 Existing Embedded Solutions Sensor Algo Standard Solution Sensor Algo General Purpose Microcontroller Pros - Cons Low Power Number of dice 1 communication bus Number of Algos Configurability Pros - Cons (Low) Power Number of dice 2 communication bus Number of Algos Configurability

3 Conclusions 3 Innovative Embedded Solution Inertial Sensor LSM6DSOX FSM ALG1-16 MLP ALG Ultra Low Power Number of dice 1 communication bus Number of Algos Configurability Pros - Cons Two digital configurable embedded modules for fast and effective implementation of motion detection processing have been presented. A Common application cases shows 20 to 100 times current consumption reduction respect to standard solutions.

4 Motion Detection Algorithms 4 Inertial algorithms overview Face Up/ Down Glance Wake Up Fitness Activity Rec Carry Pos Free Fall Tap Tap 6 D Step Count Vibration Monitor

5 Motion Detection Algorithms 5 Inertial algorithms overview Face Up/ Down Glance Wake Up Fitness Activity Rec Carry Pos Free Fall Tap Tap 6 D Step Count Vibration Monitor

6 Two Motions Comparison 6 Face-Down Transition Foot Impact Medium Strong Slow Medium Strong Slow Facing down the phone determines a sign change on Z component of acceleration (1 g -> -1g) Foot impacts determine peaks, which shape, magnitude and frequency are heavily variable

7 Two Motions Comparison 7 Face Down Peak to Peak Occurences Foot Impact Peak to Peak Occurences Narrow Broad

8 Motion Detection Algorithms 8 Inertial algorithms overview DEDUCTIVE INDUCTIVE Face Up/ Down Glance Wake Up Fitness Activity Rec Carry Pos Free Fall Tap Tap 6 D Step Count Vibration Monitor

9 Inductive or Deductive? 9 Divide et impera DEDUCTIVE Based on general hypothesis that predict the observation with high precision Robust and fully validated hypothesis needed INDUCTIVE Based on generalization from specific observations Campaigns of data collecting needed The algo is dependant on the population under consideration (gender, age, provenience...).

10 Motion Detection Algorithms 10 Inertial algorithms overview Finite State Machine For motion Detection Machine Learning Processing Face Up/ Down Glance Wake Up Fitness Activity Rec Carry Pos Free Fall Tap Tap 6 D Step Count Vibration Monitor

11 Motion Detection Finite State Machine 11 LSM6DSOX is provided with an interpreter that decodes and executes up to 16 independent Finite State Machines Each FSM is intended to detect a single specific gesture. Wrist Tilt Free Fall Pick Up Wake-Up Shake Glance Tap Motion /Stationary Etc Easy and Effective Application Development STM Library of gestures available High level of parametrization STM development GUI High level of customization

12 Single Finite State Machine Strategy 12 Motion Detection in Three Steps Sensors Data States Motion Detection Accelerometer Conditions Interrupts Gyroscope External Sensor Commands Parameters Source Long Counter

13 Finite State Machine Inputs 13 Select Appropriate Sensor Data for the Application Sensors Data Accelerometer Gyroscope External Sensor A wide Set of Inputs to be chosen 1. Accelerometer [a x a y a z a v ] 2. Gyroscope [g x g y g z g v ] 3. Calibrated magnetometer [m x m y m z m v ] 4. Filtered signal #1 [f x f y f z f v ] 5. Filtered signal #2 [f x f y f z f v ] 6. Filtered signal #3 [f x f y f z f v ] 7. Filtered signal #4 [f x f y f z f v ] 8. Integrated gyroscope [d x d y d z d v ] Magnitude Available

14 Finite State Machine Core (1) 14 Smart Memory Allocation FSM Programs Variables and Instructions A simple program is made by: a data section, composed of a fixed part (same size for all the FSMs), and a variable part (size is specific for each FSM) an instructions section, composed of commands and conditions (size is specific for each FSM) States Fixed data Section Variable Data Section Instructions Section FIXED DATA SECTION VARIABLE DATA SECTION ISTRUCTION SECTION

15 Finite State Machine Core (2) 15 Highly Configurable Core Instructions Set States START Commands are immediately evaluated. When a command is executed, the program pointer is set to next command/condition line. RESET RESET NEXT NEXT Conditions are executed at data input arrival. If RESET is true the program pointer is set to last configured Reset Point; If NEXT is true the program pointer is set to next line. Parameters are application mask, threshold and timers Conditions Commands Parameters COMMAND PARAMETERS RESET NEXT CONT

16 Motion Detected: Status Outputs 16 High Reconfigurability on Interrupts Routing: Interrupts and Status Interrupts available for each state machine can be routed on INT1/INT2 pin Motion Detection Interrupts can be latched/pulsed. Source registers available for each state machine keeps axis positive/negative triggering the interrupt event Long counter countes an event in a FSM Interrupts Source Long Counter

17 Example: Build a Simple Effective Face Up to Face Down Detection (1) Application case: rotate phone on the table up to down to silence a call 17 Start: Phone on the table Application Definition 1) Be still, zeta axis up GNTH1 Threshold1 Threshold2 - Threshold1 2) Start transition 3) Check turning zeta respecting timer 4) Debounce 5) Be still, zeta axis down Timer End: Phone reversed

18 Example: Build a Simple Effective Face Up to Face Down Detection (2) Application case: rotate phone on the table up to down to silence a call 18 Start: Phone on the table Threshold1 GNTH1 Threshold2 - Threshold1 Timer End: Phone reversed

19 Finite State Machine Conditions 19 States are defined by means of RESET / NEXT conditions CONDITION DESCRIPTION CONDITION DESCRIPTION 0x0 NOP None execution - condition on current sample 0xB GRTH1 Any triggered axis > -THRS1 0x1(234) TI1(234) Timeout 1(234) expired 0xC LRTH1 Any triggered axis <= -THRS1 0x5(6) GNTH1(2) Any triggered axis > THRS1 0xD PZC Any triggered axis crossed zero pos. slope 0x7(8) LNTH1(2) Any triggered axis <= THRS1 0xE NZC Any triggered axis crossed zero neg. slope 0x9 GLTH1 All triggered axis > THRS1 0xF MLPCHK MLP check 0xA LLTH1 All triggered axis <= THRS1 State X Main Conditions Groups Timers Single axis threshold comparison Multi axis threshold comparison Zero Crossings RESET TRUE Reset State YES NO NEXT TRUE YES State X+1 NO

20 Finite State Machine Commands 20 Commands Overview STHR1(2) SELTHR1(3) STOP JUMP CONT Commands are immediatly evaluated (until a State is found) SELMA SELMB SM(ABC) SELMC STIMER3(4) STCT0(1) SRP CRP UMSKIT MSKIT MSKITEQ CANGLE REL SINMUX Thresholds Mask Timers Execution Output Various

21 Machine Learning Processing 21 An interpreter decodes and executes up to 8 independent decision trees Each Application is intended to detect user contexts. Activity recognition Fitness activities Motion intensity Vibration intensity Carry position Context awareness False positive rejection Etc Easy and Effective Application Development STM Library for context awareness Fully reconfigurable STM development GUI Designed to be used with machine learning tool

22 Machine Learning Approach 22 Define Classes to be recognized Self learning approach Collecting multiple Logs Configure the device and run the application. Define Features that better characterized the defined classes; Define Decision Tree using machine learning tools

23 Machine Learning Processing Strategy 23 Context Awareness defined in Three Steps Sensors Data Computation Block Decision Tree Accelerometer Filters Connections Gyroscope External Sensor Features Connections Meta-classifier Results

24 Motion Processing Inputs 24 Select Appropriate Sensor Data and Filtering for the Application Sensors Data Accelerometer A wide Set of Inputs to be chosen 1. Accelerometer [a x a y a z a v ], [a v2 ] 2. Gyroscope [g x g y g z g v ], [g v2 ] 3. External sensor [m x m y m z m v ], [m v2 ] Gyroscope External Sensor

25 Motion Detection Processing(1) 25 Filters: Smart Memory Allocation Inputs to Computationals Block are Filtered Data 1. High Pass [f x f y f z f v ], [f v2 ] 2. Band Pass [f x f y f z f v ], [f v2 ] 3. First Order IIR [f x f y f z f v ], [f v2 ] 4. Second Order IIR [f x f y f z f v ], [f v2 ] Computation Block Filters FIXED DATA SECTION VARIABLE DATA SECTION

26 Motion Detection Processing(2) 26 Features: Smart Memory Allocation Filtered Data is Trigger (to be configured as feature) Mean Variance Energy Peak Zero crossings Min Max Duration Etc. Computation Block Features Connections FIXED DATA SECTION VARIABLE DATA SECTION

27 Motion Detection Decision Trees (1) 27 Outputs from Computation Blocks are Inputs to Decision Trees Fully Reconfigurable Decision Trees Topology Tree Example Typical node Decision Trees Start Input Condition node node True Path aaaa False Path Connections Topology Connections define the algorithm behaviour

28 Motion Detection Decision Trees (2) 28 Decision Tree Outputs Can Be Filtered Up to 8 decision trees, Up to 8 Meta-classifier for 8 Decision Tree Family Results A Meta-classifier is a outliers filter [M,N] Decision Trees Example: [3,4] meta-classifier IN a a a b a b b b Ca Cb OUT x x a a a a a b Meta-classifier Results Result Interrupts and Status When a change of result is detected

29