Activity Recognition Using Cell Phone Accelerometers Jennifer Kwapisz, Gary Weiss, Samuel Moore Department of Computer & Info. Science Fordham University 1
We are Interested in WISDM WISDM: WIreless Sensor Data Mining Powerful portable wireless devices are becoming common and are filled with sensors Smart phones: Android phones, iphone Music players: ipod Touch Sensors on smart phones include: Microphone, camera, light sensor, proximity sensor, temperature sensor, GPS, compass, accelerometer 2
Accelerometer-Based Activity Recognition The Problem: : use accelerometer data to determine a user s s activity Activities include: Walking and jogging Sitting and standing Ascending and descending stairs More activities to be added in future work 3
Applications of Activity Recognition Health Applications Generate activity profile to monitor overall type and quantity of activity Parents can use it to monitor their children Can be used to monitor the elderly Make the device context-sensitive Cell phone sends all calls to voice mail when jogging Adjust music based on the activity Broadcast (Facebook( Facebook) ) your every activity 4
Our WISDM Platform Platform based on Android cell phones Android is Google s s open source mobile computing OS Easy to program, free, will have a large market share Unlike most other work on activity recognition: No specialized equipment Single device naturally placed on body (in pocket) 5
Our WISDM Platform Current research was conducted off-line Data was collected and later analyzed off-line In future our platform will operate in real-time In June we released real-time sensor data collection app to Android marketplace Currently collects accelerometer and GPS data 6
Accelerometers Included in most smart phones & other devices All Android phones, iphones, ipod Touches, etc. Tri-axial accelerometers that measure 3 dimensions Initially included for screen rotation and advanced game play 7
Examples of Raw Data Next few slides show data for one user over a few seconds for various activities Cell phone is in user s s pocket Earth s s gravity is registered as acceleration Acceleration values relative to axes of the device, not Earth In theory we can correct this given that we can determine orientation of the device 8
Standing 9
Sitting 10
Walking 11
Jogging 12
Descending Stairs 13
Ascending Stairs 14
Data Collection Procedure User s s move through a specific course Perform various activities for specific times Data collected using Android phones Activities labeled using our Android app Data collection procedure approved by Fordham Institutional Review Board (IRB) Collected data from 29 users 15
Data Preprocessing Need to convert time series data into examples Use a 10 second example duration (i.e., window) 3 acceleration values every 50 ms (600 total values) Generate 43 total features Ave. acceleration each axis (3) Standard deviation each axis (3) Binned/histogram distribution for each axis (30) Time between peaks (3) Ave. resultant acceleration (1) 16
Final Data Set ID Walk Jog Up Down Sit Stand Total 1 74 15 13 25 17 7 151 2 48 15 30 20 0 0 113 3 62 58 25 23 13 9 190 4 65 57 25 22 6 8 183 5 65 54 25 25 77 27 273 6 62 54 16 19 11 8 170 7 61 55 13 11 9 4 153 8 57 54 12 13 0 0 136 9 31 59 27 23 13 10 163 10 62 52 20 12 16 9 171 11 64 55 13 12 8 9 161 12 36 63 0 0 8 6 113 13 60 62 24 15 0 0 161 14 62 0 7 8 15 10 102 15 61 32 18 18 9 8 146 16 65 61 24 20 0 8 178 17 70 0 15 15 7 7 114 18 66 59 20 20 0 0 165 19 69 66 41 15 0 0 191 20 31 62 16 15 4 3 131 21 54 62 15 16 12 9 168 22 33 61 25 10 0 0 129 23 30 5 8 10 7 0 60 24 62 0 23 21 8 15 129 25 67 64 21 16 8 7 183 26 85 52 0 0 14 17 168 27 84 70 24 21 11 13 223 28 32 19 26 22 8 15 122 29 65 55 19 18 8 14 179 Sum 1683 1321 545 465 289 223 4526 % 37.2 29.2 12.0 10.2 6.4 5.0 100 17
Data Mining Step Utilized three WEKA learning methods Decision Tree (J48) Logistic Regression Neural Network Results reported using 10-fold cross validation 18
Summary Results 19
J48 Confusion Matrix Predicted Class Walk Jog Up Down Sit Stand A c t u a l Walk Jog Up 1513 16 88 14 1275 23 72 16 323 82 12 107 2 1 2 0 1 2 C l a s s Down Sit Stand 99 4 4 13 0 1 92 2 2 258 3 7 1 270 1 2 3 208 20
Conclusions Able to identify activities with good accuracy Hard to differentiate between ascending and descending stairs. To limited degree also looks like walking. Can accomplish this with a cell phone placed naturally in pocket Accomplished with simple features and standard data mining methods 21
Related Work At least a dozen papers on activity recognition using multiple sensors, mainly accelerometers Typically studies only 10-20 users Activity recognition also done via computer vision Actigraphy uses devices to study movement Used by psychologists to study sleep disorders, ADD A few recent efforts use cell phones Yang (2009) used Nokia N95 and 4 users Brezmes (2009) used Nokia N95 with real-time recognition One model per user (requires labeled data from each user) 22
Future Work Add more activities and users Add more sophisticated features Try time-series based learning methods Generate results in real time Deploy higher level applications: activity profiler 23
Other WISDM Research Cell Phone-Based Biometric identification 1 Same accelerometer data and same generated features but added 7 users (36 in total) If we group all of the test examples from one cell phone and apply majority voting, achieve 100% accuracy Can be used for security or automatic personalization Interested in GPS spatio-temporal temporal data mining 1 Kwapisz, Weiss, and Moore, Cell-Phone Based Biometric Identification, Proceedings of the IEEE 4 th International Conference on Biometrics: Theory, Applications, and Systems (BTAS-10), September 2010. 24
Thank You Questions? 25