Tizen apps with. Context Awareness, powered by AI. by Shashwat Pradhan, CEO Emberify

Transcription

1 Tizen apps with 1 Context Awareness, powered by AI by Shashwat Pradhan, CEO Emberify

2 Introduction Context refers to information that characterizes a situation, between: Apps People Surrounding environment 2 Making apps smarter and more relevant to every individual user by understanding them

3 Introduction Contextual apps understand what s going on in the user s digital and physical world 3

4 Introduction Today, an average smartphone has about 10 sensors Contextual data: Current location, time, surrounding brightness, user activity User s digital world: Apps being used, Facebook API, Instagram API Wearables & IoTs are bringing in many new data points 4

5 Some contextual experiences Alarm based on weather & traffic to work by location sensing Phone goes on vibrate based on proximity to office or a movie theatre Reminders based on travel tickets on Adaptive UI App theme changes according to surrounding brightness 5

6 Contextual Lifecycle Sense, understand and adapt Get user data from sensors or social networks Build algorithms to understand the contextual data Personalize content & provide proactive recommendations Sense Understand Adapt 6

7 Contextual Lifecycle Example of a context aware alarm app Senses the location of the user Understands the current weather & traffic through an API and usual waking up time Adapts by letting the user sleep longer or shorter based on these conditions Sense Understand Adapt 7

8 Context with Tizen Tizen on mobile came with a large set of Context APIs so developers don t need to access sensors directly Activity Recognition (Wearable Also) Contextual History Contextual Trigger Gesture Recognition (Wearable Also) 8

9 Context with Tizen Activity Recognition Stationary, walking, travelling, running with accuracy Contextual History Device Usage Patterns like app usage, peak time and commonly used settings Contextual Trigger Based on a contextual event the app or notification is triggered Gesture Recognition Shake, Tilt, Snap, Orientation and other gestures are detected 9

10 Context with Tizen Apart from the data from the Context API s, user data can be obtained from Social API s Calendar and contacts can provide apps with information about the user s physical world Using Sensor API s sensors can be directly polled For most cases Tizen Context API s are doing the sense and understand part for developers 10

11 Context with AI With Artificial Intelligence, context aware apps can be more proactive and intelligent AI algorithms can help make future context predictions Context awareness can make AI applications like Chatbots smarter 11

12 Context with AI Example- The chatbot needs to recommend the user places when it s raining Chatbot: Since it is/is going to be raining outside I am recommending you indoor places Gives the Chatbot more human intelligence properties considering the user s context 12

13 Context with AI AI can help with contextual apps with: Sensing with abstraction of data Sensors can generate huge amounts of data from which AI algorithms can help extract the relevant data Also, understanding data through audio, images and video can be done through AI branches like computer vision & speech recognition 13

14 Context with AI User Profiling Gathering preferences of a user through sensors, behavior, social networks or even explicitly Personalizing the content and Adaptive nature of the app according to different user profiles AI techniques like association rules or case-based reasoning can be used 14

15 Context with AI User Profiling Example- News app profiles users to give relevant articles based on user s interests General preferences (Time), Short-term interests (Sporting event) & Long-term interests (Politics) Keywords can be weighted to prioritize stories 15

16 Context with AI Context Reasoning Based on the context situation an app may need to adapt Examining the contextual information and making a decision based on rules and logic The decision logic can further evolve with Machine Learning algorithms 16

17 Context with Machine Learning ML algorithms learn from and make predictions on data ML algorithms work on models have to be made based on sample inputs 17 Enables context prediction which sensor data could be most important in the future

18 Context with Machine Learning Using a combination of sensors, Machine Learning models can be used to determine user activity Extract sensor data and train ML models Multiple context data used together can give more specific information about the user Example Accelerometer & Barometer can be used together to detect walking vs cycling Sensor Data Machine Learning 18 Server / On-device model

19 Context with Machine Learning ML algorithms make sense of noisy/conflicting data from sensors Large datasets are useful to train & fine tune Machine Learning models 19 ML algorithms use raw sensor data to churn out signals based on training models like high level activities

20 Technology Powering Contextual Apps Six technology forces powering contextual apps: Mobile Social Media Sensor evolution Cloud & Big Data Wearable & other IoTs Artificial Intelligence 20

21 Case Study Launchify App recommendation widget Predicts which app the user needs right now in the widget 21 Context signals measured for prioritizing information: User travelling App Usage Patterns Location

22 Case Study Launchify App recommendation widget Using Machine Learning algorithms to learn based on user behavior Simple weighing algorithm to give each contextual parameter weight to priorities Senses where, how long, how often, what situations are the apps being used 22

23 Experiences with Contextual Apps Some common sense assumptions are needed in addition to the sensor data based on general human behavior to get more accuracy Sometimes sensors can give us conflicting data. Use multiple sensors to confirm it Simple logic can be applied to the algorithm like repeating of a certain event occurrence before counting it to avoid random events 23

24 Use Cases Proactive Recommendations Recommending the user outdoor places on their lockscreen if there's a chance of rain Lifelogging Quantified Self apps to track the user s life automatically Adaptive User Experience Automatically changing the theme according to sorrounding brightness 24

25 Use Cases Context is the secret sauce making an app smart & unique Foursquare doubled down on locations services to give proactive recommendations of food when you re sitting at a restaurant The contextual fabric can provide a personalized experience New value for users can help apps find interest in App Stores crowded with millions of apps 25

26 Limitations Contextual data is not always accurate Allow the user to correct and edit the contextual data 26 Eg. Slow driving is often confused as cycling Machine Learning models take huge amounts of data to train for accuracy

27 Limitations - Privacy With Context Aware apps you need to be transparent what the app is doing with the user s data There needs to be a clear privacy policy User s should be able to disable the services Encryption and security protocols need to be in place 27

28 Limitations - Privacy Rather than providing the wow factor some contextual apps go over the freaky line Nokia s Trapster (Similar to Waze) would allow it s users to stalk other users accurately Huge user privacy & trust issues 28

29 Limitations Battery life Sensors and background services can consume lots of battery life Data should be polled on triggers rather than a timer 29 Rather than going to the sensor every time it would be more efficient to get data through an app that just polled the data

30 Future More IoTs and wearables will bring in new sets of data and better quality too Smart Cars & Smart Homes will also add to user information Apps will be more automatic with better sensing Contextual Apps will be more proactive in nature Smartphone OS s will take more contextual information to become more intelligent 30

31 Future Apps will be Headless, will require minimum interface interaction Smart Notifications, voice and chatbots will be the new interfaces since apps will need lesser input with contextual information 31

32 Questions Thank 32