Mobile AI: Challenges and Opportunities Mérouane Debbah Mathematical and Algorithmic Sciences Lab, Huawei Geneva, January 29th, 2018 1
Networks are becoming very complex 4G MBB Voice Data HD Video B2C B2H B2V WTTx 3D Video VoLTE embb urllc Smart Transportation Drone Smart Energy WTTx AR/VR mmtc Smart meter Smart manufacturing One Network 4 Services One Network Hundreds of Services 2
Wireless AI Big data and machine learning technology network management can overcome the Wireless SON network management problem when the network cannot be accurately expressed with formulas. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 3
AI in Wireless HF channel map construction E2E Performance learning Network auto configuration Slice resource management Self operation Failure detection AI management platform ASFN adjustment Policy management MBB AI Slice LTE power control PAPR non-linear DTX compensation adaptation AI NE HF/LF collaboration AI Network Comp mode selection Link Adaption RAN minimal deployment RAN AI chipset AI base station AI Data AI chipset AI service Algorithm Regression Clustering Classification Re-enforcement learning Dynamic optimization GMM/HMM Association rule mining Deep Learning RDL Transfer Learning Graphic algorithms HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 4
AI in Telecommunications is not new HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 5
AI in Telecommunications is not new HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 6
AI in Telecommunication is not new HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 7
Brain Empowered Wireless Communications HUAWEI TECHNOLOGIES CO., LTD. Huawei Proprietary - Restricted Distribution Page 8
Learning Algorithms for SON HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 9
Learning Algorithms for SON Best Response Dynamics (BRD) Fictitious Play (FP) Reinforcement Learning (RL) Joint Utility Strategy Learning (JUSTE) Trial and Error Learning (TE) Regret Matching Learning Q-Learning Multi-Arm Bandits Imitation Learning HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 10
The exploration versus exploitation dilemma HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 11
Why now? Machine-learning algorithms have progressed in recent years, especially through the development of deep learning and reinforcement-learning techniques based on neural networks. Computing capacity has become available to train larger and more complex models much faster. Graphics processing units (GPUs), originally designed to render the computer graphics in video games, have been repurposed to execute the data and algorithm crunching required for machine learning at speeds many times faster than traditional processor chips. Key Trend Emerging: Specially design chips and Hardware for Machine Learning workloads(tensor Units). Massive amounts of data that can be used to train Machine Learning models are being generated, for example through daily creation of billions of images, online click streams, voice and video, mobile locations, and sensors embedded in the Internet of Things devices. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 12
With Big Data, we can learn from OUR past experience and the Past Experience of Others HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 13
Where to compute AI? HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 14
The exploration versus exploitation dilemma HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 15
HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 16
HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 17
HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 18
On Device AI HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 19
How to speed up the learning process with local Data? Collaborative AI How, without any detailed modeling of the overall system, can AI based users learn and interact without data exchange? Collaborative AI are based on learning mechanism via imitation that help a user to select its strategy faster by exploiting its local data and the learning outcome of neighboring users. In this mechanism, instead of exchanging all the local data between users, only the outcome of their learning algorithms is exchanged. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 20
Similarity Learning for User Cell Association Neighboring devices are in the same conditions. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 21
Similarity Learning for User Cell Association How to exploit similarity between users to reduce the communication load between users and make the decision process faster? arrow SBS 1 SBS 2 HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 22
Similarity learning for User Cell Association Each user has to choose the best SBS among its options. After selection it will receive a reward based on the congestion level of the SBS and the channel gain between itself and the SBS. Each user tries to solve an optimal control problem for selecting the SBS over the time. Network is ultra-dense so we model it using mean field games. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 23
Users Reward Model Users reward function Effect of congestion on SBS: a function of the total number of users connected to SBS s Reward for user u when connected to SBS s Interference: a function of the total number of users connected to SBS s HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 24
Ergodic Control Problem Optimization problem for user u located at x: SBS selected by user u located at y at time t Distance between user u located at x and user u located at y u Gaussian kernel HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 25
Mexican Wave as a Mean Field Equilibrium HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 26
Mean-Field Game (MFG) Formulation Hamilton-Jacobi-Bellman equation: vector that represents the Fokker-Planck transition equation probability (the evolution from SBS of s to all other SBSs in S users distribution over time): probability that a user connecte to SBS j switches to SBS s at time t HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 27
Transforming MFG to MDP Solving the mean-field game depends on the form of the utility function which restricts the application domain of MFG Given that the MFG problem is difficult to solve, we transform MFG problem into a Markov decision process (MDP). For solving MDP, we use deep reinforcement learning. Users approximate the value function using adaptive linear neuron (ADALINE) neural networks. The user then trains its network using Widrow-Hoff algorithm (exploration) with the probability ε and chooses the best BS with probability 1 ε. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 28
Simulation Results We have a system with 10 SBSs in equilibrium 20 imitator users and 20 non-imitator users enter the system and start to learn the environment. Imitator users can learn the environment faster than non-imitator users. Imitator users use the experience of the existing users in the system. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 29
Simulation Results Equilibrium for the users for the imitator users. Average reward for 20 imitator users and 20 non-imitator users in the second part. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 30
Cloud AI HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 31
Wireless Caching HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 32
Caching and Machine Learning Challenges in caching systems Base stations (BSs) and devices must have knowledge of each users content request and mobility patterns in advance. Users content requests determine the contents to cache and users mobility patterns determine the users association. How to solve? Machine Learning for the prediction of content request distribution and mobility pattern of each user HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 33
Echo State Networks (ESN) ESN is a concept that describes an engineering approach for training and using recurrent neural network. Introduced by H. Jaeger [Jae01] Why use the ESNs? Easy to train (only need to train ) W n out Good at dealing with time related data Herbert Jaeger [Jae01] H. Jaeger, The echo state approach to analysing and training recurrent neural networks, technical report, 2001. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 34
Advantage of ESNs Only one hidden layer W Input layer W in and hidden layerw are generated randomly. Only need to train the output layer. W Can mimic a target out system with arbitrary accuracy. Disadvantage of ESNs Echo State Networks (ESN) The Performance of ESN depends on the input layer and hidden layer which are generated randomly. Therefore, it needs to adjust the input and hidden layer to improve ESN. HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 35
ESN for content distribution prediction Four components: Agents: BSs/devices (Each ESN predicts one user s content request distribution) Inputs: User s context including content request time, gender, occupation, age, and device type: Outputs: Content request distribution ESN Model: Input weight matrix Recurrent matrix Output weight matrix a,in W j a W j W j a,out HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 36
ESN for content distribution prediction Training: Dynamic reservoir state update (record the ESN history data): Output: Output weight matrix update HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 37
ESN for mobility pattern prediction Four components: Agents: BSs (Each ESN predicts one user s content request distribution) m t, j Inputs: Current location (ESN can find the relationship with historical locations) Output: Predicted location: ESN Model: Recurrent matrix Input weight matrix Output weight matrix in W j out W j HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 38
ESN for mobility pattern prediction Training: Dynamic reservoir state update (record the ESN history data): Output: Output weight matrix update HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 39
Algorithm HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 40
Simulation settings Real data from Youku for content simulations and use the realistic measured mobility data from the Beijing University of Posts and Telecommunications for mobility simulations. M. Chen, W. Saad, C. Yin, and M. Debbah, "Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks with Mobile Users", accepted for publication, IEEE Transactions on Wireless Communications, 2017 HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 41
Simulation results HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 42
Simulation results HUAWEI TECHNOLOGIES CO., LTD. 华为保密信息, 未经授权禁止扩散 Page 43
Thank You. Copyright 2016 Huawei Technologies Co., Ltd. All Rights Reserved. The information in this document may contain predictive statements including, without limitation, statements regarding the future financial and operating results, future product portfolio, new technology, etc. There are a number of factors that could cause actual results and developments to differ materially from those expressed or implied in the predictive statements. Therefore, such information is provided for reference purpose only and constitutes neither an offer nor an acceptance. Huawei may change the information at any time without notice.