A Data Driven Approach for Motion Planning of Autonomous Driving Under Complex Scenario

Transcription

1 A Data Driven Approach for Motion Planning of Autonomous Driving Under Complex Scenario Chenyang Xi 1, Tianyu Shi 1, Jie Chen 2 Abstract To guarantee the safe and efficient motion planning of autonomous driving under dynamic traffic environment, the autonomous vehicle should be equipped with not only the optimal but also a long-term efficient policy to deal with complex scenarios. The first challenge is that to acquire the optimal planning trajectory means to sacrifice the planning efficiency. The second challenge is that most search based planning method cannot find the desired trajectory in extreme scenario (e.g. lane change in crowd scenario). In this paper, we propose a data driven approach for motion planning to solve the above challenges. We transform the lane change mission into Mixed Integer Quadratic Problem (MIQP) with logical constraints, allowing the planning module to provide feasible, safe and comfortable actions in more complex scenario. Furthermore, we propose a hierarchical learning structure with classification layer and action learning layer to guarantee online, fast and more generalized motion planning. Our approach s performance is demonstrated in the simulated lane change scenario and compared with related planning method. I. INTRODUCTION Autonomous driving has gained much popularity in recent years, from industry to academy. To guarantee the safety and efficiency in dynamic traffic, an automated vehicle (AV) should be able to equip with optimal or near optimal maneuvers which can satisfy the above requirements. Typically, autonomous driving systems are composed of perception, decision-making, motion planning and vehicle control [1]. Motion planning of the maneuver can be a quite challenging part under the interactive traffic flow. As in a typical lane changing scenario, after generating the lane change intention, the autonomous vehicle needs to consider both the proceeding vehicle on its ego lane and surrounding vehicles on the target lane so as to generate the appropriate motions. There are quite lots of literatures about these problems. Typically, the traditional motion planning methods lie in two folds, search based methods [2, 3] and optimization based methods [4, 5]. The search-based methods generate a graph or sample representation of the planning problem, then search for the solutions among the state space. These methods can generate lots of candidate paths easily but cannot guarantee its smoothness and optimality. The optimization based method, such as Model Predictive Control (MPC), is an effective and reliable method for motion planning of autonomous vehicle due to its capacity to systematically handle input constraints and admissible states [4,5]. However, the optimization based method can be computational demanding, which cannot guarantee real-time applications in terms of solving the problems with limited runtime and fast reaction to the changing environment. The another group of researchers focus on applying deep learning methods to formalize a smooth and safe driving policy for the autonomous driving maneuvers by imitating from expert data or training in simulated environment [e.g. 6,7,8]. For example, Mehta, A. et al [8] proposed a framework involves end-to-end trainable network for imitating the expert demonstrator s driving commands. However, there are some drawbacks of the imitation learning methods. For end-to-end learning, the training data are collected from experienced human drivers. However, when there is large disturbance of the label acquired by sensors, the learning behavior may be not reliable. Furthermore, the learned behaviors may not be generalized to wide scenarios because most of the data are collected under some simplified situations with limited features. On the contrary, deep reinforcement learning can directly learn policies that map observations to controls without needing demonstrators. Pin. et al [9] proposed a reinforcement learning based approach to train the vehicle agent to learn an automated behavior under unforeseen environment. However, the policy may not be sufficient enough to implement into real driving scenario since there are large difference between simulation and real-world scenario. Integrating data driven approach and classical planning method (e.g. MPC, A*) show effectiveness for improving training performance. In previous work, data-driven approach have often been combined with classical control method to formalize a robust policy against a variety of perturbations [e.g. 2, 1]. For instance, Zhang, T. et al [1] propose to combine MPC with data driven method in the framework of guided policy search, which can successfully control the robot without knowledge of the full state. For work related to speed up training speed, De Iaco, R. et al [9] propose the method to compute a sparse lattice planner control set by learning from a representative dataset of vehicle paths, which can speed up planning time. However, it cannot guarantee the optimality. II. MOTIVATION AND CONTRIBUTION A typical lane change scenario is shown in the following Fig 1. Fig. 1. Lane change scenario construction The ego vehicle (EV) is in grey, which has an intention to change lane is assumed to consider three surrounding obstacle current vehicles (CV) is in blue. Vehicle 1 is in the ego lane (EL), Vehicle 2 and Vehicle 3 in the target lane (TL). 1. This work was done when Chenyang Xi (caralhsi@gmail.com) and Tianyu Shi (tianyu.s@outlook.com) work as intern at Momenta.AI, 2. Jie Chen is the senior research scientist at Momenta.AI, Dongsheng Plaza A, Haidian District, Beijing,18

2 Specifically, to satisfy the obstacle avoidance constrains and vehicle dynamics constraints, the ego vehicle need to determine a suitable lateral yaw rate ww eeeeee and longitudinal acceleration aa eeeeee in a given interval ττ to steer to the target lane TL. In addition to ensuring the success of lane changing, the motion planning module should also guarantee the efficiency and safety of the ego vehicle s movement (small acceleration aa and jerk JJ ). The motion planning module should also be able to satisfy the safe distance ssssssss both on target TL and ego lane EL in addition to the interval between ego vehicle EV and surrounding current vehicles CV. Some related works propose several methods for solving this kind of problem. To be specific, Rasekhipour, Y. et al [4] integrated potential functions into the objective of the model predictive path planning controller, which make it capable of tackling different obstacles while planning the optimal path. However, this kind of method cannot guarantee the fast and online planning task due to the complexity of potential field generation. Meanwhile, it is not quite easy to handle the avoidance of moving obstacles. To move on, Mixed-Integer Programing has demonstrated the effectiveness for handling moving obstacles [e.g.12]. The large numbers of linear functions of this MIQP method will decrease the approximation error, but still increase the computational time of solving the optimization problem. On the other hand, it should be noted that data driven method can be utilized to improve the computational efficiency. Sun, L. et al [7] propose a fast integrated planning and control framework via imitation learning. However, the constraint design of the circular safe buffer in their long-term model predictive control layer make it hard to handle some extreme situation (e.g. crowd lane change). Furthermore, if the previous initial states are not reliable to generate the optimal actions, their learning method cannot distinguish the unreliable initial condition, which will cause the motion planning module not adequate enough to implement in wide scenarios. In our work, we initially consider this problem as a convex optimization problem solved by iterative process. In order to obtain feasible actions, we introduced more precise rectangular safe constraints for tackling more generalized problem. The problem solving process is accelerated from two aspects, the initialization process and solution mapping process. Specifically, we define the initialization process based on polynomial lane change trajectory. This will be more feasible for this task and speed up finding optimal solutions. Furthermore, a hierarchical leaning framework is proposed to not only accelerate the problem solving process but also increase the scalability of our motion planning module. We designed the learning framework with classification layer and action generation layer. The first layer includes the feature extraction and performance classification. By learning from the abstract initial features based on the driving scenario, the classification layer can output the motion planning performance prediction based on our evaluation criteria. These predictions can help retort the good initial features for action learning, providing safe guarantees for the motion generation. Next, the second layer can generate corresponding planning actions based on the input features. Due to the generalization characteristics of the training data, we find that the proposed data driven framework can provide feasible trajectory even the planning result is unsatisfying. Our main contributions in this study are: 1) Ensure feasibility of the motion planning with changing constraints in wide scenario. 2) Provide safety estimation within data driven framework to guarantee the reliability of the motion planning 3) Improve the unsatisfied planning results via generalization of data driven framework This paper is organized as follows. In section III, the logic programming is designed for motion planning problem. In section IV, the hierarchical learning structure is proposed and the training results are presented and discussed. Section V demonstrates the effectiveness of the algorithm in simulated lane change scenario from two aspects. Finally, conclusion and future work are illustrated in section VI. III. LOGIC PROGRAMMING FOR MOTION PLANNING In this section, we consider the lane change process as the logical programming problem, which can be solved by Mixed Integer Quadratic Problem (MIQP). Integrating the changing constraints and complicated task objective into existing optimization architecture to be efficient and general will be most challenging. In this chapter, the complex dynamic constraints are tackled as logical constraints. Then, we solve the mixed integer quadratic problem. Beyond that, a rational initialization of state variables is introduced to accelerate problem solving. A. Formulation of logical constraints It is noted that the obstacle vehicles that we considered, are all accelerating constantly in the EL and the TL. In order to avoid collision with these three obstacle vehicles, we can classify the constraints. That is to say, when the EV is in the EL, it should avoid the Vehicle 1. When the EV is in the TL, Vehicle 2 and Vehicle 3 should be avoided. Since we assume that the EV cannot avoid obstacles in one lane, the selfdriving vehicle cannot exceed Vehicle 1 in EL. Similarly, the EV can only travel between the Vehicle 2 and Vehicle 3 in the TL. After considering the safety distance, the logical obstacle avoidance constraints can be expressed by the following equation: xx eeeeee xx llllllllllll ssssssss1, xx eeeeee x follow + ssssssss2, xx eeeeee xx tttttttttttt ssssssss3, iiii EEEE EEEE iiii EEEE TTTT iiii EEEE TTTT Where ssssssss1, ssssssss2, and ssssssss3 represent the safe distances between the surrounding vehicles. Furthermore, vehicles shape is considered in rectangle. Thus, to represent which lane that the ego vehicle belongs to can written as: EEEE EEEE yy eeeeee ww eeeeee 2 < ww (2) llllllll EEEE TTTT yy eeeeee + ww eeeeee 2 > ww llllllll (1)

3 Such a logical constraint problem can be transformed into a convex problem by the mixed integer programming method. As an if-then logic constraint is considered in the programming problem, mixed-integer programming is adopted to convert the continuous constraints into discrete variables. Take one of the original constraints as an example: xx eeeeee xx llllllllllll ssssssss1, iiii yy eeeeee ww eeeeee 2 < ww llllllll First, Let: z 1 = 1 yy eeeeee ww eeeeee 2 < ww (4) llllllll Where z 1 and z 2 are both binary integer variables. Then the constraint turns to be To represent the if-then relation: Then Big-M method is used to link the relationship between the binary integer variables with their corresponding equality or inequality, which is represented in these equations below: Where MM >, MM ww llllllll yy eeeeee + ww eeeeee, MM xx 2 llllllllllll ssssssss1 xx eeeeee for all possible xx eeeeee. Finally, the constraint is converted to: Which can be seen as a normal convex constraint. Other two constraints in equation 1 can be tackled similarly. Using the method described above, the highly nonlinear complex constraints are converted into simple convex constraints, which will be part of the optimization problem introduced in next section. B. Optimization problem for lane change We formulate this problem as a nonlinear programming problem. The optimization objective is selected to minimize the acceleration of the ego vehicle and the difference between the real trajectory and the goal lane. (3) z 2 = 1 xx eeeeee xx llllllllllll ssssssss1 (5) if z 1 = 1, ttheeee z 2 = 1 (6) z 1 z 2 (7) ww llllllll > yy eeeeee ww eeeeee 2 (1 z (8) 1)MM xx llllllllllll ssssssss1 > xx eeeeee (1 z 2 )MM (9) yy eeeeee > ww llllllll ww eeeeee 2 (1 z 1)MM (1) xx eeeeee > xx llllllllllll + ssssssss1 (1 z 2 )MM (11) tt (12) Cost = cc 1 aa + cc 2 JJ + cc 3 (yy yy tttttttttttt )dddd Where the a represents the acceleration of the ego vehicle, the JJ represents the jerk of the ego vehicle and (yy yy gggggggg ) represents the distance from the lateral position of the ego vehicle to the target point. The dynamic transition model that we use is the rigid model which is shown as follow: dddd dddd = vvvvvvvvvv (13) dddd dddd = vvvvvvvvvv dddd dddd = aa dddd dddd = ωω Where xx, yy, vv, θθ represent the state of the ego vehicle, i.e. the longitude, latitude, velocity and yaw angle respectively, which have the same meaning of x ego, y ego, v ego, θ ego mentioned in the last section for simplicity. aa and ωω are acceleration and yaw rate. The above function can be written in matrix form. xx = ff(xx) + BB(xx)uu (14) where xx = [xx, yy, vv, θθ] TT, uu = [aa, ωω] TT vvcccccccc ff(xx) = vvvvvvvvvv (15) BB(xx) = 1 1 (16) The dynamic model of the vehicle can be transformed into a linear system, as shown below: xx (17) yy = ff xx VV (kk) + ff xx(kk) xx xx (kk) + BBBB xx θθ = AA (kk) xx + BBBB + CC (kk) Where xx (kk) is the state obtained by kk tth (latest) iteration. cos (θθ (kk) ) VV (kk) sin (θθ (kk) ) AA (kk) = sin θθ (kk) VV (kk) cos (θθ (kk) ), BB =, 1 1 VV (kk) sin (θθ (kk) )θθ (kk) CC (k) = VV (kk) cos (θθ (kk) )θθ (kk). In the above equation, matrix BB has no superscript because in the original dynamic formulation the control term is already linear. The total horizon of the path planning process tt ff is set to be 5s, an interval the prediction module can provide for planning module. We separate the number of discretization points to be N = 5, i.e. the step size tt is.1s. Therefore, the problem can then be discretized with N+1 equally spaced

4 discretized points. (i.e., {xx,, xx NN } ), and the dynamic constraints are discretized using trapezoidal rule, which will be expressed as follows. Then the problem can be expressed in this way: mincost (18) NN+1 = cc 1 a i + cc 2 (y i yy gggggggg ) tt + cc 3 JJ ii ii=1 s. t. xx nn+11 xx nn tt = 1 2 [AAkk (xx nn+11 ) + AA kk (xx nn )] + BBBB [CCkk (xx nn+11 ) + CC kk (xx nn )], ffffff nn = 1,2 NN vv mmmmmm < vv nn < vv mmmmmm, ffffff nn = 1,2 NN + 1 aa mmmmmm < aa nn < aa mmmmmm, ffffff nn = 1,2 NN ωω mmmmmm < ωω nn < ωω mmmmmm, ffffff nn = 1,2 NN xx nn < xx llllllllllll nn safe1, iiii yy nn ww eeeeee < ww 2 llllllll, ffffff nn = 1,2 NN + 1 xx nn > xx ffffffffffff nn + safe2, iiii yy nn + ww eeeeee 2 < ww llllllll, ffffff nn = 1,2 NN + 1 xx nn > xx tttttttttttt nn safe3, iiii yy nn + ww eeeeee 2 < ww llllllll, ffffff nn = 1,2 NN + 1 x = yy = vv = vv eeeeee θ = (19) Where vv mmmmmm, vv mmmmmm, vv mmmmmm, aa mmmmmm, aa mmmmmm, ωω mmmmmm, ωω mmmmmm are the lower and upper bound of the ego vehicle s velocity, acceleration and angular acceleration respectively. ww eeeeee is the width of the ego vehicle, ww llllllll is the width of the road. Using this formulated module, when rationally initialized the optimization variables, iterative calculation is conducted until convergence. Then an initialization is utilized to accelerate the converge process in next section IV. C. Acceleration of optimizing process To guarantee the fast and exact convergence to optimal solution, we need to provide a reasonable initialization of the state variables. In our method, we use a polynomial lane change trajectory for the initialization of the state pairs (xx, yy, vv, θθ). We adopt the cubic polynomial which can combine the advantage of smooth, closed loop and continuous three order derivable. The initialization can be defined as: x(t)=aa 5 tt 5 + aa 4 tt 4 + aa 3 tt 3 + aa 2 tt 2 + aa 1 tt + aa (2) y(t)=bb 5 tt 5 + bb 4 tt 4 + bb 3 tt 3 + bb 2 tt 2 + bb 1 tt + bb (21) considering the initial state and finish state of the lane change process, the boundary conditions are listed as follows: xx() = xx, xx () = vv xx,, xx () = aa xx, yy() = yy, yy () = vv yy,, yy () = aa yy, xx tt ff = xx, xx tt ff = vv xx,ff, xx tt ff = aa xx,ff yy tt ff = yy ff, yy tt ff = vv yy,ff, yy tt ff = aa yy,ff (22) In the equation (22), xx and yy represent the current longitudinal and lateral position. vv xx, and vv yy, represent the current longitudinal and lateral velocity, aa xx, and aa yy, represent the current longitudinal and lateral acceleration. We assume that when the autonomous vehicle finished the lane change process, it will drive along the center of the objective lane and follow the leading vehicle. Therefore, when the lane change finished, yy ff = ww, vv xx,ff = vv ffffffffff, aa xx,ff = aa yy,ff =. IV. LEARNING OPTIMAL POLICY Although the complex lane change scenario can be solved with fewer iterations using the approach described in the previous chapter, the method is still limited by the solution time and cannot be used online. Moreover, for some road conditions, it can be judged immediately that the lane change mission cannot be completed. However, by calling for the planning module, it must take several iterations to return a failure result. In this case, if the planning module is called, it will occupy computing resources and space. To solve this problem, in this section, we propose a data-driven approach with a hierarchical structure to learn whether to change lanes and how to change lanes. As in most optimization based planning method, there will be some cases cannot generate feasible solution, which we define as fail. Meanwhile, there will also be some cases that may be inconsistent with the normal driving behavior, which we define as bad. The rest of the scenarios we defined is good. Thus, we divide the learning process into two layers. The first layer is defined as the classification layer, which is used to judge whether the current road condition satisfy lane changing. The second layer is defined as the action generation layer. When the first layer generates the command enabling lane changing, the second layer will be activated, giving specific instructions for the lane change. In addition, during the lane change process, collision detection is continuously performed, and when a possible collision is detected, it is necessary to take over. The learning architecture is shown in the follow: Algorithm 1 Optimal Policy Learning Process 1.Initialize task scenario 2. Classify initial states 3. If (initial states == good or bad ) 4. For episode p = 1:M do 5. { use DNN output actions; 6. Collision check; 7. If (collision check == True ) 8. { conduct actions;} 9. Else if (collision check == False ) 1. { take over;} 11. }

5 12. End for; 13. Else if (initial states == fail ) 14. { conduct vehicle following;} 15. End if A. Classification layer To distinguish the good, bad and fail cases, we design a classification layer to judge a given road condition. Any supervised learning classifier can be used in this layer. Support Vector Machine (SVM) is adopted for scenario classification. Actually, reasons to failure of planning and the bad route can be explained by two ways. Firstly, the planning module highly depend on the initialization of all states before iteration, thus an improper initial states may lead to a suboptimal result, which is irrational for a real driving mission. In our research, we assume the initialization of the discretization states are reasonable and this reason is ignored. Secondly, it is reasonable to fail to find a rational route when the obstacle vehicles at the target lane drive in too low speed or the cut-in position is too far from the ego vehicle. As the motion of the obstacle vehicles are assumed to be constantly accelerating, the later positions of obstacle vehicles are only related to their states at the start points. Then, the states of all vehicles at tt are chosen as classification features. We used three categories including good, i.e., the lane change mission success, bad, i.e., the result was solved out, but the lane change curve was not reasonable, and fail, i.e., there was no solution. 16 tracks in each category were classified, and the classification accuracy shown in the following table was finally achieved. Fig. 2. SVM classification result based on location TABLE Ⅰ. SVM ACCURACY FOR THREE KINDS OF ROUTE Good Bad Fail % % % As can be seen from the above Table Ⅰ., an accurate classification result can be obtained. We further observed this 1-dimensional classification data. As shown in Fig. 2 and 3, the blue points mean good cases (represent as -1 in classifier output table), red points mean bad cases (represent as ), and yellow points means fail cases (represent as 1). We found that when the initial distance of the target vehicle is too close (smaller than 1m) to ego vehicle, we tend to plan the result of bad, and tend to plan the result of fail when the follow vehicle locates at the front, as shown in Fig 2. Another conclusion can be drawn from Fig 3. It is easy to get a failure result when xx ffffffffffff is quite large (larger than - 4m). When the follow vehicle is in a moderate position (smaller than -4m) but the speed of ego vehicle is too high (larger than 32m/s), it is easy to appear bad. In the case, only when the speed of the ego vehicle is not high, and the position of the follow vehicle is at least 4m behind the ego vehicle, it can be easy to plan a lane changing route successfully. Fig. 3. SVM classification result based on location and velocity B. Action generation layer 1) Feature Design In most research [e.g. 7,9], the feature is likely to be designed as the distance between the ego vehicle the obstacle vehicles. However, according to our experiment, it does not perform better than simply feed the original position, velocity and acceleration into the neural network. For a specific scenario, the planning module would return a 51 4 state matrix recording x, y, v, θ of the ego vehicle for 5 discretization points and a 5 2 control matrix recording a, ω for each duration between two neighbor states. Then we choose the first 5 state groups of state accompanied with the initial state of the obstacle vehicles to formulate 5 groups of input data and 5 control groups to be the corresponding output data. 2) Network Design and Test We adopted the multi-layer neural network to train this state-action policy model. After training process, the trained action generation network can give a corresponding control command at the current state, then guide the ego vehicle to the next state. Thus step by step,

6 the ego vehicle is guided to the target lane. When compared with the MIQP solution method, we randomly sampled 4 paths and calculate the states at each discrete points using the trained network, then get a mean difference value between our network and MIQP result as in table Ⅱ. TABLE Ⅱ. NEURAL NETWORK ACCURACY Longitudinal Latitudinal Velocity Theta difference difference difference difference.266(m).55(m).188(m\s).155(rad) It can be seen that under the control of the neural network, it is possible to plan a lane change trajectory similar to the planned path of MIQP, and the difference between them is small. The most obvious advantage of neural networks is that when the trajectory is planned online, the slow iterative process is no longer needed, instead the forward propagation module is called directly. The above is to consider the existence of three obstacle vehicles. For the case where only a few obstacle vehicles exist or there are no other vehicles in the area of interest, we re-collected the data for classification. Data collection for bad and fail routes will be more difficult because the vehicle is more likely to change lane successfully in the absence of obstacle vehicles. Of the 1966 paths, there are only 163 'bad' paths and 94 'fail' paths. When we categorized these data, the bagged tree classifier achieved the best results. The accuracy is shown in the table below. TABLE Ⅲ. a vehicle2 ~U( 1, 1) v ego ~U(.9v vehicle1, 1.1 v vehicle1 ) a vehicle3 ~U( 1, 1) x vehicle1 3 vv eeeeee v vehicle1 ~U(2, 4) x vehicle2 ~U(, 5) v vehicle2 ~U(2, 4) x vehicle3 x vehicle2 3 vv cccccc3, ~U(, 1) B. Model validation in simulation In this section, we evaluate our model s performance in three distinct scenarios. We construct the simulation scenario in MATLAB 219a and run a laptop with 2.5Gz Intel Core i7. We also demonstrate the efficiency of our MIQP problem solver and the generality of the network For each scenario, we draw the historic trajectories of the ego vehicle and the surrounding vehicles. The ego vehicle is represented in red one, and the surrounding vehicles are represented in deep or light blue one and purple one. Case 1: Crowd lane changing Lane changing is a very complex but common scenario. As the simulation result in reference [7], the ego vehicle can conduct a smooth lane change in a less congested traffic scenario. However, in most cases, the ego vehicle will intend to cut in a crowded traffic (e.g. merge out). Therefore, we create Case 1, results are shown in Fig 4, the ego vehicle intends to change to the left lane because it need to exit the ramp. Meanwhile, as lots of surrounding vehicles want to merge out, the gap in the target lane is not quite big enough. As shown in Fig 4, the ego vehicle has found a strategy to conduct a safe and efficient lane change in congested situation. NEURAL NETWORK ACCURACY FOR SPARSE OBSTACLE VEHICLES Good Bad Fail % % % We believe that with more data, the classifier will perform better. A. Data collection V. SIMULATION There are three obstacles considered in our driving scenario, as shown in Fig 1. Thus, we need to randomly generate the initial state of these vehicles including the ego vehicle and surrounding vehicles. The velocity of the vehicle 1 and vehicle 2 are generated randomly between 2 to 4 km/h. The ego vehicle is assumed to follow its proceeding vehicle in a safe following distance based on the three-second-rule, which means that a vehicle should drive after its proceeding vehicle about three seconds driving. The initial states of the simulation system is shown in the follow table Ⅳ. TABLE Ⅳ Fig. 4. Crowd lane change with proposed model SIMULATION ENVIRONMENT PARAMETERS a vehicle1 ~U( 1, 1) v vehicle3 ~U(.9v vehicle2, 1.1 v vehicle2 )

7 Case 2: Generalize ability As the results shown in the second case of section V, we found that out model have ability of generalization which could output a reasonable path while the off-line MIQP method cannot give. However, the number of obstacle vehicles is limited in our model. We get a good result for 3 obstacles but a fair result for changeable sparse obstacles. In the meantime, we assume all obstacle vehicles are constantly accelerating. This assumption is limited to deal with real dynamic scenarios. In the future, we will focus on using more powerful model, such as graph network, to model the interaction between the vehicles. Fig. 5. Illustration of generalization ability To increase the travel efficiency of the ego vehicle, the ego vehicle will overtake the front vehicle if it is too slow. That is very challenging and dangerous to overtake the front vehicle on a two-way road, while there is an oncoming vehicle in the adjacent lane. In our case 2, the ego vehicle must be formalized a strategy to accelerate and steer appropriately in order to avoid collision. The simulation results showed that a single planning trajectory by optimization method in Fig 5 bad case, is not quite satisfy because it may collide with the oncoming vehicle and drive the vehicle onto the solid line. However, after learning state space by neural network, the trajectory will be optimized, meeting our needs. Remark I: To explain the generalization, we first consider the reason for why bad routes are planned. In the view of the MIQP algorithm, such a bad route is a solution makes the lowest cost while satisfying the constraint. However, since the action generation layer only retains good planning results, it learns the good paths that the vehicle can drive. Thus, for such bad cases, it gets the similar planning results as good. Although we can avoid the generation of bad paths by formulating much more complex constraints, the higher nonlinearity they take would make MIQP more difficult to converge. Further, it is more difficult for neural networks to learn such a complex optimization algorithm. VI. CONCLUSION AND FUTURE WORK In this paper, we designed the fast MIQP algorithm for solving more generalized and complexed automated vehicle motion planning problem. We introduced the logical constraint to tactic obstacle avoidance in extreme conditions (e.g. crowd lane change). Meanwhile, a hierarchical machinelearning framework was proposed to improve the solution space, learning speed, and performance. In our learning framework, a classification layer based on SVM functioned to select the good initial state for learning. Moreover, we have evaluated different state features performance to improve the learning results of the action generation layer. VII. ACKNOWLEDGMENT We are grateful for the discussion with Sen Tian, Xufei Wang, Yuhang Zhao, Xiangqi Zhang, Liumeng Wang, Fagnfei Li, Qiuru Dai from Momenta AI and the funding of Momenta AI ( REFERENCES [1] Badue, C., Guidolini, R., Vehicleneiro, R. V., Azevedo, P., Vehicledoso, V. B, Forechi, A.,... and Oliveira-Santos, T. Self- Driving Vehicles: A Survey, arxiv: , 219. [2] Shi, T., Wang, Z., Wang, S., Bao, Y., Yuan, S., Li, X., & Zhou, J. (218). Research on Control Method and Evaluation System of Ground Unmanned Vehicle Formation Transform. arxiv preprint arxiv: [3] Kuwata, Y., Teo, J., Fiore, G., Karaman, S., Frazzoli, E., & How, J. P. (29). Real-time motion planning with applications to autonomous urban driving. IEEE Transactions on Control Systems Technology, 17(5), [4] Rasekhipour, Y., Khajepour, A., Chen, S. K., & Litkouhi, B. (217). A potential field-based model predictive path-planning controller for autonomous road vehicles. IEEE Transactions on Intelligent Transportation Systems, 18(5), [5] Ji, J., Khajepour, A., Melek, W. W., & Huang, Y. (217). Path planning and tracking for vehicle collision avoidance based on model predictive control with multiconstraints. IEEE Transactions on Vehicular Technology, 66(2), [6] Shi, T., Wang, P., Chan, C. Y., & Zou, C. (219). A Data Driven Method of Optimizing Feedforward Compensator for Autonomous Vehicle. arxiv preprint arxiv: [7] Sun, L., Peng, C., Zhan, W., & Tomizuka, M. (218, September). A fast integrated planning and control framework for autonomous driving via imitation learning. In ASME 218 Dynamic Systems and Control Conference (pp. V3T37A12-V3T37A12). American Society of Mechanical Engineers. [8] Mehta, A., Subramanian, A., & Subramanian, A. (218). Learning endto-end autonomous driving using guided auxiliary supervision. arxiv preprint arxiv: [9] Wang, P., Chan, C. Y., & de La Fortelle, A. (218, June). A reinforcement learning based approach for automated lane change maneuvers. In 218 IEEE Intelligent Vehicles Symposium (IV) (pp ). IEEE [1] Zhang, T., Kahn, G., Levine, S., & Abbeel, P. (216, May). Learning deep control policies for autonomous aerial vehicles with mpc-guided policy search. In 216 IEEE international conference on robotics and automation (ICRA) (pp ). IEEE. [11] De Iaco, R., Smith, S. L., & Czarnecki, K. (219). Learning a Lattice Planner Control Set for Autonomous Vehicles. arxiv preprint arxiv: [12] Miller, C., Pek, C., & Althoff, M. (218, June). Efficient mixed-integer programming for longitudinal and lateral motion planning of autonomous vehicles. In 218 IEEE Intelligent Vehicles Symposium (IV) (pp ). IEEE.