SAE Aerospace Control & Guidance Systems Committee #97 March 1-3, 2006 AFOSR, AFRL. Georgia Tech, MIT, UCLA, Virginia Tech

Transcription

1 Systems for Aircraft SAE Aerospace Control & Guidance Systems Committee #97 March 1-3, 2006 AFOSR, AFRL Georgia Tech, MIT, UCLA, Virginia Tech controls.ae.gatech.edu/avcs Systems

2 Systems MURI Development of sound methods that utilize 2-D and 3-D imagery to Enable aerial vehicles to autonomously detect and prosecute targets in uncertain complex 3-D adversarial environments Include capabilities and approaches inspired by those found in nature Do these things without relying upon highly accurate 3-D models of the environment New strategies of Target recognition/tracking Obstacle/hazard avoidance Navigation, guidance, and control Systems March 06 2

3 The Team Anthony J. Calise, Georgia Tech Eric N. Johnson, Georgia Tech Allen Tannenbaum, Georgia Tech Patricio Vela, Georgia Tech Anthony J. Yezzi, Jr., Georgia Tech George Barbastathis, MIT Naira Hovakimyan, Virginia Tech Stefano Soatto, UCLA and many others Systems March 06 3

4 Focus Areas Air/ground target location in congested environment, including the notion of tracking/pursuing things that move, that are occasionally occluded Pursuit/evasion, See and avoid Formation or coordinated flight UAV/munition operating in an environment including obstacle/terrain/hazard avoidance The use of visual information as replacement for traditional flight control sensors Systems March 06 4

5 Tracking with Region-Based Deformotion Simultaneous segmentation, registration flow Registration using energy minimization with gradient descent Handle occlusions Systems March 06 5

6 Dynamic Bayesian Segmentation Combines Kalman filtering, dynamic Bayesian estimation, and uses dynamically adapted priors Takes advantage of mutually beneficial interaction of components Image input Warping forward of the segmentation Bayesian segmentation output Motion estimation Kalman filter update Systems March 06 6

7 Tracking in Clutter Systems March 06 7

8 Air-to-Air Vision-Based Tracking Acc Com Guidance Autopilot UAV Position Velocity Acceleration Rates IMU + Navigation Estimated: LOS Rate Range Object Size Kalman Filter Image Processing+ Computer Vision No information is communicated between aircraft, and only passive 2-D vision information is available to maintain Camera formation Camera Z X Y Camera Image Plane Image frame α : subtended angle u = (ux, uy, uz) : unit vector x y r : range Target b : target size Estimation State u, u,.,. 1 r r, b Measurement u, α Systems March 06 8

9 Estimating Dynamics of Maneuvering Target Box Target Maneuver Sinusoidal Target Maneuver } } Neural Network corrects estimation bias Systems March 06 9

10 Formation Flight Movie (adaptation on) Altitude (m) - Leader -Follower X (m) Y (m) Systems March 06 10

11 Intelligent Excitation Improves Estimator Performance r c k rˆc Disturbance Rejection Guidance Law bˆ b ˆ, d ˆ 0 ( e) sin( ωt) Intelligent Excitation V F Aircraft Camera V F Adaptive estimation r + e e Systems March 06 11

12 Long Range Surface Profilometry Object Feature Size ~ 10 cm Working Distance ~ 1 km original object Object Distance = 46 cm Object depth features 225 µm z=0 µm z=50 µm z=200 µm z=250 µm Thanks to Prof. Chee Wei Wong for the micro-turbine Systems March 06 12

13 Laboratory/Flight Testing Tools Simulation process Algorithms tested in software, real-time hardware, and then flight tested Flight tests on proven platforms, instrumented to provide precise truth data to maximize research productivity GTMax research UAV GTEdge agile maneuvering test aircraft New larger airplane Other Aircraft Systems March 06 13

14 Neural Network Adaptive Flight Control Attitude corrections External Command Outer Loop Inner Loop Online trained Neural Network Systems March 06 14

15 Georgia Tech s Yamaha RMAX: GTMax More than 280 research test flights since March 2002 Systems March 06 15

16 Agile Maneuvering Test Aircraft: GTEdge 33% scale Edge 540T Transition to zero speed using trajectory adaptive autopilot m c e 1500 height (ft) knot using autopilot West (ft) North (ft) 1000 Systems March 06 16

17 MASS Helispy Custom Small autopilot Weight lbs Duct Diameter - 11 in Autopilot Tests on GTSpy Systems March 06 17

18 AVCS Test Aircraft 4.8m Fixed-wing Wing Span: 4.8m Aircraft Length: 3.3m Large payload comp. Removable nose-cone First flight Feb 2006 Second aircraft partially complete Systems March 06 18

19 AVCS Test Aircraft 4.8m Fixed-wing First Flight February 17, 2006 Systems March 06 19

20 Vision-Only Guidance, Navigation, and Control Systems March 06 20

21 The Measurements Available measurements from a candidate window Size Rotation Horizontal pos. in camera Vertical pos. in camera Systems March 06 21

22 Hardware in the Loop for Vision Systems Actual glider with video camera and transmitter looking at the simulated camera output Playback of recorded video Systems March 06 22

23 Flight Test Results Systems March 06 23

24 Vision-Based Approach and Landing Autonomous landing at a landing zone, when the exact location, orientation, and elevation of the landing zone is unknown Systems March 06 24

25 Vision-Based Approach and Landing Approach: Landing beacons (optical targets) are placed at known relative position and elevation Vehicle flies an approach towards the beacons Vehicle relative position and attitude is estimated, with sufficient accuracy for touchdown Image processor determines beacon locations in image State of vehicle with respect to the beacons estimated Systems March 06 25

26 Simulation Results Longitudinal Velocity (ft) Lateral Velocity (ft) Vertical Velocity (ft) time (sec) time (sec) Position Estimate truth estimate truth estimate truth estimate time (sec) Roll Angle (deg) Pitch Angle (deg) Yaw Angle (deg) 5 0 truth estimate time (sec) time (sec) Attitude Estimate truth estimate truth estimate time (sec) Systems March 06 26

27 Vision-Aided Inertial Navigation Inertial Navigation System (INS) aided by 2-D vision sensor looking at a selected image target Autopilot UAV IMU + Navigation Image Processing+ Computer Vision Camera Systems March 06 27

28 Approach Vision-Aided Inertial Navigation Use assumed range (altitude), position in image, size of object in image, and aircraft state to estimate object position, size, and orientation Use subsequent measurement of object image position and size in image to update INS Utilize inertial data to maintain lock on target y Operate without GPS z Location (or any other position aiding) Last Object Location Extrapolated New Object Actual New Object Location Systems March 06 28

29 Vision-Aided INS Trials Using Existing Features (in this case a window) Special Purpose Optical Target Systems March 06 29

30 Close Approach to Building, no GPS GT 2 Systems March 06 30

31 Flight Test Results: Error Plots error = estimate GPS Max Errors: 9.6 ft (x) 8.3 ft (y) 4.0 ft (z) Systems March 06 31

32 Vision-Based Formation Flight Vision-based guidance and navigation for formation flight using a single 2D passive vision sensor without communication assuming known follower s state Leader aircraft Follower aircraft Camera Camera image Navigation System ^ Xfollower Estimator for Leader state ^ Xleader xleader Image Processor Flight Controller Guidance for Formation Flight Systems March 06 32

33 6-DOF Image-in-the-Loop Sim Results Image Processor z EKF x^ Controller Simulator Relative position Relative velocity Systems March 06 33

34 Tests on Recorded Videos Fast marching methods + target acquisition process = 10 frames/sec Video 1 Video 2 Systems March 06 34

35 Estimator Design Apply Extended Kalman Filter (EKF) to Measurement calculated by using image processor outputs Estimation state Unit vector expressed in a camera frame Measurement and Estimates leader s size : b Leader alat : leader s lateral acceleration α : subtended angle r : range Unit vector expressed in a NED frame Xc u : unit vector North Camera frame Leader s lateral acceleration In the leader s wind frame ( perpendicular to leader s velocity vector ) Yc Zc Follower Down East Systems March 06 35

36 Simulation Results Image source : recorded video in flight test Simulated camera motion : circling with a constant speed Commanded camera position : 100ft behind, 20ft below the leader Image processor outputs and EKF measurement Leader s positions in image Unit vector and subtended angle Systems March 06 36

37 Simulation Results (cont d) Position Trajectory Velocity Acceleration Systems March 06 37

38 Leader Aircraft : GTEdge Follower Aircraft : GTMax Body-Fixed Camera Flight Test Formation Flight Configuration GTEdge : circling with a constant speed ( ~600ft radius, 400ft altitude, 70ft/sec speed ) GTMax : maintain a relative distance (100ft behind, 20ft below ) GTEdge GTMax Guidance for formation flight initially is using GTEdge navigation outputs (GPS/INS) All processing onboard Systems March 06 38

39 Formation Flight Tests (GTMax/GTEdge) Systems March 06 39

40 Formation Flight Tests (GTMax/GTEdge) Systems March 06 40

41 Formation Flight Tests (GTMax/GTEdge) Systems March 06 41

42 Flight Test Results Data recorded for 70sec at 1Hz Image processor outputs and EKF measurement Leader s positions in image Unit vector and subtended angle Systems March 06 42

43 Flight Test Results (cont d) # 16 Estimation Results Position and Velocity Estimates Systems March 06 43

44 Status Currently going to close the loop on guidance Added air data system to GTEdge to allow constant airspeed flight Current efforts include more advanced vision processing, estimation, and guidance Leads into other intended scenarios: see/avoid, pursuit Systems March 06 44

45 Moving Ground Target Tracking Goal - keep target in camera field of view Interpret image Orient camera Generate trajectory Camera commanded to point at target s ground position Trajectory is generated to follow target with small attitude changes Systems March 06 45

46 Open-Loop Tracking Results Systems March 06 46

47 Closed Loop Tracking Result DARPA Heterogeneous Urban RSTA Teams (HURT) Program Victorville, CA in September 2005 GTMax automatically chasing a van through suburban neighborhood Systems March 06 47

48 Obstacle Avoidance # 02 Assumptions Known Camera State: Position and Attitude Stationary Obstacles Straight Obstacle Edges Image Processor Outputs Line Segments on a 2D Image Plane corresponding to Obstacle Edges e.g. Buildings UAV camera obstacles Estimator Outputs Obstacle Edge Estimates in a 3D Space Systems March 06 48

49 Camera Motion Vision Sensor image Estimator Design Obstacle Avoidance Database Line Estimates Data Update # 03 Image Processor Measurement Line Assignment Line Segments Estimator Line Segment Obstacle Edge expressed by two endpoint positions Line-based EKF Line Addition / Deletion XC Line Extension / Shrinkage x Image plane z1 ZC y Line Segment z2 XC1 Obstacle Edge XC2 Camera YC Systems March 06 49

50 Vision-Based Obstacle Detection: 3-D Obstacle Database Construction Current View (yellow) True Terrain (green) Terrain 3-D Estimate (blue) Systems March 06 50

51 Future Plans Continue to expand on air-to-air tracking Cluttered background Flight testing and facilitate technology transition Air-to-ground tracking (by airplanes) Leverage air-to-air work (particularly cluttered background and guidance work) Obstacle avoidance Initial emphasis on unknown fixed obstacles, ownship state known Using passive monocular 2-D sensor Requires considerable work in image processing, 3-D estimation, and guidance Using new sensor design (profilometry) Systems March 06 51

52 Expected Outcomes and Transitions New capabilities of autonomous sensing and control, enabling operations: In a clandestine/covert manner In close proximity to hazards, structures, and/or terrain In uncertain and adversarial 3-D environments Relevant flight test validation Enable more capable/reliable existing air vehicles and guided munitions Enable entirely new systems to be developed (for example, capable of operating in urban environments) Systems March 06 52