UAV-Enabled Wilderness Search & Rescue (WiSeR( WiSeR) Michael A. Goodrich Computer Science Dept. Brigham Young University

Transcription

1 UAV-Enabled Wilderness Search & Rescue (WiSeR( WiSeR) Michael A. Goodrich Computer Science Dept. Brigham Young University

2 Thanks Utah County Search & Rescue Ron Zeeman Faculty Mike Goodrich CS Bryan Morse CS Tim McLain ME Funding Sources NSF BYU MEG Program DARPA Students Damon Gerhardt Nathan Rasmusssen Brett Millar Joe Jackson Joseph Cooper Brad Huber

3 The Problem Find and assist lost people Forests Deserts Waterways Etc. Thousands of hours Hundreds of thousands of $ Volunteer teams Trained searchers Not trained pilots

4 UAV-Enabled WiSeR: : Constraints Speed For each hour Search radius ++ 3km Search area: (radius) 2 Cost Equipment: $5k - $10k Hours Organization New technical search team

5 Field Test Lessons: Finding Annie and Shrek Two field tests March 2005 July 2006 October 14, Hardware Video Process HAG Autonomy Interface Offline Interface 2 Misc Gimbal Participant feedback Debriefing as a group Ranking priorities as individuals

6 Gimbal Presentation Agenda Presentation Agenda Video Process HAG Autonomy Interface Offline Interface 2 Misc Hardware

7 Hardware and Control Algorithms McLain and the MAGICC

8 Hand Launch

9 Target Tracking and Localization

10 Autonomous Landing on Target

11 Gimbal Presentation Agenda Presentation Agenda Video Process HAG Autonomy Interface Offline Interface 2 Misc Hardware

12 Stabilizing Video (Gerhardt, Morse Rasmussen) UAVs are in the air Narrow field of view UAVs travel fast Short time on target UAVs are small Jittery video Which way is North?

13 Solution #1: Mosaic Payoffs: Increases view time Reduces rotation Shortcomings: Jitter when frame nears edge Mosaic not perfect- -can mislead user. Methods: Filter point correspondence between frames Estimate translation & rotation btw frames Paint current frame over previous frame

14 Which way is North? Mosaic

15 Pseudo-code Identify and pair common points in successive frames. Fit a homography (3x3 matrix) which maps previous image points to current Find best H using a RANSAC Approach Use H to filter out bad point pairs Estimate rotation R(t), center of rotation C(t), and translation T(t) ) using these point pairs Use R(t), Rc(t), and T(t) ) to calculate a relative (previous to current) motion vector, Vr(t) Using Vr(t) ) and Vc(t-1), calculate a cumulative (first to current) motion vector, Vc(t) Compute a cumulative rotation Rc(t) ) matrix using R(t) ) and C(t) and Rc(t-1) Compute a smoothed transformation using Vt(t) ) and Rc(t), Vc(t) Transform and paint the current frame onto the larger viewing mosaic frame using Vc(t).

16 Solution #2: Stabilize Payoffs: Reduces jitter Improves orientation Shortcomings: Short time on target Methods: Find point correspondences Estimate translation & rotation btw frames Smooth cumulative motion Paint on larger viewing frame using difference btw cumulative motion and smoothed motion

17 Can you track an object? Stabilized Video

18 Pseudo-code Estimate rotation R(t), center of rotation C(t), and translation T(t) ) using these point pairs Use R(t), Rc(t), and T(t) ) to calculate a relative (previous to current) motion vector, Vr(t) Using Vr(t) ) and Vc(t-1), calculate a cumulative (first to current) motion vector, Vc(t) Compute a smoothed motion vector, Vs(t) Using an N sized history of Vc's,, [Vc(t[ Vc(t-N), Vc(t)] Compute a degree-n bezier mid-point, Vs(t) Compute current transformation vector using Vc(t) - Vs(t), Vt(t) Compute a cumulative rotation Rc(t) ) matrix using R(t) ) and C(t) and Rc(t-1) Compute a smoothed transformation using Vt(t) ) and Rc(t), S(t) Transform and paint the current frame in the larger viewing frame using S(t).

19 Stabilized Mosaics

20 Gimbal Presentation Agenda Presentation Agenda Video Process HAG Autonomy Interface Offline Interface 2 Misc Hardware

21 Height Above Ground Field test two incident: We found a possible hit in the video. The operator selected two waypoints and commanded the UAV to fly between them. The operator then watched the video. The two waypoints were on opposite sides of a ridge. The UAV crashed Solutions HAG sensors Terrain elevation data open loop HAG User interface

22 HAG User Interface

23 Gimbal Presentation Agenda Presentation Agenda Video Process HAG Autonomy Interface Offline Interface 2 Misc Hardware

24 WiSeR Processes

25 WiSeR Search Stages

26 WiSeR Search Strategies Hasty search Goal: walk right to them Methods: tracking, dogs Constraining search Goal: limit search area Methods: point last seen, crossings Priority search Goal: focus search on most likely area Methods: collectors, likely behaviors Exhaustive search Goal: complete coverage Methods: combing

27 Automated Search Strategies Model victim movement Flat terrain Hilly terrain Trails, vegetation, water Model detection rates Plan optimal search paths

28 Optimal Spiral Camera aim point spiral UAV circle via Hopf equations

29 Gimbal Presentation Agenda Presentation Agenda Video Process HAG Autonomy Interface Offline Interface 2 Misc Hardware

30 Aviation-based GUI

31 Data Layering

32 Context, Video, Control

33 Gimbal Presentation Agenda Presentation Agenda Hardware Video Process HAG Autonomy Interface Offline Interface 2 Misc Validation and Future Work

34 Camera angle UAV height Coverage Stability Mosaics Heuristics Prioritized search Frames of reference Future Work 1: Quantify Usefulness

35 Future Work 2: Better Information Display Enhanced stabilization/mosaicing for offline viewing Construction of large-scale mosaics Use of reference imagery Use satellite or other images to give context to live video Integration with terrain models Indexed video Show me all the other parts of this video that see this point Geographically based fast-forward forward / rewind Annotated video Place a marker with notes at a point in a video frame Displayed when you (or others) when seen again

36 Future Directions 3: Enhancement and ATR Enhance the unusual One of these things is not like the others Learn from video and highlight unusual content Change detection Indexed video allows users to rapidly jump through the video to spot changes Can also provide automated assistance Probabilistic models for visibility Map and display detection likelihood at each point based on visibility, distance, speed, etc. Use to plan flight paths or coordinate ground-based search

37 Victims tend to Future Work 4: Victim Models Go down hill Follow paths and drainages Panic Go in random directions Follow ridges MATLAB concept