Cyber-Physical Systems: Bringing Engineering and Computer Science Together

Transcription

1 Cyber-Physical Systems: Bringing Engineering and Computer Science Together Raj Rajkumar Professor, ECE and Robotics Institute Co-Director, GM-Carnegie Mellon Autonomous Driving Collaborative Research Lab Co-Director, GM-Carnegie Mellon Vehicular IT Collaborative Research Lab

2 Outline The past and the future - a 50,000 + ft view Controlling your World from anywhere Smart Eyes Autonomous Mobile Devices Summary

3 Cyber-Physical Systems A cyber-physical system integrates computing and communication capabilities with the monitoring and/or control of entities in the physical world dependably, safely, securely, efficiently and in real-time. Long-term goal: Cyber-physical systems transform how we interact with the physical world just like the internet transformed how we interact with one another. Transcend space, yet control the physical environment. Seek scientific foundations and technologies to integrate cyber- concepts with the dynamics of physical and engineered systems.

4 Application Sectors Transportation: Automobiles, Aerospace, Rail, Maritime Vehicular networks Smart highways, trains and boats Civil Infrastructure: Bridges, Water, Sewer, Agriculture, Industrial plants, Electrical Grid, Highways, Health-care: Smart hospitals, smart ORs, tele-medicine, personal monitors Manufacturing: Flexible and efficient manufacturing Military: The side with the better robots wins 1. Thou shalt not harm humans in general and civilians in particular. 2. Thou shalt not destroy natural resources. 3. Thou shalt not destroy civilian assets (farmlands, structures, ). In general, any X by wire(less) where X is anything that is physical in nature.

5 Grand Challenges Zero automotive traffic fatalities, injuries minimized, and significantly reduced traffic congestion and delays Blackout-free electricity generation and distribution Energy-aware buildings Perpetual life assistants for busy, older or disabled people Location-independent access to world-class medicine Extreme-yield agriculture Reduce testing and integration time and costs of complex CPS systems (e.g. avionics) by one to two orders of magnitude Physical critical infrastructure that calls for preventive maintenance Self-correcting, self-certifying CPS for one-off applications

6 Technical Challenges A Science of Cyber-Physical Systems Theory of Uncertainty Composability of physical and cyber aspects Dependability, Safety, Security and Privacy in Real-Time Physics of wear and tear Automatic run-time optimization Model-based Design Verification and Validation Cyber-Physical Systems Engineering Componentization to Systems of Systems Specialization for Domains Testbeds for Cyber-Physical Systems Cost minimization with dependable operation Robust shutdown mechanisms and policies

7 How does a CPS-Centric Centric Future look? Transcend space Actuate things even if you are not physically there Even if you are not doing it yourself Eyes and ears everywhere For anomaly monitoring and pro-active response Safety,disasters, personal angel Higher standards of living Greater comfort and convenience

8 Transcending Space and Controlling Your World

9 Points of Control Electricity consumption (US: $347B per year) Plug-ins: every car is a sink and a source Heating and cooling Where do the big bills come from? Security Smart anomaly notification Water usage Save natural resources Gas consumption Save $ All these sub-systems are connected/integrated. Everyusage point is monitored and controlled. Global coordination is possible. Fine-grained information for optimization.

10 Energy Monitoring and Control Sensing True power, apparent power, power factor RMS current, RMS voltage Monitor power features to forecast device failure Control On/Off control of outlet Solid State Relay can provide PWM dimming Single and dual outlet versions Communication FireFly sensor node Low-Power Runs directly from outlet Idle operation at less than 0.2 Watts

11

12 Energy Profiles & Savings Some questions: What do distributed sensor readings mean? Can we predict dynamics and (say) heat leakage? How/when do we control? Can we apply budget limits? How do we synthesize locations and controls? What global properties can be proved? Can peak consumption be reduced?

13 FireFly Hardware Family FireFly sensor nodes Time Synchronization Smart Camera Nodes Line Voltage Control Badge

14 Nano-RK

15 Rate-Harmonizing Scheduling (RHS) Maximize Sleep-time of processors given {Sleep, Idle, Active} modes of operation Power state Power (mw) Upward Transition Time Active 30 mw n/a Idle 6 mw 6 µs Sleep 5 µw 10 ms * ATmega1281 running at 8Mhz at 3.3 volts

16 Energy-Saving RHS (ES-RHS) Normal RMS RMS with tasks being serviced only at specific periodic boundaries RHS with a highpriority sleep task C sleep, T sleep = T 1

17 RHS Properties A task set is feasible under RHS if A task set is feasible under ES-RHS if: Every idle duration in an ES-RHS schedule can be used to put the resource into a deep-sleep mode without any time penalty.

18 How do we know we can scale?

19 Campus-wide infrastructure for sensing and control Goals Ubiquitous large-scale monitoring and control Easy to manage, configure & use Scalable and extensible Secure and private Evolves Sensor Andrew

20 Sensor Andrew Devices BACnet The Energy Detective Hobo Sensors Enersure Hi-Fidelity Energy Panel Sensor FireFly Gateway JigaWatt What sup Pro? FireFly Sensors

21 Sensor Andrew Layers Middleware/ Central Hub Campus Network Gateway Gateway IP Camera User/ Application Layer Sensor Layer IP Camera IP Camera Gateway Layer

22 Sensor Andrew Services Data Viewing Map Server Agent Agent Server Agent Server Web Services Gateway Mobile Device Sensors Domain Data Handler Server Gateway Sensors Notification Agent Agent Registration

23 Sensing and Actuation Adapter Adapter SOX lib Gateway Actuator Sensor Over XMPP (SOX) SOX lib Subscribe / Publish Historical Data Agent 4 Database Historical Data Agent 5 Web Services Sensor Gateway Publish Registry Meta-Sensor Agent 1 Domain Data Handler Discovery Agent 3 Subscribe Event Nodes Agent 2

24 Communications Standard Messaging Format Extensible Message Types Point-to-Point and Multicast Messaging Support for Data Tracking and/or Event Logging Security, Privacy, Access Control Internet-Scale Performance Namespace management with servers acting as gateways <DeviceInstallation id="0x000003a2" type="firefly" timestamp=" t10:23:00"> <TransducerInstallation name="light" id="0001" > <TransducerValue value="100"/> <TransducerDescription Manufacturer="Advanced Photonics Inc" PartNumber="PDV-P9003" MinValue="0.00" MaxValue="1024.0"/> </TranducerInstallation> <TransducerInstallation name="temperature" id="0002" > <TransducerValue value="57.6" timestamp=" T10:23:00"/> <TransducerValue value="58.1" timestamp=" T10:21:30"/> </TranducerInstallation> <TransducerInstallation name="voltage" id="0004" >

25 FireFly Hardware D Cell Node AA Cell Node Development Node Gateway Node

26 Wean Hall Deployment

27 Roberts Engineering Hall

28 CIC Deployment

29 Control Panel

30 Smart Eyes and Ears Everywhere

31 Eyes Everywhere: Video Surveillance Video surveillance is becoming widespread. Public infrastructure monitoring Campus and building security Theft prevention Intruder alerts Human operators do most (if not all) recognition of objects on the screen. System designed for archival and labor-intensive search. Timestamps are often the only automatic search keys. Visual information is not always sufficient.

32 Embedded Vision Processor Objective Develop a Low Cost Embedded Vision Processor Exports High Level information to another processing board Where is the brightest spot on the image? What part of the image changed? Current Capabilities 320 x 240 RGB CMOS Camera 60 Mhz ARM7 processor 50 Frames Per Second Color Tracking Applications Sensor Networks Intelligently select when to send images Down Sample and Compress Images for Low Bandwidth Transmission Mobile Robotics Horizon Tracking (UAV)

33 Integrated Video/Sensing 60 MHz ARM7 Camera Board FireFly Sensor Node SD FLASH Memory Slot Battery Pack

34 FireFly DSPcam VGA Color Camera 640x480 images at 30Hz frame rate DMA Frame Capture Analog Devices Blackfin DSP Media Processor 600MHz, 32MB RAM and 4MB Flash Networking 10/100Mbps Ethernet b/g FireFly Interface µclinux Operating System Supports most Linux utilities GNU Development Environment Open-source with GCC compiler DMA CMOS Camera Driver

35 Real-Time Video + Sensor Data View from FireFly DSPcam A (through ) View from FireFly DSPcam B (through ) Sensor Data from FireFly sensor nodes (over ) JPEG compression performed locally on FireFly DSPcam s

36 OmniEye Capture images and sensor data, detecting events and anamolies of interest Correlate motions and events across multiple fields of view Transmit annotated data and any associated clips Notify operator(s) Archive for future

37 How Can Eyes and Ears Help? Mobile Gateway Drill Hole Mobile Node Infrastructure Node Hazardous Obstruction

38 NIOSH Research Coal Mine near Pittsburgh NIOSH: National Institute for Occupational Safety and Health

39 Improve Standards of Living Building Vehicles that do not Crash by making Autonomous Mobile Platforms Real

40 Society and Automobiles More than 1 million people die every year in automotive accidents globally Largest global killer of aged Tens of millions of injuries Global annual cost of road injuries in medical care, disability and property damage is $518 billion. Loss of independence and self-esteem of senior and disabled citizens Traffic delays are expensive. The average US driver spends a week stuck in traffic per year. In the EU, 80 billion euros wasted per year due to traffic congestion.

41 The DARPA Urban Challenge Urban Autonomous Vehicle Race November 3rd, miles in less than 6 hours Multiple autonomous vehicles competing with each other on a simulated urban course Other traffic vehicles driven by humans Qualifiers determines the finalists 30 mph max speed limit

42 Urban Driving Skills Safe and Defensive Driving California DMV rules of the road Negotiate Stationary and Moving Vehicles Blocked Roadways Intersections and Roundabouts Capable of turns, stops, passing, merging, following Reasoning about traffic Roads included curves, paved/unpaved roads Parking (in unstructured environments) Progressing safely in the face of adversity

43

44 Urban Challenge Excerpts Improvising The Course Passing

45 A Close Call

46 > 3200km of testing at 4 different sites across US Race Performance 54.3 miles Avg. Speed: 14.0 mph Avg. Moving Speed:14.7 mph Post-Race Simulation (with no traffic) 52.8 miles Avg. Speed: 15.2 mph Avg. Moving Speed: 16.3 mph 2 14mph 9 min Imperfections

47 Sensors

48 Vehicle Tracking Raw Data RAW Data (13 Sensors)

49 Vehicle Tracking Raw Data Feature Extraction

50 Vehicle Tracking Raw Data Feature Extraction Measurement Validation

51 Vehicle Tracking Raw Data Feature Extraction Measurement Validation Data Association Object known from previous cycle (vehicle)`

52 Vehicle Tracking Raw Data Feature Extraction Measurement Validation Data Association Proposals & Observation Sensor proposes alternative to prediction

53 Vehicle Tracking Raw Data Feature Extraction Measurement Validation Data Association Proposals & Observation Model Voting Estimation Statistics

54 Vehicle Tracking Raw Data Feature Extraction Measurement Validation Data Association Proposals & Observation Model Voting Estimation Statistics Prediction

55 Challenges Exogenous: The complexity & uncertainty of the real world Weather, lighting, and road conditions; construction; accidents; obsolete information, loss of GPS. Endogenous: Online and safe recovery from failures of sensors, actuators, computing or communications. Sensors Calibration, wear and tear, failures. Occasional loss of GPS Inter-acting: Vehicular Networks communicate securely and coordinate carefully Consumer Acceptance Reliability, cost and maintenance Legal implications Incremental deployment

56 Intermediate Milestones Active safety and convenience features Pedestrian, child, bicyclist, animal warnings Virtual Valet Highway Chauffeur Traffic Jam Chauffeur Dependable/safe embedded computing and communications

57 Autonomy Summary Fully autonomous driving is within our grasp. Multiple intermediate active safety and convenience features will yield significant benefits and justify investments. Cooperating vehicles will also play a major role in defense applications. Autonomous capabilities are embedded into household robots, hospital robots, assisted-living robots, factory equipment,

58 Summary of What s Coming A Science of Cyber-Physical Systems Dealing with uncertainty Abstractions, models and analyses for cyber and physical components Composability Para-functional attributes for timeliness, reliability, security, privacy and safety Leap-Ahead Applications/Systems Capabilities that the technical people can agree with Accomplishments that the public can relate to Practical Considerations Scalability, addressability, inter-operability. The future is closer than one might think