Smart Cars: An Ideal Applications Platform for Robotics and Vision Technologies

Transcription

1 Smart Cars: An Ideal Applications Platform for Robotics and Vision Technologies Alexander Zelinsky CSIRO ICT Centre, Canberra, Australia

2 Motivation: Motor Car Accidents Car accidents are horrific. WHO estimates that every year on roads 750,000+ people are killed 25+ million of people are injured Road safety success stories: Set belts. Speed cameras Alcohol breath testing Road safety hard cases: Fatigue (worst accidents) 78% of all accidents involve inattention (100 car survey, Neale et.al. 2005). Alcohol Fatigue.and the inattention problem will become more serious... We are beginning to full exploit the IT revolution!

3 Modern technologies can Distract Awash with new in-vehicle technologies. Mobile devices: phones, organizers, DVD players, ipods etc. Dangerous driver aids : Some route guidance, web browsers, gadgets etc. There is little doubt recent in-vehicle devices increase crash risk (Stutts et.al. 2001). Mobile phone users up to 4 times more likely to crash. Hands-free no safer (Redelmeiser et.al.1997).

4 Technology to Assist Driving Driver Assistance Systems (DAS) Combine the strengths of machines & humans and mitigate their short comings. Intuitive, Override-able and Unobtrusive. Integrated think of power steering Problem: 90% of all accidents are caused by human error. Who hasn t had a near miss while driving? For Driver Assistance Systems to be effective: Must keep Driver engaged. Must consider a whole system approach to road safety. The driver and vehicle together in context of road environment. Solution: Directly monitoring driver, vehicle and road scene. An automated co pilot!

5 An Automated Co-Pilot Co-pilot: Do the tedious jobs. Share cognitive workload. Warn of unexpected threat. Validate Pilot s decisions...can take control in emergency. clippit for cars? TREV: Transport Research Experimental Vehicle Driver assistance context: e.g. ACC, lane keeping. e.g. Workload managers. e.g. Blind spot, missed sign warning If important raise alarm...precrash / autonomy.

6 Safety technologies that we need Lane Departure Road Sign Recognition Road Hazard Detection Pedestrian Detection Obstacle Detection Vehicle Detection Side View Monitor Occupant Sensor Adjacent Lane Monitor Blind-spot Sensor Parking Sensor Vehicle Detection Driver State Sensor Head Pose Gaze Direction Eye Closure Blink Detection Active Safety Systems Smart Airbags Collision Avoidance (Front, Side, Rear) Pedestrian Collision Lane Keeping Lane Changing Intelligent ACC Advanced Driver Assistance Systems Night Vision HUD Distraction / Inattention Mitigation Fatigue Detection Workload Management Driver Identification Comfort & Parking Assistance

7 Monitoring driver, vehicle & road Driver Gaze Driver observation monitoring Environment Pedals Steering Driver action monitoring Driver Assistance System Vehicle dynamics Vehicle

8 DAS Components Linking road scene state with vehicle and driver state we can create new DAS applications. Driver Assistance Systems (DAS) needs to use a distributed multi-functional approach. Context is created by where and what the driver is looking.

9 DAS Components DAS Logic: - pedestrians - obstacles - lane departure -signs - inattention - monotony Driver State Vehicle state Pedestrian detector Obstacle detector Lane tracking Sign reading Inattention detector Monotony detector

10 Head pose & Eye gaze tracking Seeing Machines: Canberra based ANU start-up Founded July 2000 IPO Dec Employees Revenues > $4m pa FaceLAB: Head Pose measurement - translation: ± 1mm rotation: ±1 Eye Gaze measurement - direction: ±3 Blink detection. Saccade detection Full 3D modelling Integrated with a scene camera ANU 1998 FaceLAB v4

11 Multi-cue Lane Tracking Adaptive cue fusion Particle filter state estimation Road scene Video input Vehicle State Speed, heading P ( h e) = P( e h) P ( h) P ( e) Apply generic lane model Compute cue probabilities Cue computation Edges, lane markings and road colour

12 Road Sign Detection Detection: Fast Symmetry Transform - radial Used for eye detection developed in facelab Classification: Image cropping & scaling Online image enhancement super resolution Image correlation - ZMCC

13 Pedestrian Detection Detection: Reconstruct scene in 3D: - Stereo Vision A 3D approach - Ground to the problem plane detection - 3D depth map construction - Look for rectangular objects - height [1m - 2m ] Classification: Use an implicit shape based classification approach Support Vector Machines, as suggested by Poggio et al. Temporal information improves results 85.3% Detection rates 0.4% False detection rates At ranges up to 20 metres

14 Multi-task Co-pilot

15 Visual Monotony Detection MPEG compression ratio is used as a metric of visual monotony. Correlates well with human judged monotony. Augment this function with lane tracking for ambiguous cases (fog, gravel roads). Validated on 1600km trip. MPEG size Internal factors Time of day Sleepiness Time on task Stimulants Monotony vs MPEG gravel road Fatigue model [Thiffault 03] Alertness Vigilance External factors Task complexity Task monotony: - task variance - Distractions Driving performance Monotony detector

16 Gaze and Road-Scene Correlation Correlating Gaze with Road Scene: not trivial. object stereo camera problem. unknown object depth. camera y z y obj 40 Unknown object depth: x obj bounded disparity along epipolar line. use upper bound on disparity in error margin: upper bound (±2 ) x θ φ z gaze eye z sign facelab precision (±1 ) FOV of fovea (±2.6 ) y gaze x gaze Find difference between gaze and sign angle. IF (difference > ) THEN sign not seen.

17 Gaze and Road-Scene Correlation Object recognized Object missed Verification: 3 distances, 8 objects, 10 trials Prove that driver saw the object. (impossible!) Prove driver would not have recognized the object.

18 Integrated Systems

19 Driver Monitoring Comparison between the driver s gaze and the yaw of the vehicle

20 Conclusions Mobile technologies are flooding into cars. We must only adopt technologies that make driving safer. We have demonstrated driver assistive : Modelled on a human used in airlines. Able to monitor the driver s observations as well as actions. Enabling: Redundant / repetitive warnings to be suppressed. Important warnings to be highlighted. Warnings acknowledged at a glance. OUR GOAL To combine the tireless vigilance of machines with the flexibility and authority of the human driver.

21 Autonomous Driving!, June 2004

22 Acknowledgments Luke Fletcher Andrew Dankers Luke Cole Gareth Loy Leanne Matuszyk Harley Truong Sebastien Rougeaux Jason Chen Yoshio Matsumoto Jochen Heinzmann Rochelle O Hagan Tim Edwards Nick Apostoloff James Ashton Rowel Artienza Simon Thompson Peter Brown Lars Petersson