Leveraging AI for Self-Driving Cars at GM Efrat Rosenman, Ph.D. Head of Cognitive Driving Group General Motors Advanced Technical Center, Israel
Agenda The vision From ADAS (Advance Driving Assistance Systems) to AV (Autonomous Vehicles) AI for Self-Driving cars ADAS, AV and in-between Summary 10/19/2017 General Motors 2
The Vision Mobility one of the most significant revolutions of modern times Self-driving cars will take mobility to a completely new phase Zero Crashes, Zero Emissions, Zero Congestion (Mary Barra, GM CEO)? 10/19/2017 General Motors 3
The Vision Increase Safety Increase Productivity Increase Mobility: anywhere, anytime Increase Car Sharing & Reduce Road Capacity and Parking needs 10/19/2017 General Motors 4
From ADAS to AV L5:Full automation Level 4: High automation Level 3: Conditional automation Level 2: Partial automation Level 1: Driver assistance Level 0: Driver in full control Anywhere, anytime Fully autonomous specific scenarios Highway driving (driver takes control with notice) Traffic jam assist Cruise control, lane position Info, warnings 10/19/2017 General Motors 5
From ADAS to AV Will incremental steps get us to the top of this pyramid? 10/19/2017 General Motors 6
Components of self driving cars Sensing Mapping Perception Decision Making Control 10/19/2017 General Motors 7
Components of self driving cars AI AGENT serves as the brain of the car Perception Decision Making Control 10/19/2017 General Motors 8
AI for Self-Driving Cars 9
AI in Perception Unsupervised learning Finding structure in point clouds Feature learning Supervised learning Object detection 2D object recognition (Classification) 3D scene understanding and modeling (3D objects pose) Semantic segmentation (boundaries of objects, free space) 10/19/2017 General Motors 11
AI in Perception - E2E trend Classification: Pixels Key Points SIFT features Model Labels Scene understanding: Pixels Segmentation Object Contextual detection relations Scene description Perception: Sensors 2D object detection Depth estimation Pose estimation 3D World state 10/19/2017 General Motors 12
AI in Perception - E2E trend Classification: DNN Pixels Key Points SIFT features Model Labels Scene understanding: DNN Pixels Segmentation Object Contextual detection relations Scene description Perception: Sensors 2D object detection DNN Depth estimation Pose estimation 3D World state 10/19/2017 General Motors 13
Towards E2E: Sensors Fusion Low Level: raw data combined in input stage High Level: tailored hierarchy between sensors All sensors contribute Enables learning of complex dependencies optimally Sparse Vs. dense sensors Larger models, harder to learn Utilizes domain knowledge Model is explainable Based on tailored rules Suboptimal performance 10/19/2017 General Motors 14
Towards E2E: Multi-Task Learning Most our outputs are inter related Objects, free space, lanes, etc. Cross regularization allows reaching a better local minima TPT Major parts of the Deep Net are used for multiple tasks Data Efficiency Mask R-CNN Facebook AI Research (FAIR); Apr 2017 10/19/2017 General Motors 15
What about data? 16
Automatic Data Annotation Data is the key contributor to perception accuracy With no visible saturation How can we create annotated data Manual annotation Expensive and inaccurate Automatically Revisiting Unreasonable Effectiveness of Data in Deep Learning Era, Google 2017 10/19/2017 General Motors 17
Automatic Data Annotation Technology High end sensors (Lidar, IMU, etc.) High accuracy detectors (on behalf of computation time) 10/19/2017 General Motors 18
Example AGT for StixelNet StixelNet - Monocular obstacle detection Based on stixel representation Identify road free space Ground truthing is based on Lidar [Badino, Franke, Pfeiffer 2009] Compact, local representation Dan Levi, Noa Garnett, Ethan Fetaya. StixelNet : A Deep Convolutional Network for Obstacle Detection and Road Segmentation. In BMVC 2015. Lidar (Velodyne HDL32) is used to identify obstacle on each stixel in the image 10/19/2017 General Motors 19
Is Perception solved? Challenge of Cost Sensors Mapping Computation Challenge of false positive & false negative Data uncertainty (noise) Model uncertainty (confidence) Label: Cyclist RGB: Pedestrian (0.56) 10/19/2017 General Motors 20
Decision Making Perception Decision Making Control 10/19/2017 General Motors 21
Learning Decision Making Decision Making cannot learn from static examples Need interactive domain - > Reinforcement Learning (RL) RL has seen some major successes in the recent years: Atari [Google Deepmind] source: nbcnews Go [Google deepmind] source: uk business insider Autonomous Helicopter Poker Flight [Bowling et al] source: wikipedia [Ng et al] source: ai.stanford.edu 10/19/2017 General Motors 22
RL challenges in Self-Driving agents Learn to act in a very high dimensional space Plan sequences of driving actions Predict long term behaviors of other road users Few sec Complicated situations Negotiate with other road user Guarantee safety 10/19/2017 General Motors 23
Simulation Advanced simulations are required Multi-agent Various conditions Focus on interesting miles Drive billions of virtual miles (fuzzing) Waymo simulation: https://www.engadget.com/2017/09/11/waymo-selfdriving-car-simulator-intersection/ Any system that works for self driving cars will be a combination of more than 99 percent simulation.. plus some on-road testing. [Huei Peng director of Mcity, the University of Michigan s autonomous- and connected- vehicle lab] 10/19/2017 General Motors 24
Safety Guarantees - From ADAS to AV Will incremental steps get us to the top of this pyramid? The technological heart is different in kind 10/19/2017
What s the difference? For ADAS Safety guarantee is based on the driver For autonomous Safety guarantee should come from the system itself 10/19/2017 General Motors 26
Example: Highway Driving in Super Cruise The 2018 Cadillac CT6 will feature Super Cruise - a hands-free driving technology for the highway It includes an Exclusive driver attention system to support safe operation 10/19/2017 General Motors 27
Safe Driving for level 4/5 System should handle 100% of the cases Redundancy requires at all levels Sensing Algorithm Computing Control Fallback strategies Guarantee of Safety is a must to the acceptance of AV Statistical data-driven approach [miles-per-interrupts] requires driving billions of miles to validate an agent Should be repeated with every SW version Need safety constrains (rule-based/model-based) 10/19/2017 General Motors 28
Summary Advances in AI are key to success of selfdriving cars AI-based features can bring ADAS to a new level in terms of accidence avoidance, productivity gain and saving in human lives Level 4/5 AV should be a parallel effort focus on redundancy and safety constrains 10/19/2017 General Motors 29
GM Advanced Technical Center in Israel (ATCI)
Thank you 10/19/2017 General Motors 31