The path towards Autonomous Driving Dr. Martin Duncan AEIT Seminario Catania, 17th November
2 Introduction Cameras and the path towards autonomous driving Need of supplementary sensors Conclusions
The Rapidly Transforming Car 3 The automobile is being transformed by connectivity and technology improving safety, enhancing the driver experience and lowering the environmental impact Safer Assisted driving, autonomous driving Enhanced vision Precise positioning Active safety Adaptive lighting, auto braking More Connected Vehicle to vehicle, vehicle to infrastructure communication Smartphone integration Enhanced telematics, insurance box Data and video streaming Cyber security Greener Vehicle electrification Efficient engine management Eco Navigation Efficient LED lighting
Semiconductors are Driving Change in the automotive industry 4 Safer More Connected Greener Car Digitalization Multicore microprocessors Sensor fusion Video processing Data streaming Infotainment processors Radio Frequency transmitters and receivers Analog and Power Technologies High integration smart power Power drivers High energy motor controllers Battery managers Video cameras Sensors
Safety Critical Applications Leopard: 2 FIT (hard errors) + 1200 FIT (soft errors) SPC5x ASILD HW Platform RAW FIT Safety Measures dangerous FIT <<1 FIT (0.x) Devices must be defined to meet ASILD!! Other Compone nts system level safety critical function 10 FIT Probability of violation of safety goals Example: Leopard ASIL Random hardware failures target values D < 10-8 h -1 10 FIT C < 10-7 h -1 100 FIT B < 10-7 h -1 100 FIT
Networks must be protected against security attacks 6 Vehicle Security Threats Security means protecting ECU, sensors Functions that require multi-ecu interactions and data exchange Protecting data in/out of vehicular systems Protecting privacy of personal information Integrating safety, security, and usability goals Dealing with the full lifecycle of vehicular and transportation systems Potential attack entry points Wireless Key, TPMS, V2X Tx/Rx ECUs (ADAS, Lighting, Engine, Transmission, Steering, Braking, Access, Airbag) Smartphone, Bluetooth, USB, OBD, Remote Link
ADAS Driving Forces 7 ADAS adoption is being driven by safety standards and the move to automated driving Standards Product The development of New Car Assessment Programmes help drive the growth of ADAS Consumers consider the star ratings of the worldwide NCAP organisations Constructors move from providing ADAS on all cars and not just premium models Automated Product Driving New entrants to the traditional vehicle manufacturing market target assisted / autonomous driving Consumers expectation rising that autonomous driving will make cars safer Advantages for new mobility models and fleet management
Crash Avoidance Technologies & Effectiveness 8 Automatic brake Lane departure Blind spot Headlight % Incidence Automatic brake Lane departure Blind spot Headlight Rear end (29%) Crossing (24%) Off road (19%) Lane (12%) Animal 6%) Wrong sense (2%) Reversing (2%) Ped/cyclist (2%) Source: NHTSA
Global NCAP Requirements 9 2014 2015 2016 2017 2018 2019 2020 LDWAEB PEDLKA LDWAEBPED LDWAEB PED LKA LDW AEB PED Potential LDWAEBPED LDW AEB PED LKA LDW = Lane Departure Warning AEB = Autonomous Emergency Braking PED = Pedestrian Detection LKA = Lane Keeping Assist Source : Regional NCAP agencies
ADAS Adoption rates in U.S. Market 10 25 % Fit Rate 20 15 10 5 0 Blind Spot ACC LDW MY13 MY14 MY15 ACC = Automatic Cruise Control LDW = Lane Departure Warning Source : Wards Auto
Benefits of Collision Avoidance 11 0 % Change with same vehicle with technology fitted FCW & LDW AEB AEB & LDW ACC & FCW -5-10 -15-20 -25-30 -35-40 Collisions Property costs Injury costs LDW = Lane Departure Warning, AEB = Autonomous Emergency Braking, FCW = Forward Collision Warning, LKA = Lane Keeping Assist Source : IIHS
Timeline For Vehicle Automation 12 Level 1 (Feet Off) Level 2 (Hands Off) Level 3 (Eyes Off) Level 4 (Mind Off) Driver Engagement Fully Active Fully Active Available Not Required Introduction Pre 2010 2015 2016 2021 Example AEB,ACC,LKA Highway Autopilot Country & City Roads Fully Autonomous
13 Introduction Cameras and the path towards autonomous driving Need of supplementary sensors Conclusions
Park assist Park assist Roadmap for Sensor Introduction 14 Secure V2X Radar L1 L2 L3 L4 L1 L2 L3 L4 NO NO Y/N YES Surround View YES YES YES YES Active Cruise Control Pedestrian Detection Radar sensors Radar system Radar Baseband Radar Control MCU ADAS Fusion MCU Automotive Precise GNSS V2X (w/ AutoTalks) L1 YES Camera L2 L3 YES YES L4 YES Lane Keep Assist Camera sensors Vision Processor (w/ MobilEye) Machine Vision system Surround View Lidar Surround Vision Add ons L1 Y/N Lidar L2 L3 Y/N YES L4 YES Level 1 (Feet Off) Level 2 (Hands Off) Level 3 (Eyes Off) Level 4 (Mind Off)
Trend is that Vision will drive future systems 15 Richest source of raw data about the scene - only sensor that can reflect the true complexity of the scene. The lowest cost sensor - nothing can beat it, not today and not in the future. Cameras are getting better - higher dynamic range, higher resolution Combination of Radars/Lidar/Ultrasonic: for redundancy, robustness
The next phase for Vision Technology 16 From sensing to comprehensive perception Machine learning used already for object sensing Autonomous driving needs Path planning based on holistic cues Dynamic following of the drivable area 3 2 30 50 Deep learning is being used by Mobileye for Free space evaluation Path planning General objects detection 1000 traffic signs detection Classical object enhancement 1 150
Two approaches to Autonomous Driving 17 Pre-drive recording High Definition Sparse Recording No Recording of 360 surround 3D Maps Google Mobileye Store & Align Sense & Understand
Road Experience Management 18 Mobileye approach :10kB/km versus 1GB/km
19 Introduction Cameras and the path towards autonomous driving Need of supplementary sensors Conclusions
Different Systems Different Views 20
Antenna Antenna Antenna Antenna Antenna Radar Surround System 21 Short range Radar Short range Radar MMIC Radar sensors Signal Processing MMIC Radar sensors Signal Processing Radar Module Radar Module MMIC Radar sensors Signal Processing Fusion ECU Long range Radar Radar Module MMIC Radar sensors Signal Processing Radar Module MMIC Radar sensors Signal Processing Radar Module Short range Radar Short range Radar
SAE Levels Levels of Automated Driving SAE name driving Envir. Monitoring Fallback L0 No Autom. Driver Driver Driver L0 L1 Driver Assisted D+System Driver Driver L1 L2 Part. Autom. System Driver Driver L2 L3 Cond. Autom. System System Driver <10sec L3 L4 High Autom. System System Safe Halt 3/4 L5 Full Autom. System System System 3/4 NHTSA Levels 22 Car Speed 10 km/h 40km/h 80km/h 100km/h 200km/h L3 need visibility 28m 111m 222m 278m 556m Radar (24GHz, 77GHz) 180m 300m Lidar 30m 300m Cameras 250m
Model for Automated Driving Car 23 10m Ultrasonic Sensors 30m - 300m 30m 250m 300m Lidar Sensors SR-Radar Sensors Camera Sensors LR-Radar Sensors Model of the Environment System s Decisions L1 L2 L3 L4 L5 GNSS+ Maps L0 Warnings L1 Car 1000m Cars+Infrastruct. Driver s senses Driver s perception and experience Driver s Decisions L0 L1 L2 L3
LiDAR Sensors 24 Emitter Laser/VCSEL Receiver PIN, APM, SPAD Fixed Beam LIDAR (3-32 Beams) Rotating Mirror LIDAR (single and multiple LASER) MEMs-Mirror LIDAR 1D scanning MEMs-Mirror LIDAR 2 D scanning SPAD Line VCSEL Line Non moving parts LIDAR Flash or electronic scanning 2D SPAD Array 2D VCSEL Array
Collaboration with Universities 25 Almost a decade since first SPAD trails in STM technology Initial work with University of Edinburgh resulting in more than 30 joint publications Jan 2007 Jan 2008 May 2009 April 2011 MF 32x32 Ranging Test FLIM prototype Initial spad trials in ST process Spad Single Diode Trials Single + Small Array trials Deep Spad Trials
Conclusions 1/2 26 ADAS is to be dominated by cameras The primary sensor for automated driving will also be the camera Multiple cameras will be used Radars and Lidars will be used for redundancy and for additional robustness when cost allows Visual interpretation is difficult if done at high image quality and will require a massive validation resources Automated Driving requires environmental modeling and path planning
Conclusions 2/2 27 Connectivity is changing the rules radically Car Communication enables safety features Road experience management takes it to new levels The safety requirements bar is being raised and requires complex techniques Highly automated driving needs all of the above and is coming to a road near you soon! Watch this space, it may be a bumpy ride but it will be fun