AI Driven Environment Modeling for Autonomous Driving on NVIDIA DRIVE PX2 Dr. Alexey Abramov, Christopher Bayer, Dr. Claudio Heller, Claudia Loy Chassis & Safety
Agenda 1 2 3 4 5 6 7 Introduction Autonomous Driving and Cruising Chauffeur AI Driven Environment Modeling Advanced Lane Perception Road Topology Construction Site Detection Summary 2
Autonomous Driving Cruising Chauffeur Project Autopilot for highways: Germany, USA, Japan, China Worldwide development and testing We are just next door! Munich part of the environment model team 3
Environment Modeling Perception and Interpretation of Vehicle Surroundings 4
Cruising Chauffeur Highly Automated Driving on Highways Development vehicle Sensor setup Continental Serial Mono Camera High Resolution Camera Continental Short Range Radar Continental Long Range Radar Driving at high speeds on highways in Germany, USA, Japan, China Lane keeping, automatic breaking and accelerating when necessary Automatic lane change / highway interchange 5
AI Driven Environment Modeling Combining Deep Learning with Conventional Approaches Advanced Lane Perception Road Topology for Localization Construction Site Detection NVIDIA DRIVE PX2 AI computing platform in the car 6
Lane Perception Conventional Image Processing Pipeline Final Building Eliminating Feature measurement Input Sobel object image coupling filter outliers masks points 7
Lane Marking Feature Extraction Drawbacks of Traditional Approach False positives guardrails / barriers False positives cars / trucks False negative lane markers 8
Convolutional Neural Networks for Lane Perception Deep Learning Approach Fully Convolutional Networks for Semantic Segmentation [Long, Shelhamer et al. 2016] 9
CNN Based Lane Feature Classification Processing Pipeline: Verify Image Patches of Candidates CNN [0,1,0,0] 10
CNN Based Lane Feature Classification Comparison: Traditional Method False positives on guardrail False negatives due to object masks False negatives due to thresholding (Sobel & gradient orientation) 11
CNN Based Lane Feature Classification Comparison: Combined Approach 12
CNN Based Lane Feature Classification Result 13
Road Topology for Localization Deep Learning Approach N C D - D C S Traditional techniques: combine detected lane markings and road boundaries Challenge: lane markings and lanes have various appearances all over the world Idea: Deep Learning to determine topology of the road 14
Road Topology for Localization Deep Learning Approach Lane classes: ego lane and two lanes on each side N C D - D C S Learning: Left and right sides are treated independently Data Augmentation: Various lighting and weather conditions 15
Road Topology for Localization Deep Learning Approach: Challenges 16
Road Topology for Localization Result 17
Construction Site Detection CNN Based Classification construction zone ahead construction zone Aim: Construction Site (CS) / No Construction Site (NCS) given sensor data If (yellow lane markings && low speed && ) CS Scene understanding using Deep Learning 18
Construction Site Detection Result 19
Summary AI Driven Environment Modeling Advanced Lane Perception Road Topology for Localization Construction Site Detection Online inference on embedded supercomputer NVIDIA DRIVE PX2 in the vehicle 20
Space for Sender Information Confidential