BigRoad. Scaling Road Data Acquisition for Dependable Self-Driving. The first two authors are co-primary student authors. *

Transcription

1 BigRoad Scaling Road Data Acquisition for Dependable Self-Driving Luyang Liu *, Hongyu Li *, Jian Liu #, Cagdas Karatas *, Yan Wang, Marco Gruteser *, Yingying Chen #, Richard P. Martin * The first two authors are co-primary student authors. * WINLAB, Rutgers University, North Brunswick, NJ, USA # Stevens Institute of Technology, Hoboken, NJ, USA Binghamton University, Binghamton, NY, USA

2 Self-Driving Cars Opportunities Challenges 2

3 3

4 Pedestrian crossing highway Deer crossing street Heavy snow on the road Tree failing down on the road 4

5 HUMAN DRIVER SELF-DRIVING CARs 100 million miles / fatality Billions of miles / fatality 5

6 Existing Approaches Mileage Goal 1,000,000,000 Year Mileage / Year Vehicle Num 100, = 100 Highly instrumented vehicles Reverse Engineering vehicle CAN Bus 6

7 BigRoad Mileage Goal 1,000,000,000 Year Mileage / Year Vehicle Num 100,000 10, ,000 =

8 BigRoad Contributions Internal Driver Inputs Steering Wheel Angle Driving Speed Vehicle Acceleration Training Self-Driving System External Perceptions Road Condition Front-view Video Unusual Situations Detection 8

9 Challenges How can we accurately determine proprietary driver inputs such as the steering operations with such light-weight instrumentations? How can we make the setup widely deployable across vehicle models? 9

10 Steering Wheel Angle Estimation Remove error caused by vehicle motion Smartphone Track steering wheel angle IMU 10

11 Steering Wheel Angle Estimation Steering Wheel Coordinate System Y θ X X Z Z Z g 11

12 Steering Wheel Angle Estimation X Z g 12

13 Steering Wheel Angle Estimation Accelerometer based approach Gyroscope based approach θ accel = artan2(accel X, accel Z ) θ gyro = න gyro Y dt Complementary Filter t t 1 θ t t 1 = 1 cc θ accel + cc θ t 1 + Δθ gyro Low-pass filter High-pass filter 13

14 Steering Wheel Angle Estimation Accelerometer based approach θ error θ error = artan2(accel Xv, accel Zv ) θ accel = artan2(accel X, accel Z ) Acceleration from Vehicle motion (Turning, Accelerating and Braking) accel v g 14

15 Challenges How can we accurately determine proprietary driver inputs such as the steering operations with such light-weight instrumentations? How can we make the setup widely deployable across vehicle models? 15

16 Device Calibration Vehicle System Steering Wheel System Y s Z s 16

17 Device Calibration Steering Wheel Coordinate System Z v Y s X v Z s Vehicle Coordinate System Y v 17

18 Device Calibration Steering Wheel Coordinate System Z v Y s Z v φ pitch Y v X v Z s Vehicle Coordinate System Y v 18

19 Other Outputs Vehicle Speed Fuse the GPS and accelerometer readings to derive high frequency vehicle speed on smartphone. V t t 1 t = 1 cc V gps + cc V t 1 t + ΔV accel Vehicle Acceleration Road Condition Estimation Detect binary road condition using the crowd sourced data from smartphone. 19

20 Experimental Setup 6 vehicle models. 7 drivers. 143 daily commute drives. 3 month period. Various types of road in NJ and NY, USA. IMU Smartphone 20

21 21

22 Evaluation: Steering Wheel Angle Mean error: 0.96 degrees 90-percentile error: 1.99 degrees 22

23 Evaluation: Driving Speed & Road Condition Mean error 0.65 km/h. 90-percentile error: 2.11 km/h Determine road conditions with 95% 23

24 Evaluation: Usefulness of BigRoad Self Steering Algorithm 3 Convolutional Layers 2 Fully Connected Layers 24

25 Evaluation: Usefulness of BigRoad Self Steering Algorithm Model 1: Trained with In-vehicle data Model 2: Trained with BigRoad data 25

26 26

27 Conclusion and Future Work We present BigRoad that can scale road data acquisition for self-driving. We show that driving and road data can be estimated accurately using very few off-the-shelf sensing devices. BigRoad can calibrate itself to different vehicle types, steering wheel positions and sensor placements. Build an easy-to-install and sustainable setup for BigRoad. Design an interesting and unusual events detection method on smartphone. Deploy the BigRoad system. 27

28 Thank you! Luyang Liu WINLAB, Rutgers University