Odin s Journey. Development of Team Victor Tango s Autonomous Vehicle for the DARPA Urban Challenge. Jesse Hurdus. Dennis Hong. December 9th, 2007

Transcription

1 Odin s Journey Development of Team Victor Tango s Autonomous Vehicle for the DARPA Urban Challenge Dennis Hong Assistant Professor Robotics & Mechanisms Laboratory (RoMeLa) dhong@vt.edu December 9th, 2007 Jesse Hurdus Graduate Student Unmanned Systems jhurdus@vt.edu

2 Presentation Outline Introduction System Overview Software Qualifying and Final Race

3 The DARPA Grand Challenge

4 DARPA Grand Challenge I Cliff was a ClubCar converted in only a few months and on a shoestring budget. One of three vehicles to finish the Qualifying course without hitting anything Completed about 100 yds in the desert before a drive belt failed

5 DARPA Grand Challenge II Virginia Tech had two entries in the 2005 Grand Challenge Cliff was rebuilt for another try at the desert Rocky was built to test new software algorithms

6 DARPA Grand Challenge II: The Desert Final course was 130 miles long with a 10 hour time limit Finished 8 th and 9 th, both vehicles traversed about 40 miles of the course Rocky had a generator failure, Cliff an engine shutdown

7 DARPA Announces Urban Challenge! Race set for November 3 rd, st Place: $2,000,000 2 nd Place: $1,000,000 3 rd Place: $500,000 $1,000,000 research contracts to given to 11 teams, including - Stanford University - Carnegie Mellon University - MIT - Cornell University - California Institute of Technology - Honeywell Aerospace - Raytheon - Virginia Tech!!

8 Team VictorTango was formed + Advisory Board

9 The Urban Challenge addressed city driving Navigate approximately 60 miles of roads in 6 hours Obey California State driving regulations and interact with moving vehicles Master many driving behaviors: Stop, check, and go Vehicle following Lane changing U-turns Parking All of this must be completely autonomous

10 A Priori data was supplied in two course definition files 1. Route Network Definition File (RNDF): Defines road segments and lane widths, zones, parking spots, stop lines, and possible waypoints throughout the course 2. Mission Definition File (MDF): Defines the current vehicle mission, which includes active checkpoints to achieve and speeds limits to obey building stop line waypoints checkpoints 1 5

11 Odin was VictorTango s Urban Challenge entry

12 VT System: Modified 2005 Ford Hybrid Escape (Odin) Control and Monitoring: Electronic Control of Steering, Throttle, Shift Linear Brake Actuator E-Brake, CAN Bus Three computers on board: Windows Quad Core Server Linux Quad Core Server NI RIO (FPGA & Controller)

13 Power Distribution Uses the 330 VDC and 12 VDC from the Ford Escape Hybrid battery pack 2 kw of power available for added components Continuous runtime with the vehicle charging capabilities

14 Sensor Configuration Overview 100m Sensor Ranges NovAtel SICK Cameras IBEO laser ALASCA Propack are range used laser GPS/IMU finders for scanners vision perform provides line finding road localization obstacle and and small object detection obstacle classification detection

15 Entire Software Architecture IBEO LIDAR Route Planner Determine list of exits to achieve checkpoints. Replan to save time or upon request. Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Dynamic Obstacle Predictor Predict likely paths of dynamic obstacles, based on traffic rules Optimal Route (Exits) Request Replan Blockage/ Traffic Updates Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Object Classification Locate and classify dynamic and static obstacles Driving Behaviors Produce target points for highlevel maneuvers (Lane Change, Passing, Intersection Precedence, Parking) Monocular Vision GPS/INS Localization Compute global position, local position, and velocity state Target Points, Max Speed, Lane Mgmt, Direction Points Achievable? SICK LIDAR Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Stop Line Position Road Detection Locate drivable area, lanes, and stop lines Motion Planner Achieve target points, Follow roads, Avoid obstacles Turn Signals Local Position and Velocity State Static Obstacles Dynamic Obstacles User Interface Provide interface to set MDF, RNDF, and Vehicle Mode Control Mode Curvature, Speed Steering Angle Platform Specs Vehicle Interface Control vehicle speed and curvature, signals, E-stop Obstacle Predictions Lane Position Drivable Area Steering Angle and Odometry Status Health Monitor Track health of all software components Status Emergency Stop Vehicle-Specific DBW Signals Drive-by-Wire (DBW)

16 JAUS Communication Software uses JAUS conventions and messages Software uses standard defined messages as well as custom messages Software can be moved without reconfiguration Node Manager Node Manager Component A Node 1 Component B Component C Node 2

17 Perception Software Overview Perception Modules Perception Messages Object Classification Static Obstacles Sensors Localization Dynamic Obstacle Predictor Dynamic Obstacle Predictions Position, Velocity Planning Modules Road Detection Drivable Area Lane Position Raw Perception data is Messages Modules taken in process output from are transmitted each sensor the of raw the independent data sensing via JAUS looking systems to Perception Planning for specific Modules Messages types of information

18 Obstacle Classification & Prediction IBEO Alasca laser scanner returns segmented obstacles as well as raw scan data Classification software uses additional velocity and size constraints for classification Classification results verified with vision Prediction software uses lane data and history to predict future actions

19 Road Detection RNDF Based: Cubic spline interpolation generated between RNDF points Georeferenced imagery allowed for tweaking in complex areas Vision Based: Initial processing based on color Road coverage is formed with morphological operations LIDAR Based: Curb detection algorithm uses statistical methods Tracks around low visibility corners and brief occlusions

20 Planning Software Overview The Route Planner component focuses on mission level goals, specifically which roads are traveled. The Driving Behaviors component is responsible for following the rules of the road, such as when to proceed at an intersection. Motion Planning controls the vehicle path, avoiding obstacles and setting travel speed. Vehicle Interface executes the motion commands

21 Route Planning Route planner chooses roads to travel and produces sequence of exits Minimizes total travel time using A* search Receives updates: Blockages Traffic speed A Planned Route

22 Driving Behaviors Driving behaviors makes decisions concerning the rules of the road Classified Situation: Intersection Stop Classified Situation: Standard Road Checkpoint Multiple behaviors are integrated based off the current situation Disabled Vehicle Directs motion planning with target points and a behavior profile 1. My turn? 2. Intersection clear? Odin Route Driver Passing Driver

23 Special Considerations in Driving Behaviors State machines within behaviors are used to deal with noisy perception data Route Driver Passing Driver Blockage Driver A winner takes all strategy is used to deal with competing behaviors Behavior Integrator Desired Lane Desired Speed Desired Turn Signal

24 Motion Planning Motion Planning Motion planning produces vehicle commands to reach goal points and decides if path is blocked Speed Limiter Speed limiter software monitors stop points and other vehicles Trajectory Search Searches achievable trajectory space to plan final travel path Vehicle Interface

25 Cost Map Creation Cost map stores cost of area around vehicle Lane Center Vehicle is at bottom center of map Costs from Obstacles Lane Obstacles Lane Boundary Cost increases with obstacle proximity Vehicle

26 Trajectory Search Search algorithm used to evaluate sequences of actions For each action, vehicle motion is simulated Vehicle motion is checked against cost map and goal X

27 Trajectory Search Search algorithm used to evaluate sequences of actions For each action, vehicle motion is simulated Vehicle motion is checked against cost map and goal Actions are combined into a final path

28 Vehicle Modeling Simple bicycle model used Model used to determine vehicle path for cost map φ L Vehicle Parameters Wheel Base Understeer Coefficient State Variables Heading Steering Angle Position (x,y) Inputs Forward Velocity Steering Rate

29 Motion Planning Example

30 Software Starts in Simulation Simulation provides many advantages Controlled environment Division of labor Visual Overlays Simulation step increases team safety Simulated 4-way Stop Simulation visualization also used during field testing Dynamic Obstacle Prediction

31 Odin ready for competition

32 National Qualification Event: Oct. 2007, Victorville, CA 35 teams traveled to Victorville for NQE DARPA presented 3 courses to test the teams: A, B, & C Course requirements were unknown until arrival at the site

33 The Competition Many different approaches to the problem were represented - Massachusetts Case- Insight Osh-Kosh Western Racing Institute of Technology

34 NQE Course A tested traffic behaviors

35 Odin was first up and performed well

36 but not all teams did as well

37 NQE B tested navigation and static obstacles

38 Odin started well, but had difficulties

39 Odin encountered some unexpected situations The Gauntlet caused the most difficulty Unnecessary blind spot check Lane widths too thin Had to pass in-between lanes

40 Follow-up Testing in Practice Area

41 NQE C tested intersections and blockages

42 Odin was first vehicle to fully complete NQE C

43 Odin was quickly selected for the finals Second of six teams to receive early selection Five more teams selected after further NQE runs DARPA fell short of 20 team target

44 Urban Challenge Final Event

45 Practice start revealed interesting complications So many robots had never been put so close together! Sensor interference GPS outages Safety radio system failures Odin had no problems DARPA changed start procedure

46 November 3 rd, 2007 DARPA Urban Challenge

47 Bot-on-Bot Interactions were common

48 Bot-on-Bot Interactions were common

49 After almost 6 long hours, Odin crossed the finish

50 The next day, the places were announced VictorTango took 3 rd place, winning $500,000 Tartan Racing was 1 st, Stanford 2 nd Only 6 of the 11 teams completed the course

51 The Celebration

52 Questions?

53 Reference Slides

54 Dynamic Obstacle Filtering

55 Motion Planning INPUTS Object Data ZONE Navigation MOTION PLANNING Vehicle Localization Zone Segment Parking SEGMENT Lane Speed Limiter Monitors dynamic obstacles and stop lines to manage desired speed Driving Behaviors Target Points, Speed, Lane Road Detection Segment U-turn Intersection Achievable Trajectory Search Determines fastest path through the given environment based on: Drivable area (lane/zone/segment) Static obstacles Speed Vehicle dynamics Pause? Target Points Achievable? Speed, Curvature, Acceleration OUTPUTS Driving Behaviors Vehicle Interface

56 Computing Systems Only three computers on board Windows Quad Core Server Vision processing Linux Quad Core Server Multiple virtual machines NI RIO Integrated FPGA and controller Controls drive by wire components

57 GPS / INS Novatel SPAN System consisting of: Novatel Propack LB plus GPS Honeywell HG1700 IMU OmniStar HP corrections Position further refined by localization software filter incorporating: Wheel speeds Stop line location

58 Sick LIDAR Single plane LIDAR common in robotics Mounted on roof pointed downwards Used to profile terrain Road Detection Negative Obstacle Detection

59 IBEO LIDAR Using IBEO s A0 scanner as well as latest XT Fusion System Multi-plane and Multi-pulse technology System includes processor to fuse sensors and classify obstacles

60 Cameras Odin uses a pair of industrial firewire cameras Only monocular image processing techniques used (no stereo) Vision used for road detection, lane segmentation and obstacle classification

61 Lane Finding Lanes are determined by analyzing detected road area and lane markings A priori lane information is used for initial lane position estimate Line/curve fitting is used to find road markings from camera images

62 Example 1: Cutting Off Traffic Problem narrowed to a bug in the IBEO ECU software To avoid object overflow, a region of interest is used This region of interest was not working properly Commanded ROI Odin Actual ROI Rectangular ROIs were chosen

63 Odin also encountered a GPS problem Drove off course on to curb Determined to be GPS pop problem 10m pop due to unknown phenomena Other teams encountered similar problems No solution found during event NQE B run otherwise successful

64 Site Visit: July 29, Blacksburg, VA DARPA down selected from 86 to 36 teams based on Site Visit performances Tested Skills: Basic navigation Intersection behavior Passing disabled vehicles

65 Bot-on-Bot Interactions were common