Odin s Journey Development of Team Victor Tango s Autonomous Vehicle for the DARPA Urban Challenge Dennis Hong Assistant Professor Robotics & Mechanisms Laboratory (RoMeLa) dhong@vt.edu December 9th, 2007 Jesse Hurdus Graduate Student Unmanned Systems jhurdus@vt.edu
Presentation Outline Introduction System Overview Software Qualifying and Final Race
The DARPA Grand Challenge
DARPA Grand Challenge I Cliff was a ClubCar converted in only a few months and on a shoestring budget. One of three vehicles to finish the Qualifying course without hitting anything Completed about 100 yds in the desert before a drive belt failed
DARPA Grand Challenge II Virginia Tech had two entries in the 2005 Grand Challenge Cliff was rebuilt for another try at the desert Rocky was built to test new software algorithms
DARPA Grand Challenge II: The Desert Final course was 130 miles long with a 10 hour time limit Finished 8 th and 9 th, both vehicles traversed about 40 miles of the course Rocky had a generator failure, Cliff an engine shutdown
DARPA Announces Urban Challenge! Race set for November 3 rd, 2007 1 st Place: $2,000,000 2 nd Place: $1,000,000 3 rd Place: $500,000 $1,000,000 research contracts to given to 11 teams, including - Stanford University - Carnegie Mellon University - MIT - Cornell University - California Institute of Technology - Honeywell Aerospace - Raytheon - Virginia Tech!!
Team VictorTango was formed + Advisory Board
The Urban Challenge addressed city driving Navigate approximately 60 miles of roads in 6 hours Obey California State driving regulations and interact with moving vehicles Master many driving behaviors: Stop, check, and go Vehicle following Lane changing U-turns Parking All of this must be completely autonomous
A Priori data was supplied in two course definition files 1. Route Network Definition File (RNDF): Defines road segments and lane widths, zones, parking spots, stop lines, and possible waypoints throughout the course 2. Mission Definition File (MDF): Defines the current vehicle mission, which includes active checkpoints to achieve and speeds limits to obey building stop line waypoints 2 3 4 checkpoints 1 5
Odin was VictorTango s Urban Challenge entry
VT System: Modified 2005 Ford Hybrid Escape (Odin) Control and Monitoring: Electronic Control of Steering, Throttle, Shift Linear Brake Actuator E-Brake, CAN Bus Three computers on board: Windows Quad Core Server Linux Quad Core Server NI RIO (FPGA & Controller)
Power Distribution Uses the 330 VDC and 12 VDC from the Ford Escape Hybrid battery pack 2 kw of power available for added components Continuous runtime with the vehicle charging capabilities
Sensor Configuration Overview 100m Sensor Ranges NovAtel SICK Cameras IBEO laser ALASCA Propack are range used laser GPS/IMU finders for scanners vision perform provides line finding road localization obstacle and and small object detection obstacle classification detection
Entire Software Architecture IBEO LIDAR Route Planner Determine list of exits to achieve checkpoints. Replan to save time or upon request. Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Dynamic Obstacle Predictor Predict likely paths of dynamic obstacles, based on traffic rules Optimal Route (Exits) Request Replan Blockage/ Traffic Updates Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Object Classification Locate and classify dynamic and static obstacles Driving Behaviors Produce target points for highlevel maneuvers (Lane Change, Passing, Intersection Precedence, Parking) Monocular Vision GPS/INS Localization Compute global position, local position, and velocity state Target Points, Max Speed, Lane Mgmt, Direction Points Achievable? SICK LIDAR Sensor Interface Acquire, timestamp, and log sensor data Sensor Data and Specs Stop Line Position Road Detection Locate drivable area, lanes, and stop lines Motion Planner Achieve target points, Follow roads, Avoid obstacles Turn Signals Local Position and Velocity State Static Obstacles Dynamic Obstacles User Interface Provide interface to set MDF, RNDF, and Vehicle Mode Control Mode Curvature, Speed Steering Angle Platform Specs Vehicle Interface Control vehicle speed and curvature, signals, E-stop Obstacle Predictions Lane Position Drivable Area Steering Angle and Odometry Status Health Monitor Track health of all software components Status Emergency Stop Vehicle-Specific DBW Signals Drive-by-Wire (DBW)
JAUS Communication Software uses JAUS conventions and messages Software uses standard defined messages as well as custom messages Software can be moved without reconfiguration Node Manager Node Manager Component A Node 1 Component B Component C Node 2
Perception Software Overview Perception Modules Perception Messages Object Classification Static Obstacles Sensors Localization Dynamic Obstacle Predictor Dynamic Obstacle Predictions Position, Velocity Planning Modules Road Detection Drivable Area Lane Position Raw Perception data is Messages Modules taken in process output from are transmitted each sensor the of raw the independent data sensing via JAUS looking systems to Perception Planning for specific Modules Messages types of information
Obstacle Classification & Prediction IBEO Alasca laser scanner returns segmented obstacles as well as raw scan data Classification software uses additional velocity and size constraints for classification Classification results verified with vision Prediction software uses lane data and history to predict future actions
Road Detection RNDF Based: Cubic spline interpolation generated between RNDF points Georeferenced imagery allowed for tweaking in complex areas Vision Based: Initial processing based on color Road coverage is formed with morphological operations LIDAR Based: Curb detection algorithm uses statistical methods Tracks around low visibility corners and brief occlusions
Planning Software Overview The Route Planner component focuses on mission level goals, specifically which roads are traveled. The Driving Behaviors component is responsible for following the rules of the road, such as when to proceed at an intersection. Motion Planning controls the vehicle path, avoiding obstacles and setting travel speed. Vehicle Interface executes the motion commands
Route Planning Route planner chooses roads to travel and produces sequence of exits Minimizes total travel time using A* search Receives updates: Blockages Traffic speed A Planned Route
Driving Behaviors Driving behaviors makes decisions concerning the rules of the road Classified Situation: Intersection Stop Classified Situation: Standard Road Checkpoint Multiple behaviors are integrated based off the current situation Disabled Vehicle Directs motion planning with target points and a behavior profile 1. My turn? 2. Intersection clear? Odin Route Driver Passing Driver
Special Considerations in Driving Behaviors State machines within behaviors are used to deal with noisy perception data Route Driver Passing Driver Blockage Driver A winner takes all strategy is used to deal with competing behaviors Behavior Integrator Desired Lane Desired Speed Desired Turn Signal
Motion Planning Motion Planning Motion planning produces vehicle commands to reach goal points and decides if path is blocked Speed Limiter Speed limiter software monitors stop points and other vehicles Trajectory Search Searches achievable trajectory space to plan final travel path Vehicle Interface
Cost Map Creation Cost map stores cost of area around vehicle Lane Center Vehicle is at bottom center of map Costs from Obstacles Lane Obstacles Lane Boundary Cost increases with obstacle proximity Vehicle
Trajectory Search Search algorithm used to evaluate sequences of actions For each action, vehicle motion is simulated Vehicle motion is checked against cost map and goal X
Trajectory Search Search algorithm used to evaluate sequences of actions For each action, vehicle motion is simulated Vehicle motion is checked against cost map and goal Actions are combined into a final path
Vehicle Modeling Simple bicycle model used Model used to determine vehicle path for cost map φ L Vehicle Parameters Wheel Base Understeer Coefficient State Variables Heading Steering Angle Position (x,y) Inputs Forward Velocity Steering Rate
Motion Planning Example
Software Starts in Simulation Simulation provides many advantages Controlled environment Division of labor Visual Overlays Simulation step increases team safety Simulated 4-way Stop Simulation visualization also used during field testing Dynamic Obstacle Prediction
Odin ready for competition
National Qualification Event: Oct. 2007, Victorville, CA 35 teams traveled to Victorville for NQE DARPA presented 3 courses to test the teams: A, B, & C Course requirements were unknown until arrival at the site
The Competition Many different approaches to the problem were represented - Massachusetts Case- Insight Osh-Kosh Western Racing Institute of Technology
NQE Course A tested traffic behaviors
Odin was first up and performed well
but not all teams did as well
NQE B tested navigation and static obstacles
Odin started well, but had difficulties
Odin encountered some unexpected situations The Gauntlet caused the most difficulty Unnecessary blind spot check Lane widths too thin Had to pass in-between lanes
Follow-up Testing in Practice Area
NQE C tested intersections and blockages
Odin was first vehicle to fully complete NQE C
Odin was quickly selected for the finals Second of six teams to receive early selection Five more teams selected after further NQE runs DARPA fell short of 20 team target
Urban Challenge Final Event
Practice start revealed interesting complications So many robots had never been put so close together! Sensor interference GPS outages Safety radio system failures Odin had no problems DARPA changed start procedure
November 3 rd, 2007 DARPA Urban Challenge
Bot-on-Bot Interactions were common
Bot-on-Bot Interactions were common
After almost 6 long hours, Odin crossed the finish
The next day, the places were announced VictorTango took 3 rd place, winning $500,000 Tartan Racing was 1 st, Stanford 2 nd Only 6 of the 11 teams completed the course
The Celebration
Questions?
Reference Slides
Dynamic Obstacle Filtering
Motion Planning INPUTS Object Data ZONE Navigation MOTION PLANNING Vehicle Localization Zone Segment Parking SEGMENT Lane Speed Limiter Monitors dynamic obstacles and stop lines to manage desired speed Driving Behaviors Target Points, Speed, Lane Road Detection Segment U-turn Intersection Achievable Trajectory Search Determines fastest path through the given environment based on: Drivable area (lane/zone/segment) Static obstacles Speed Vehicle dynamics Pause? Target Points Achievable? Speed, Curvature, Acceleration OUTPUTS Driving Behaviors Vehicle Interface
Computing Systems Only three computers on board Windows Quad Core Server Vision processing Linux Quad Core Server Multiple virtual machines NI RIO Integrated FPGA and controller Controls drive by wire components
GPS / INS Novatel SPAN System consisting of: Novatel Propack LB plus GPS Honeywell HG1700 IMU OmniStar HP corrections Position further refined by localization software filter incorporating: Wheel speeds Stop line location
Sick LIDAR Single plane LIDAR common in robotics Mounted on roof pointed downwards Used to profile terrain Road Detection Negative Obstacle Detection
IBEO LIDAR Using IBEO s A0 scanner as well as latest XT Fusion System Multi-plane and Multi-pulse technology System includes processor to fuse sensors and classify obstacles
Cameras Odin uses a pair of industrial firewire cameras Only monocular image processing techniques used (no stereo) Vision used for road detection, lane segmentation and obstacle classification
Lane Finding Lanes are determined by analyzing detected road area and lane markings A priori lane information is used for initial lane position estimate Line/curve fitting is used to find road markings from camera images
Example 1: Cutting Off Traffic Problem narrowed to a bug in the IBEO ECU software To avoid object overflow, a region of interest is used This region of interest was not working properly Commanded ROI Odin Actual ROI Rectangular ROIs were chosen
Odin also encountered a GPS problem Drove off course on to curb Determined to be GPS pop problem 10m pop due to unknown phenomena Other teams encountered similar problems No solution found during event NQE B run otherwise successful
Site Visit: July 29, Blacksburg, VA DARPA down selected from 86 to 36 teams based on Site Visit performances Tested Skills: Basic navigation Intersection behavior Passing disabled vehicles
Bot-on-Bot Interactions were common