Risk-Based Collision Avoidance in Semi-Autonomous Vehicles

Independent Work Report Spring, 2016 Risk-Based Collision Avoidance in Semi-Autonomous Vehicles Christopher Hay 17 Adviser: Thomas Funkhouser Abstract Although there have been a number of advances in active vehicle safety, such as Autonomous Emergency Braking[2] and Steering[5], there is significant room for improvement. By using a riskminimization approach that utilizes a neural network trained from simulation and re-evaluates its control decision 30 times per second, we are able to significantly improve upon Autonomous Emergency systems. In extreme simulation scenarios, we are able to reduce the rate of collision/going offroad by 39.41% when compared to Autonomous Emergency systems. For these simulations, we reduce the overall rate of collision/going offroad from 46.15% for Autonomous Emergency systems to 27.72% with our system. 1. Introduction In the U.S., roughly 31,000 people die in road crashes every year, and another 2.5 million are injured [4]. Globally, things are much worse; 1.3 million people die in road crashes every year, and another 20-50 million are injured [4]. In addition to the cost to life, there s also a substantial cost to capital. In 2012, U.S. auto insurers lost roughly $27.5 billion for collision related expenses [6]. Although there have been a number of advances in active vehicle safety, such as Autonomous Emergency Braking and Steering, there is still much left to be desired. With current safety mechanisms, people tend to assume that these safety mechanisms will do more than what they actually do [7]. As a result, these safety features simply enable drivers to drive more aggressively [7]. In order to combat this trend, one popular solution is through the use of semi-autonomous vehicles. As computers gain more and more control over the vehicles, the vehicles are better able to account for human error. Thus, the goal of this project is to develop a collision avoidance controller for semi-autonomous vehicles that utilizes a risk-dependent combination

of Autonomous Emergency Braking/Steering that outperforms Autonomous Emergency Braking systems and Autonomous Emergency Steering systems. So, the approach of this paper is threefold. First, we use simulation to generate tens of thousands of training samples. Because collisions are both costly and rare, it is very difficult to acquire a training set from the real world. As such, we turn to simulation to provide the necessary ground truths. Once we have generated simulation training data, we can move onto the task of risk estimation. Here, we aim to estimate the risk associated with every possible control choice. For a single obstacle vehicle ahead of us, control involves choosing the magnitude of the braking and steering systems that we wish to apply. For multiple obstacle vehicles, this also involves choosing which lane to steer into. By using DeepDriving [3] to estimate the distance to each vehicle and to estimate the speed of our own, we can run every possible control decision through a neural network to predict the probability of colliding or going offroad. Then, in the risk-minimization step, we choose the control behavior that minimizes those risks. If we run this risk estimation/risk minimization loop repeatedly 30 times per second, we are able to micromanage the control of the vehicle much more effectively than both Autonomous Emergency Braking and Autonomous Emergency Steering. In extreme simulation scenarios, we are able to reduce the rate of collision/going offroad by 39.41% when compared to Autonomous Emergency systems. For these simulations, we reduce the overall rate of collision/going offroad from 46.15% for Autonomous Emergency systems to 27.72% with our system. 2. Related Work 2.1. Autonomous Emergency Braking Dating to the mid 1990s, there has been much research on brake-based forward collision avoidance mechanisms. With brake assist, the vehicle preemptively increases the braking response if a vehicle is detected (typically via radar) with a time-to-collision metric below some threshold. However, it does not actually apply the brakes. Instead, when the driver presses on the brakes with brake assist activated, the brakes will immediately be applied with full force, instead of slowly ramping 2

up as the hydraulic system activates. Taking this a step further, we get the Autonomous Emergency Braking designed by Ford[2]. With Autonomous Emergency Braking, we combine braking assist with electronic actuation of the brakes. If a vehicle is detected with a time-to-collision below a different threshold (that s lower than the brake-assist threshold), then we apply the brakes. Since this threshold is smaller than the brake assist threshold, brake assist will have already been activated, and this actuation of the brakes will be at full force. 2.2. Autonomous Emergency Steering In addition to braking-based collision avoidance, there s also steering-based collision avoidance. While Autonomous Emergency Braking has been in consumer vehicles for the last decade or so, the steering-based corollary, Autonomous Emergency Steering, only exists in high-end consumer vehicles released in the last couple of years. So, this area is still being actively researched. With Hyundai s and Nissan s models[5], Autonomous Emergency Steering involves detecting both a vehicle ahead as well as detecting the absence of vehicles in either the left or right lane. In order to accomplish this, scene recognition algorithms are used to determine the current lane configuration (one lane, two lanes, three lanes, etc.) as well as the current position of the vehicle in respect to the lanes. In addition, object detection algorithms are run to determine the presence of all nearby vehicles. Then, if a vehicle/pedestrian is detected ahead with a time-to-collision metric below some threshold, and we are going too fast for Autonomous Emergency Braking to be effective, then we attempt a lane-changing maneuver into an empty lane. 3. Problem Background 3.1. Combination of Autonomous Emergency Braking and Steering Thus, with the current state of active safety mechanisms, we attempt Autonomous Emergency Braking at low speeds, and Autonomous Emergency Steering at high speeds. However, the physics behind all of this complicates things immensely. Again, for low speeds, the vehicle is able to avoid a collision by simply braking before a threshold distance. This threshold distance is determined by 3

the physics of deceleration, and is a factor of distance, velocity, and various coefficients of friction that vary based on vehicle model and on road surface conditions. If we assume constant deceleration (which is somewhat realistic with brake assist and Anti-lock Braking Systems), then we can model braking as such: d f = d i + v2 f v2 i 2a a = µg So, we avoid a collision if v 2 them v2 us 2µg < d i So, how do we determine the coefficient of friction or the velocity of the obstacle vehicle? Additionally, at high speeds we add new complications to the mix. In addition to the above complications, the magnitude of the steering response needs to vary with speed and distance in order to avoiding oversteering and ending up offroad. Conversely, we don t want to understeer and fail at avoiding the collision. However, the amount of steering that causes oversteering is dependent on the lane configuration, the current position within the road, and the coefficient of friction with the road, each of which is difficult to estimate. Each of these complications makes collision avoidance an extremely complicated matter to solve without the use of differential systems. Additionally, there is an interesting decision boundary at hand: when should we brake, steer, or perform some combination of the two? While some behavioral research has been done on analyzing where human drivers make mistakes, the only automated implementations use simple heuristics or are implemented in proprietary systems. In order to analyze this problem, one needs vast amounts of detailed collision data. However, this data is very difficult to come by. Collisions are both rare in the real world and undesirable to replicate in test conditions due to the high costs associated with vehicle damage. So, there is a lack of scenario-dependent, or risk-based, safety mechanisms that are publicly available. This paper aims to provide one such mechanism. 4

3.2. DeepDriving In order to utilize active safety mechanisms such as AEB or AES as explained in Sections 2.1 and 2.2, one needs to be able to perceive the current state of the vehicle s environment. For Autonomous Emergency Braking, we need the distance to the vehicle ahead, our current velocity, and the vehicle ahead s current velocity, since braking can roughly be modeled with constant deceleration. In an ideal world, we would also be able to perceive the coefficient of friction between the road and the tire. For Autonomous Emergency Steering, we also need to know the distances to the nearest vehicles both ahead and behind our vehicle in both the left and right lanes. While this level of sensing can be accomplished with expensive radar or LIDAR systems, we can also get a reasonable estimate of these variables with just images if we use the right algorithm. This is what DeepDriving gives us. It is a convoluted neural network that takes in an image from a camera mounted in the center-front of the vehicle and gives us affordance indicators for driving, including the distances to the nearest vehicles in each lane, our current speed, and the distances to each of the lane markings.[3] Additionally, the architecture for DeepDriving offers a new paradigm for vision-based autonomous vehicle control that allows simulation to become extremely useful. Historically, visionbased autonomous vehicles have used either mediated perception or behavior reflex to control the vehicle. With mediated perception, we first try to determine bounding boxes for vehicles and lane markings. Afterwards, we determine the distance from us to each bounding box. Finally, we determine the control to be issued, which is then translated into brake, throttle, and steering output. This is an extremely complicated system, but it allows parts of the system to be easily replaced. However, many of those pieces require hard-to-acquire or hard-to-extend datasets that constrain the overall effectiveness of an automated control system, and thus requires vast amounts of manual control logic. In contrast, behavior reflex simply trains a black-box (typically a convoluted neural network) to take in an image as input, and output brake, throttle, and steering commands. While simple, this model requires incredibly large amounts of training data in order to be even remotely effective and robust. As a result, mediated perception is typically preferred. So, DeepDriving offers a third paradigm, coined direct perception. With this architecture, we train a black-box that 5

takes in an image as input, and outputs affordance indicators for driving. These include distances to all nearby vehicles, the current speed of our vehicle, and the distances to all lane markings. From this, one can construct an extremely simple controller that is roughly as good as a rather complicated mediated perception controller. Additionally, this use of a black-box means that we have no dataset dependencies, and can use any dataset that contains an image and the associated affordance indicators. In the research for DeepDriving, a dataset was constructed via simulation. Remarkably, the resulting black-box algorithm was found to have similar accuracy in simulation as real-world performance; this is paramount to its use in this paper. 4. Approach When utilizing Autonomous Emergency Braking or Autonomous Emergency Steering, the necessary controller essentially boils down to a long series of If-Else statements that can be thought of as a manually constructed decision tree. In the case of a direct perception based controller that uses AEB and AES, this involves taking the affordance indicators as input to the conditional statements in the If blocks, and ultimately outputting braking, throttle, and steering. Instead of painstakingly manually constructing this controller to handle every possible circumstance, we propose a method of automatically learning a controller for the specific purpose of collision avoidance. In order to train a controller using typical AI algorithms, we need a training dataset and some way to evaluate our performance. In order to create the dataset, we propose using The Open Source Racing Car Simulator [8] to generate random collision scenarios where we know the ground truths for everything. For risk estimate evaluation, we want to be able to avoid collisions and also avoid going offroad, so for simplicity, we simply assign equal cost to either. Going offroad is likely to be equally as disastrous as a collision with another vehicle. In order to decide control, our approach is to simply evaluate the risk of collision/going offroad for every possible control choice at every time step (30 times per second). Since we know that we re in a collision avoidance scenario, there s little use for the throttle. So, we only need to choose braking and steering controls. To do this, we can discretize the steering and braking outputs and 6

evaluate the risk for choosing each possible combination of steering and braking. For this paper, we propose an approach where the vehicle evaluates the risk of every possible control behavior at every time step, then chooses the control behavior that minimizes this risk. In doing so, we need no temporal information about the environment and can rely directly on classification-based machine learning techniques. In order to estimate risk in each timestep, we need to be able to convert an image into a probability of collision/going offroad. In order to do this, we propose a method that first uses the direct perception algorithm DeepDriving to estimate 13 affordance indicators (including the distance to all nearby vehicles and the current location of our vehicle). From these affordance indicators, we can then predict the probability of colliding or going offroad, and then choosing the action that minimizes these probabilities. 4.1. Key Insight In order to train the classifiers, we need the ground truths for the variables of interest: the distance to each nearby vehicle, the speed of our vehicle, and the control being used. The control being used is trivial to acquire (we decide it!), but the speed of our vehicle and the distance to neighboring vehicles is extremely difficult to acquire with traditional datasets. Here lies the advantage of simulation: we know everything, because we put it there in the first place! Now, if simulation is so amazing, why isn t it used for everything? In order to use simulation effectively, the results found within the simulation must translate well to reality. Since we use a two tiered system, showing that both sub-components satisfy this constraint is sufficient to show that the entire system satisfies this constraint. So, we need a perception algorithm that has real world results comparable to simulation results. Similarly, we need a physics engine for steering and braking that is comparable to real world steering and braking. With a direct perception algorithm, such as DeepDriving, we fulfill the first requirement. Now, we just need a decent physics engine. For this, we use The Open Racing Car Simulator, as its physics model was designed to be highly realistic in order to allow competitive AI races. For example, the physics engine takes into account air drag, friction between the ground and the wheel, friction between the wheel and the brake pad, the temperature of the brake pad 7

and how this effects the coefficient of friction between the brakes and the wheel, among countless other things. In addition, it also takes wheel slip into account. So, after implementing an Anti-lock Braking System and Traction Control for the vehicles, we gain a simulation environment that closely resembles the driving of real vehicles. 5. Implementation for Single Obstacle For the single obstacle task, we look at collision avoidance when we are in the center lane of a three lane road with a single obstacle vehicle directly ahead. However, this can easily be generalized to any situation in which we know that there is a free lane to either side and an obstacle vehicle ahead. 5.1. Generation of Training Data In order to estimate the risk of each control behavior, we need to be able to get data with the following requirements. First, we must have an image of the scene from a point on the vehicle that would be feasible to put a camera for a real vehicle. Second, we must have the true distance to the nearest vehicle in the left lane, the current lane, and the right lane. Third, we must have the current speed of our vehicle. Fourth, we must know the control behavior being used by the vehicle. Last, we must know whether the vehicle will end up colliding or going offroad. Because we are using simulation, we gain the advantage of fully knowing all ground truths. We do not need to hand label every single image, which incurs an enormous cost and a significant reduction in accuracy (since nobody knows the true distances to objects due to human error or sensor error depending on how the images are labeled). So, we know the true positions and velocities for every vehicle by nature of using simulation, and can easily compute the true distances. Now, we just need someway to determine if we ll end up colliding or going offroad. Thus, we ll now delve into experimental design. In the final system, we want to be able to estimate the risk of every possible control choice based off of our estimates of the current state. So, in order to assess those risks, we can simply simulate every possible control choice for every possible state ahead of time. Since we re dealing with extremely non-linear systems that take in continuous input, it is impossible to simulate everything. However, we can sample from this space 8

via Monte Carlo simulation, and then use machine learning techniques to fill in the gaps. So, the implementation chosen was to heavily modify The Open Source Racing Car Simulator (TORCS)[8] so that every vehicle is assigned a random starting position and a random starting velocity. In addition, the host car is assigned a random control. For the single obstacle system, it is given random brake and steering in the [0,1] range that corresponds with the proportion of AEB[2]/AES[5] to use. In each sample, we initially assume that the host vehicle and the obstacle vehicle are both in the center lane of a a three lane track. The starting distance between the two vehicles is randomly sampled from a uniform distribution that ranges from 0m to 200m. Similarly, the starting velocities of both vehicles are independently randomly sampled from a uniform distribution that ranges from 0km/hr to 150km/hr. However, TORCS is not designed to allow custom reconfiguration of the vehicles. They are hardcoded to start in a racing grid formation, so modifying this setup led to some quirks in starting behavior. While the starting locations and velocities were still able to be randomly sampled, the game automatically transformed this input. What appears to be going on is that the game advances through a few timesteps before starting the game. As a result, the game occasionally starts with a collision occurring behind the scenes. Since this only occurs rarely, we simply filter out these instances when doing analysis. Notably, though, we must be careful to record the starting positions and velocities once the simulation has begun, and not when we re initializing the vehicles. Additionally, the magnitude of braking and steering applied are both independently randomly sampled from a uniform distribution ranging from 0 to 1. Since we only have two vehicles, and we re starting in the center lane, we can assume that steering direction is irrelevant. So, for simplicity, we assume that the vehicle will always attempt to turn towards the left-hand lane. In Figure 1, we can see an example of a random simulation initialization. Here, we see the host car immediately in front of the camera, and the obstacle car directly ahead. After creating the initial conditions, we then run the simulation until a stopping condition is met. For the sake of efficiency, we want to restart the simulation as soon as possible. Since TORCS was designed as a racing game, it has a built in system for handling racing positions. Since we re assuming that neither vehicle is using the throttle, if the vehicle in last place has a speed of 0km/hr, 9

then it is impossible for any future collisions to occur. Similarly, if the host car has a speed of 0m/s, then it is impossible for it to go offroad in the future. As such, we can safely restart the simulation if both of these conditions are met. Before doing so, though, we record a summary of the simulation. Since it s possible that there is a collision prior to the simulation starting, we record the starting damage taken by both vehicles. Then, at the end of the simulation, we record the final damages taken by both vehicles. Then, if the difference between starting and final damages is larger than zero for both vehicles, a collision must have occurred during the simulation. Additionally, we record if the vehicle ever went offroad. Furthermore, we keep track of the average and max acceleration experienced by the host car during the simulation. Lastly, we record the final velocities of both vehicles and the final distance between them. Then, we start again with new random positions, velocities, and control. Figure 1: Example Simulation Initialization in TORCS 5.2. Risk Estimation In this stage of the project, we are no longer training the system. So, we no longer have any of the ground truths. For most machine-learning problems, this would simply mean a lack of the 10

labels; in this case, a lack of the knowledge about whether the host car will collide or go offroad. However, since we aim to use this system outside of simulation (on real vehicles), we must place further constraints. In reality, we have no way of knowing the true distances to each car and the true velocity of ours, so we must find some way to estimate them. Outside of simulation, it is extremely simple to mount a single, low-resolution camera on the front center of the car. So, we replicate this type of input in the game. That is, we take a single image from the host vehicle s camera, and then use it to try to predict our velocity and the distances to each vehicle. To do this, we implement the DeepDriving algorithm [3]. So, if we run the image captured from the game through a DeepDriving CNN, we get estimates of affordance indicators, including the distance to the nearest vehicle in the immediately surrounding lanes, our velocity, the distances to each of the lane markings, and the angle of our orientation. In Figure 2, we can see the type of input generated by the camera, and a visualization of the affordance indicators and their errors as provided by DeepDriving. As we can see, the solid red and yellow boxes represent the ground truths, and the outlined green and purple boxes represent DeepDriving s estimations. In this example, it does rather well. Figure 2: Example of DeepDriving Input and Affordance Indicator Estimation Now that we have an estimate of the current state, we can begin to assess the risk. For this, we 11

let the risk of a control choice be defined as the probability of collision OR going offroad when utilizing that control choice in the current state. Since we are trying to predict whether a boolean variable will be true or not, a natural way to estimate this risk for unseen states is to simply train a classifier. For most classifiers, there is something related to a probability of being classified as a 0/1 that is then thresholded to create a prediction. So, we can use a classifier s function for the probability of being labeled 1 as the risk. Now, not every classifier is going to have the same type of probability measure. Those that attempt to maximize a margin that separates classes (such as SVMs) tend to push the estimated probabilities toward 0 or toward 1. As such, the estimated probability does not reflect the true probability. Additionally, bagging based methods tend to do the opposite. Due to the higher variance of their base estimators, and due to the internal averaging, the resulting probabilities are biased to be further away from the extremes. In contrast, methods that inherently try to estimate the probability directly (such as logistic regression) are able to much more accurately assess the true probability. However, this is not necessarily desirable. In general, it is more important to find the relative risk between behaviors and less important to find the true risk. As such, it s more important to correctly classify the result of choosing a type of control than it is to correctly estimate the true probability of each label. Since there is typically a tradeoff between the two, we prefer models that are more discriminative in their classification, such as Random Forests, AdaBoost, SVMs, and potentially Neural Networks. However, we also look at the performance of Logistic Regression and Bernoulli Naive Bayes. Note: the neural network implemented is a shallow neural network with two hidden layers. The first is a sigmoid layer with the second is a rectifier layer with estimation), we use a softmax output layer. (num f eatures)2 4 nodes, and (num f eatures)2 4 nodes. For prediction (as opposed to probability For future reference, let s define the features constructed outside of TORCS. SpeedDifference is equal to the starting velocity of the host car minus the starting velocity of the obstacle car. DeltaDamage is the difference in damage between the start of the simulation and the end of the simulation. A collision occurs if DeltaDamage is greater than 0 for both the host vehicle and the obstacle vehicle. Brake and steering control are discretized from the range [0, 1] to the set 12

{0,0.2,0.4,0.6,0.8,1.0}. Samples where the host vehicle started in front of the obstacle vehicle were dropped. 5.3. Risk Minimization Regardless of the choice of classifier, the system s architecture is the same. In order for the vehicle to determine the proportion of braking and steering to apply, it sends the estimated distance to the obstacle car and estimated current speed to a server. This server then evaluates the risk for every single possible combination of braking and steering control. Since the proportion of braking/steering to apply is a continuous variable that ranges from 0 to 1, we choose to discretize the space from the range [0,1] to the set {0,0.2,0.4,0.6,0.8,1.0} instead of using some optimization heuristic. This choice was made due to runtime constraints. In order to control the vehicle in real time at high velocities, we only have a few milliseconds to make a decision since DeepDriving already takes 20-30ms. Additionally, the classifiers used have been optimized via parallelization to make predictions on batches, so it is much less time consuming to predict for a large, but fixed, set of control choices in parallel than it is to sequentially predict and simultaneously run the optimization heuristic. Once we have the predicted probabilities of collision/offroad for each of the control choices, we simply return the control behavior that has the minimum probability. 6. Results for Single Obstacle In order to assess the performance of the system as a whole, we need to assess the performance of each sub-component: both risk estimation and risk minimization. 6.1. Risk Estimation In the collected data set, the probability of collision or going offroad is roughly 1/4. Because of this, a classifier can perform "well" by classifying almost everything as negative. So, to reduce this bias in training and to make the classifiers more apt at differentiating between the two classes, we alter the dataset so that the proportion of positive and negative classes is equal. This was done 13

by keeping all positive samples, and randomly sampling from the set negative samples until we reached an equal number of each. Since the goal of the classifiers is to correctly categorize whether we will collide/go offroad or not, the primary metrics used are going to be recall, precision, and F1-score. In particular, denoting the number of false positives (FP), true positives (TP), false negatives (FN), true negatives (TN), we can define precision, recall, and the F 1 -score for the positive class as: precision + = T P T P + FP, T P recall+ = T P + FN, F+ 1 = 2 precision+ recall + precision + + recall + For the negative class, we can define precision, recall, and the F 1 -score similarly: precision = T N T N + FN, recall = T N T N + FP, F 1 = 2 precision recall precision + recall In Table 1, we can see the performance metrics for the task of classifying whether the vehicle will collide OR go offroad when using k-fold cross validation with k = 10. Since this is a binary classification task, we have the metrics for both the positive class (where the vehicle collides/goes offroad) and the negative class. As we can see, most of the classifiers do rather well except for Bernoulli Naive Bayes. Because of the resampling of the dataset, we see that each of the metrics is roughly equivalent across classes, indicating that the trained classes are equally good at classifying either category. This is important since we want to be able to use the estimated probabilities to make a discriminative decision. If a classifier were biased to the negative class, as was the case before resampling the dataset, then the resulting classifier has higher uncertainty when it predicts a collision/offroad event. In addition to training performance, another important metric is the time in which it takes to make a prediction. Since we need to be able to make decisions in real-time, we must be able to calculate the probability of collision/offroad extremely quickly. So, let s look at the average time it takes to make a decision once given the affordance indicators; that is, the average time to find the minimum probability of collision/offroad for 36 different possible control choices given the 14

Classifier precision recall F1 precision + recall + F 1 + Avg. F 1 Neural Network 0.91 0.86 0.89 0.89 0.89 0.89 0.89 Random Forest Classifier 0.90 0.87 0.88 0.87 0.90 0.88 0.88 AdaBoost Classifier 0.86 0.87 0.87 0.87 0.86 0.87 0.87 Logistic Regression 0.87 0.83 0.85 0.83 0.87 0.85 0.85 Support Vector Machine 0.80 0.84 0.82 0.83 0.79 0.81 0.82 Bernoulli Naive Bayes 0.55 0.88 0.67 0.69 0.28 0.40 0.54 Table 1: Training/Risk Estimation Performance for Single Obstacle Vehicle output from DeepDriving. For this, the Neural Network takes 3.32ms, the Random Forest takes 11.59ms, AdaBoost takes 10.28ms, Logistic Regression takes 0.46ms, the SVM takes 0.27ms, and the Bernoulli Naive Bayes takes 0.41ms. So, based off of a combination of training performance and prediction speed, it seems like the Neural Network, Random Forest, AdaBoost, and Logistic Regression are worth investigating further. Figure 3: Results for Single Obstacle Controller with 100% Braking & 20% Steering Now, let s investigate the training data and see what it looks like. For this data, we want to be able to see how optimal control may look like in different circumstances. Based off of what we know about braking from Section 2.1, for a given difference in speeds between vehicles, there should be some threshold distance that causes one to switch from avoiding collision to colliding. To visualize this, let s look at Figure 3. In this plot, we see the end result given the starting distance between vehicles (in meters) and the initial difference in speeds (in km/hr, where positive means the host 15

vehicle is faster than the obstacle vehicle). Collisions are visualized in green, going offroad in red, both colliding and going offroad in purple, and the safe avoidance of both in blue. Each similarly looking plot in the future can be interpreted identically, except that it will likely be showing the results for a different controller. In this figure, we are looking at the results from using a controller that utilizes brakes at 100% and steering at 20%. As we can see, there seems to be a linear boundary between the blue and green samples; this is the boundary between collision and avoiding collision, coined the collision boundary from now on. So, if we move to the right in the figure and increase our relative speed to the obstacle car, we see that the vehicle needs to start braking at a much larger starting distance in order to avoid collision. This exactly matches our expectations as described in Section 3.1. Additionally, if we look at Table 6 in the Appendix, we can observe trends in this data as brake and/or steering control changes. As the amount of brakes used increases, we see a decrease in the number of collisions regardless of what our steering control is. Similarly, we generally see a decrease in the number of collisions as steering increases. However, the higher the steering the more likely we ll go offroad. So the impact of steering on the probability of collision OR going offroad follows more of a U-shaped curve. If we increase steering from 0 to 1 while holding brakes constant, we see that the probability of collision/offroad quickly goes down and then rises back up. No matter the level of braking (so long as we hold it constant while varying the steering), we see this U-shaped curve, though the magnitude of this curve diminishes as we look at higher levels of braking. That is, with higher brakes we still see the U-shape, but we have a lower probability of going offroad for all choices of steering. Naturally, these trends seem to suggest that there is some optimal control. Simply maximize braking and find the steering control at the bottom of the associated U-curve for steering. As we can see in Table 6, this leads to a controller with 100% brakes and 20% steering. This is the controller plotted in Figure 3. Now, we can also graphically investigate these trends by looking at Figure 5 in the Appendix. Here, we see that increasing brakes decreases the slope of collision boundary. Additionally, we see that increasing steering generally increases the proportion of red samples, i.e., the proportion 16

Method % Collisions % Offroad % Collision % Collision OR Offroad AND Offroad Proposed Controller w/random Forest 11.78% 1.30% 12.75% 0.32% Proposed Controller w/neural Network 12.02 1.62 13.15 0.49 100% Brakes, 20% Steering 13.39 0.79 13.78 0.40 Basic Autonomous Emergency Steering 13.28 11.62 20.75 4.15 Autonomous Emergency Braking 22.69 0.84 22.69 0.84 Table 2: Comparison of Controller Performance for Single Obstacle Vehicle of events where we go offroad. In addition, we can also see that there are some instances with remarkably few collisions but a lot of offroad incidents, and vice-versa. So, different sections of the plot (as in, different combinations of starting distance and speeddifference) have different optimal control levels. If we have a low speed difference and a high starting difference, it would make the most sense to simply apply the brakes and avoid any complications with steering. So, we can see how a rapidly readjusting controller may be able to offer improvement. 6.2. Risk Minimization Now, let s see how well we do at actually minimizing the risk involved. Essentially, this boils down to evaluating the system as a whole and by investigating some design decisions. As we can see in Table 2, the optimal combination of brakes and steering found earlier is able to outperform both Autonomous Emergency Braking and Autonomous Emergency Steering in nearly every category (the only exception is that Autonomous Emergency Braking has slightly smaller rate of collision). Autonomous Emergency Braking was implemented by simply applying maximum brakes with ABS[2]. Autonomous Emergency Steering was implemented by steering into the neighboring lane with the optimal amount of steering magnitude/responsiveness while also using Traction Control[5]. Additionally, we can see that this optimal combination of brakes and steering seems to get the best of both Autonomous Emergency systems; the rate of collisions is comparable with Autonomous Emergency Steering and the rate of going offroad is comparable with Autonomous Emergency Braking. However, we see that the proposed controller that utilizes a Random Forest is able to reduce the rate of collisions by another 12.02%. However, the rate of going offroad increases, causing the overall improvement on the rate of collision OR offroad to be only 7.47%. In contrast 17

with the best of AEB and AES, we see that the Random Forest based controller reduces the rate of collision/offroad by 38.55%, from 0.2075 to 0.1275. Figure 4: Results for Proposed Single Obstacle Controller w/random Forest Lastly, we can look at the collision plot in Figure 4 to compare performance. When compared with Figure 3, we can see that the slope of the collision boundary has decreased. At a speed difference of 50 km/hr, we gain an extra 5m of space before a collision becomes unavoidable. In terms of time, this means we gain an extra 359ms to react appropriately and switch to collision avoidance mode. 7. Extended Implementation for Multiple Obstacles When driving in the real world, there are situations that resemble the single obstacle model. In these, we must avoid a single car directly ahead of us, and we are capable of swerving into an empty lane (either to the left or right, as the decision in the above implementation was arbitrarily made and we can easily detect which lane we re in via DeepDriving). This is especially true at night when few people are on the road (and DeepDriving has comparable performance at night, so this is a realistic use case). However, it is much more common that there are other vehicles in the surrounding lanes. 18

In order to train a more robust system that can handle such scenarios, we need to extend each of the steps used in the single obstacle implementation. 7.1. Generation of Training Data In addition to the obstacle vehicle directly ahead of the host vehicle, we extend the scenario generator to also place vehicles in the left and right lanes that have independently random initial distances from the host vehicle and independently random initial velocities. Additionally, the distributions used to sample the starting distance was changed; although still a uniform distribution, the range was changed from [0,200] to [0,100]. Similarly, the range for the distribution for starting velocities was changed from [0,130] to [0,100]. This was done to allow training samples to more realistically approximate typical driving conditions. Since the input space is much more vast and convoluted, we need to be more careful and more efficient in how we generate training data. We are going to need to simulate many, many more samples in order to gain any sense of good classification performance. Additionally, we can no longer arbitrarily choose to steer towards the left lane since there may be a vehicle there. So, we need to add an additional parameter to the randomized control behavior: the desired lane. 7.2. Risk Estimation With the extended implementation, the risk estimation step does not change much. We simply add the new control choices to the list and continue on as usual. However, we do need to account for unintended side-effects of allowing the vehicle to choose a desired lane. In the single obstacle vehicle setup, we simply need to avoid one vehicle, and we know that said vehicle starts off directly ahead of us. As such, we can simply pick one direction for steering and stick with it, as nothing should warrant a need to change directions. Now, however, there can be uncertainty in which lane we aim to go towards, and this uncertainty leads to choppy control output. That is, in one timestep it may choose to go left, while choosing to go right in the next timestep. The end result is that we go nowhere, which is undesirable. This occurs partially because we are purposefully ignoring all temporal data. In addition, it stems from the ambiguity left from the choice of classifier. Before, we 19

wanted a more discriminative classifier that would correctly determine whether we would collide or not in order to allow for more decisive decisions. Now, we actually care about modeling the some of the actual probabilities at hand, for directly modeling the probability allows for a smoother decision boundary. With a smoother decision boundary, we are much less likely to oscillate between choices since, on average, a larger jump in the input space is needed to cross the boundary if starting from any random point. In order to get a best of both worlds type of response, we also evaluate ensemble classifiers, where we estimate risk as the sum of the probabilities output from two classifiers. However, some oscillation will still occur, so additional extensions to the implementation were made to incorporate the current horizontal position on the road. This allows for decisions to be made conditional on the vehicle s horizontal position, hopefully causing the vehicle to be less likely to decide to swerve across multiple lanes. Lastly, we now have to look at a control space that is three times larger (due to the additional control variable for desired lane). As such, various memory management related optimizations were made in order to keep everything running in under 10ms. 7.3. Risk Minimization For the risk-minimization step, everything is identical. We still choose the control behavior with the minimum estimated risk. 8. Results for Multiple Obstacles 8.1. Risk Estimation As we can see in Table 3, classification performance is much worse in the multiple obstacle scenario. However, the relative performance between classifiers is consistent, and the balance in accuracy across classes is held. As such, we should be able to continue making discriminative decisions that lead to good results. Additionally, although the feature space has grown to be much larger, the average prediction times have only slightly increased due to the optimizations made for batch processing. 20

Classifier precision recall F1 precision + recall + F 1 + Avg. F 1 Neural Network 0.76 0.72 0.74 0.76 0.79 0.77 0.76 Random Forest Classifier 0.78 0.74 0.76 0.75 0.80 0.77 0.77 AdaBoost Classifier 0.76 0.71 0.73 0.76 0.71 0.73 0.73 Logistic Regression 0.74 0.73 0.73 0.73 0.74 0.74 0.74 Support Vector Machine 0.71 0.10 0.18 0.52 0.96 0.67 0.42 Bernoulli Naive Bayes 0.54 0.56 0.55 0.54 0.52 0.53 0.54 Table 3: Training/Risk Estimation Performance for Multiple Obstacle Vehicles Method % Collisions % Offroad % Collision % Collision OR Offroad AND Offroad Neural Network 27.72% 0.00% 27.72% 0.00% Neural Network w/horizontal 28.18 0.05 28.18 0.05 Random Forest + AdaBoost 28.86 0.00 28.86 0.00 Random Forest + Logistic Regression 29.72 0.00 29.72 0.00 Random Forest 32.27 0.01 32.27 0.00 100% Brakes, 20% Steering 42.57 6.34 43.37 4.54 Autonomous Emergency Braking 45.75 0.81 46.15 0.41 Basic Autonomous Emergency Steering 63.50 20.01 68.54 14.96 Table 4: Comparison of Controller Performance for Multiple Obstacle Vehicles 8.2. Risk Minimization As we can see in Table 4, adding an additional two vehicles to the mix increases the rate of collision substantially; the best controllers now have roughly 2.5 times as many collisions as before. However, we can also see that the proposed controllers now offer a much more substantial difference in performance when compared to the optimal combination of brakes and steering. Now, we have a 38.08% reduction in the rate of collisions/offroad. Additionally, we see that Autonomous Emergency Steering no longer outperforms brakes. This is to be expected, as Autonomous Emergency Steering would not normally engage in every simulation run. Although many of the simulations had enough room for a lane change to take place, some had the host vehicle surrounding. In the latter situation, a commercial Forward Steering Assist system would not engage. Lastly, we see that the performance of the Neural Network now outperforms the Random Forest, even when a Random Forest is used in ensemble classifiers. In theory, the ensemble classifiers increase the test performance by allowing for a more accurate estimate of the probabilities. That is, the increase in classifier variation caused by using two different methods of classification allows each to cover for the other s weaknesses. As 21

such, when making estimates near 0 or near 1, we are more certain that these estimated probabilities are correct. Since the risk minimization step is choosing between many control choices with near zero probability, this allows that classifier to more accurately determine the best choice in the long run. As we can see in the table, adding AdaBoost or Logistic Regression to a Random Forest increases the test performance significantly. Interestingly, though, the Neural Network actually outperformed each ensemble method. As we increase the dimensionality of the features used, neural networks tend to get better at evaluating the effects of interaction terms at a faster rate than tree based classifiers, due to the interconnectedness between every neuron across layers. However, this comes at the cost of needing more training samples to train those effect estimates. Since we have 22409 samples in this training set, the Neural Network is able to reign supreme. In addition to the rates of collision and going offroad, we also investigate the performance in regards to maximum and average acceleration experienced. Since the main point of avoiding collisions/offroad is to keep the occupants of the vehicle safe, we also want to ensure that the forces experienced in the collision avoidance sequence are tolerable. As we can see in Table 5, we see a sharp increase in the typical max acceleration experienced with our system. This occurs when the controller makes the decision to sharply flip steering directions. Although roughly 1g larger, these accelerations are all well within the typical humans range of tolerance. For example, many amusement park rides operate at this level of acceleration [1]. In some cases, the controller would rapidly alternate between desired lanes, causing the vehicle to rapidly oscillate. In order to help stabilize the vehicle, an additional extension was made to the system. In addition to everything covered thus far, we extended the implementation in order to estimate where we are on the road in respect to our distance from the midline of the road. So, the simulation generator was modified to start the vehicle at random positions on the road. On the risk-minimization side of things, we used DeepDriving s estimates of where the lane markings are in order to determine where the center of the road is and fed that into the classifiers. The intuition behind this addition was that knowledge of our location with respect to the lanes would help stabilize the choice of desired lane. That is, if the left lane and right lane are equally safe at the 22

Method Mean Average Acceleration Mean Max Acceleration Neural Network 15.3004 26.4922 Neural Network w/horizontal 14.4773 26.5497 Random Forest + AdaBoost 13.9375 26.6499 Random Forest + Logistic Regression 15.1004 26.4374 Random Forest 10.1191 26.6735 100% Brakes, 20% Steering 11.3842 16.9672 Autonomous Emergency Braking 11.1257 16.5876 Autonomous Emergency Steering 4.5304 16.5843 Table 5: Comparison of Acceleration Experienced for Multiple Obstacle Vehicles moment, we can choose either when we re in the center lane. But if we re already in the right lane, then it will take less time to stay in the right lane than it would to swerve over to the left lane. As a result, staying in the current lane should have less risk (if all else held constant). Qualitatively, this is exactly what happens. The control becomes much smoother and the vehicle becomes much more likely to stay in the current lane. However, as we see in Table 4, this addition of the horizontal position actually decreased performance, though not significantly. This is likely due to the fact that we added an additional continuous variable to the mix, causing our Monte Carlo estimation to need many more samples. So, even though we generated 22409 samples (which took roughly 2 days of continuous simulation), this was not enough. With more training data, the Neural Network w/horizontal would likely increase its performance significantly. 9. Limitations and Future Work Although the system performs extremely well in simulation, it is still unknown how it will fare in reality. Although the physics engine used by TORCS is highly realistic, so results should translate well to reality, there may be subtle quirks to the system. So, future work may investigate how this system works outside of simulation. Additionally, this system purposefully avoided using temporal information. Utilizing temporal information to smooth the predicted affordance indicators from DeepDriving, or to smooth the control decisions being made will almost surely improve performance. This project was meant as a proof of concept, and a demonstration of what is possible. In that, including temporal information only serves to detract from the main system. Including 23

time in a more complex manner, such as through a Continuous-time Markov Decision Process, for example, would be a natural extension of the system that could potentially increase the performance substantially. However, there are many other possible choices for temporal integration, enough that such an investigation may warrant a paper in and of itself. Additionally, further work could extend the cost functions used in risk estimation. Currently, the system penalizes collisions and going offroad equally. Additionally, it doesn t take predicted g-force/acceleration into account when choosing between similar control behaviors. Modifying the cost of going offroad depending on the road scenario may offer significant benefits, as there is a much lower risk associated with going offroad into a field when compared to driving off the side of a mountain. 10. Conclusion Although there have been a number of advances in active vehicle safety, such as Autonomous Emergency Braking[2] and Steering[5], there is significant room for improvement. By using a risk-minimization approach that utilizes a neural network trained from simulation and re-evaluates its control decision 30 times per second, we are able to significantly improve upon Autonomous Emergency systems. In single obstacle scenarios, we see that the proposed controller that utilizes a Random Forest is able to reduce the rate of collisions by 38.55%, from 0.2075 to 0.1275 when compared to the best of AEB and AES. In multiple obstacle simulation scenarios, we are able to reduce the rate of collision/going offroad by 39.41% when compared to the best of AEB and AES. For these multiple obstacle simulations, we reduce the overall rate of collision/going offroad from 46.15% for Autonomous Emergency systems to 27.72% with our system. Although there is much more room for further improvement through the use of more complex cost-estimates and more complex temporal systems, these results seem very promising for future work in simulation-guided collision-avoidance systems. 24

References [1] G. Bibel, Beyond the Black Box: the Forensics of Airplane Crashes. Johns Hopkins University Press, 2008. [2] J. Bond et al., Autonomous emergency braking system, Feb. 25 2003, us Patent 6,523,912. Available: https://www.google.com/patents/us6523912 [3] C. Chen et al., DeepDriving: Learning affordance for direct perception in autonomous driving, in ICCV, 2015. [4] A. for Safe International Road Travel, Road crash statistics. Available: http://asirt.org/initiatives/ Informing-Road-Users/Road-Safety-Facts/Road-Crash-Statistics [5] K. M. S. K. S. K. S. D. H. S. K. Jeon, Dae Seok (Hwaseong, Vehicle collision avoidance apparatus and method, November 2015. Available: http://www.freepatentsonline.com/y2015/0329112.html [6] N. A. of Insurance Commissioners, Auto Insurance Database Report. National Association of Insurance Commissioners, 2015. [7] G. J. S. Wilde, The theory of risk homeostasis: Implications for safety and health, Risk Analysis, vol. 2, no. 4, pp. 209 225, 1982. Available: http://dx.doi.org/10.1111/j.1539-6924.1982.tb01384.x [8] B. Wymann et al., TORCS, the open racing car simulator, v1.3.5, http://www.torcs.org, 2013. A. Results Appendix 25

% Brake % Steering %Collisions % Offroad % Collision Mean Average Mean Max & Offroad Acceleration Acceleration 0.0 0.0 0.5812 0.1026 0.5983 3.3781 15.4518 0.0 0.2 0.166 0.1581 0.2648 4.4456 15.6121 0.0 0.4 0.1328 0.1162 0.2075 5.251 15.3107 0.0 0.6 0.1037 0.3237 0.3568 7.6989 16.4157 0.0 0.8 0.1508 0.4048 0.4643 8.4866 16.3708 0.0 1.0 0.126 0.4252 0.4646 8.0334 16.2667 0.2 0.0 0.4818 0.0931 0.5061 4.3915 15.6341 0.2 0.2 0.2014 0.0973 0.2557 5.4357 15.5538 0.2 0.4 0.1369 0.1667 0.2421 7.2001 15.9844 0.2 0.6 0.1322 0.3285 0.3884 8.5387 15.9764 0.2 0.8 0.1189 0.3864 0.4416 9.1616 15.8828 0.2 1.0 0.1174 0.4494 0.4899 10.0546 16.4605 0.4 0.0 0.4329 0.0779 0.4589 6.075 15.6565 0.4 0.2 0.1852 0.0697 0.2135 6.9905 15.7416 0.4 0.4 0.1571 0.1699 0.2675 8.5882 16.0338 0.4 0.6 0.1483 0.323 0.4187 10.1364 16.2179 0.4 0.8 0.1235 0.334 0.4089 10.1299 15.959 0.4 1.0 0.1266 0.384 0.4473 10.7368 16.2694 0.6 0.0 0.3409 0.0379 0.3523 8.2958 15.776 0.6 0.2 0.1904 0.0361 0.2044 8.5319 15.5638 0.6 0.4 0.1687 0.1423 0.2581 9.8554 15.914 0.6 0.6 0.1303 0.2811 0.3544 10.8287 16.2001 0.6 0.8 0.1128 0.3652 0.4149 11.2364 16.0562 0.6 1.0 0.1255 0.4049 0.4777 11.3197 15.9501 0.8 0.0 0.272 0.008 0.276 10.2934 15.8682 0.8 0.2 0.1239 0.0441 0.1555 10.6497 15.994 0.8 0.4 0.1231 0.0977 0.1868 11.1757 15.8737 0.8 0.6 0.1545 0.1829 0.3028 11.6541 16.0912 0.8 0.8 0.1207 0.2823 0.3491 12.0566 16.388 0.8 1.0 0.1563 0.293 0.4063 12.3934 16.7084 1.0 0.0 0.2269 0.0084 0.2269 10.961 15.8837 1.0 0.2 0.1339 0.0079 0.1378 11.6677 16.2529 1.0 0.4 0.1548 0.1297 0.2469 11.7257 16.3783 1.0 0.6 0.1473 0.1741 0.2857 12.0356 16.4335 1.0 0.8 0.1647 0.1606 0.2691 12.3708 16.4607 1.0 1.0 0.125 0.2083 0.3083 12.6826 16.4198 Table 6: Results for Each Control Behavior with Single Obstacle Vehicle 26

% Brake % Steering %Collisions % Offroad % Collision Mean Average Mean Max & Offroad Acceleration Acceleration 0.0 0.0 0.7235 0.2045 0.7652 3.8293 16.6177 0.0 0.2 0.635 0.2 0.6854 4.5304 16.5843 0.0 0.4 0.7053 0.277 0.778 5.6762 16.7712 0.0 0.6 0.6906 0.3633 0.8024 6.6654 16.7958 0.0 0.8 0.6461 0.4906 0.8127 7.4805 17.2521 0.0 1.0 0.5569 0.5894 0.8293 7.9725 17.6129 0.2 0.0 0.6772 0.1101 0.7015 4.7437 16.4873 0.2 0.2 0.5895 0.1733 0.6412 5.781 16.5277 0.2 0.4 0.6134 0.2636 0.6977 6.9384 16.8033 0.2 0.6 0.5535 0.3985 0.7177 7.8668 16.7788 0.2 0.8 0.4652 0.4887 0.7189 8.4584 16.8354 0.2 1.0 0.4476 0.53 0.7303 9.1444 17.0893 0.4 0.0 0.5932 0.0701 0.6103 6.1754 16.2104 0.4 0.2 0.5394 0.1273 0.5811 7.3499 16.4959 0.4 0.4 0.5053 0.2483 0.6044 8.2727 16.4595 0.4 0.6 0.4727 0.404 0.6544 9.087 16.8337 0.4 0.8 0.4257 0.4626 0.6651 9.4783 16.9033 0.4 1.0 0.3761 0.5083 0.6771 9.7743 17.0577 0.6 0.0 0.5057 0.0556 0.523 7.9863 16.2086 0.6 0.2 0.4585 0.0899 0.4809 8.8949 16.4634 0.6 0.4 0.4779 0.2366 0.5626 9.6555 16.6234 0.6 0.6 0.4699 0.3636 0.6325 10.081 16.8808 0.6 0.8 0.42 0.42 0.6226 10.2362 16.9258 0.6 1.0 0.4316 0.4526 0.6484 10.3878 16.795 0.8 0.0 0.499 0.0251 0.5087 10.2139 16.5762 0.8 0.2 0.4343 0.0657 0.4526 10.5001 16.7108 0.8 0.4 0.4608 0.1874 0.522 10.9187 16.8411 0.8 0.6 0.4315 0.3118 0.583 11.0678 16.9769 0.8 0.8 0.4496 0.3443 0.5998 11.0725 16.9673 0.8 1.0 0.438 0.3872 0.6147 11.1793 17.1959 1.0 0.0 0.4575 0.0081 0.4615 11.1257 16.5876 1.0 0.2 0.4257 0.0634 0.4337 11.3842 16.9672 1.0 0.4 0.4312 0.1691 0.4926 11.369 16.95 1.0 0.6 0.4152 0.2381 0.5048 11.2758 16.9032 1.0 0.8 0.4632 0.2926 0.5756 11.2182 17.1049 1.0 1.0 0.4142 0.3389 0.5858 11.5138 17.222 Table 7: Results for Each Control Behavior with Three Obstacle Vehicles 27

28 Figure 5: Training Facet Plot For Single Obstacle