Brian G. Soublet Chief Counsel California Department of Motor Vehicles 2415 1st Ave Sacramento, CA 95818-2606 Dear Mr. Soublet: The California Department of Motor Vehicles (DMV) has requested comments on its notice of proposed regulations for the testing and deployment of autonomous vehicles on public roads. The Insurance Institute for Highway Safety (IIHS) commented on two previous draft versions of these regulations and urged the DMV to provide more guidance concerning the information companies report about crashes and disengagements. We are pleased that the California DMV revised the regulation to include more details about the information that companies must provide when vehicles disengage from autonomous mode or crash while in autonomous mode. These improvements will generate consistent, reliable, and precise reporting of safety and performance data that California and other stakeholders can use to evaluate automated driving systems. However, there are additional areas where the proposed regulations can be strengthened. We additionally identified several areas where clarification is needed. Our main comments focus on the five following topics: 1. Information collected about disengagements should be as consistent as possible with information collected about crashes involving vehicles with automated driving systems; 2. The autonomous vehicle testing permits of companies who fail to meet reporting requirements for disengagements from autonomous mode or crashes involving vehicles with automated driving systems should be suspended or revoked; 3. Companies should be provided with a minimum set of variables that they must record and store in the moments surrounding a crash using an autonomous vehicle data recorder; 4. Manufacturers of vehicles with level 2 driving automation systems should submit a Safety Assessment Letter; and 5. Manufacturers deploying autonomous vehicles following testing should be responsible for ensuring that vehicles with automated driving systems can be operated only with the most recent safety-critical updates. Collect similar information about disengagements and crashes. We are pleased that the California DMV describes the information that manufacturers must include in their annual disengagement report in more detail. More precise reporting requirements would improve the consistency of this information among manufacturers. The DMV can further improve the reporting of disengagements by incorporating variables from the revised OL 316 form, which collects contextual information about crashes in a more structured and repeatable way than is currently requested when manufacturers describe the circumstances leading to disengagements. For example, form OL 316 includes a list of attributes for weather, roadway conditions, lighting, type of collision, and movement preceding the collision that also can be used to describe the facts surrounding disengagements. Using a consistent set of variables and attributes to describe crashes and safety-related disengagements will lead to a more robust and precise dataset that will facilitate the monitoring and evaluation of automated driving system performance and safety. It also will help the DMV and stakeholders determine whether safetyrelated disengagements provide any meaningful information about crashes involving vehicles with automated driving systems.
Page 2 Suspend or revoke the testing permits for companies that do not meet reporting requirements. It is critical that companies testing automated driving systems thoroughly report information about disengagements and crashes in a timely manner. Companies are required to submit an annual disengagement report and report crashes involving vehicles with automated driving systems within 10 days, but the regulations do not specify any consequences if a company does not meet these reporting requirements. The California DMV should suspend or revoke the autonomous vehicle testing permits of companies that fail to report crashes and disengagements in a timely manner or in sufficient detail. Adding these stipulations will encourage companies to diligently report key information necessary for monitoring the safety and performance of these systems. Specify a minimum set of variables to be captured by an autonomous vehicle data recorder. The information stored on an autonomous vehicle data recorder following a crash will be tremendously valuable for determining the factors that contribute to crashes involving vehicles with automated driving systems and how these factors differ from crashes involving conventional vehicles. Currently, the DMV requires that the autonomous vehicle data recorder capture and store sensor data for all vehicle functions controlled by the driving automation system at least 30 seconds before and at least 5 seconds after a collision, or until the vehicle reaches a stop. Manufacturers of vehicles with automated driving systems may have different perspectives about the information that should be stored using an event data recorder. The California DMV should provide guidance to manufacturers about which variables, at a minimum, need to be stored on an autonomous vehicle data recorder. Attached is a list of variables that IIHS believes will help the DMV and other stakeholders understand the chain of events leading up to a crash involving a vehicle with one or more driving automation systems. More importantly, these variables are specified at a coarse level which should be accessible to law enforcement or other persons without technical expertise without putting proprietary information at risk. Manufacturers of level 2 driving automation systems should submit a Safety Assessment Letter. In our comment on the Federal Automated Vehicles Policy, we recommended that the National Highway Traffic Safety Administration (NHTSA) ask companies deploying level 2 driving automation systems to submit a Safety Assessment Letter. Likewise, we recommend that the California DMV request a Safety Assessment Letter from companies testing and deploying level 2 driving automation systems. IIHS is particularly concerned that the distinction between level 2 and level 3 systems may not be apparent from a user s point of view. Vehicles with level 2 driving automation should have safeguards that keep the human driver fully engaged in the driving task, and that clearly differentiate level 2 driving automation from more capable automated driving systems. Manufacturers can provide evidence that they have implemented these and other safeguards in the Safety Assessment Letter. Disagreements about the levels of automation between manufacturers and the DMV will only exacerbate confusion. Currently, manufacturers self-certify the level of driving automation for each system, and they only apply for a testing permit in California if they judge a vehicle system to be a SAE 3 level or higher system. However, manufacturers may interpret the SAE levels of automation differently from the DMV and one another, or they may use the inherent ambiguity of the SAE levels to understate the technical capabilities of the system in their test vehicles to avoid DMV regulations. Asking manufacturers of level 2 driving automation systems to submit a Safety Assessment Letter will make the self-certification process more transparent, provide the DMV with greater oversight over the driving automation systems being tested and deployed on public roads, and discourage manufacturers from understating the capabilities of the system on their vehicles to circumvent regulation. Manufacturers deploying autonomous vehicles following testing should update automated driving systems with the most recent safety-critical updates. The proposed regulations state that the registered owner of an autonomous vehicle is responsible for ensuring the vehicle is operated using the manufacturer s most recent updates. It is challenging to get vehicle owners to voluntarily address vehicle defects and recalls, so it is unreasonable to rely on owners
Page 3 to apply safety-critical updates to automated driving systems. The manufacturer should be responsible for ensuring that safety-critical updates are applied to automated driving systems and prohibit the use of automated driving systems that are out-of-date. Owners should be responsible for applying non-safety critical updates (e.g., convenience features). In an application to the DMV for post-test deployment, the manufacturer should describe its process for delivering safety-critical updates to vehicle owners and how the use of outdated automated driving systems will be prevented. In addition to our above comments, we have identified some areas of the regulation that need clarification. 1. The California DMV will require test vehicles without a driver to submit a copy of the Safety Assessment Letter provided to NHTSA, but test vehicles with a driver are not required to do the same until post-test deployment. The DMV should require manufacturers applying to test or deploy any vehicle equipped with an automated driving system on public roads to provide the Safety Assessment Letter described in the Federal Automated Vehicles Policy. 2. In section 227.50.b, it is not clear if manufacturers are required to summarize disengagements for all test vehicles in aggregate or for each test vehicle. We recommend that the DMV amend section 227.50.b.3. to The annual report shall summarize disengagements for each permitted test vehicle for each month as follows: 3. Most automated driving systems currently rely on digital maps to support automated driving functions, but this may not be the only method for identifying changes to the physical environment. In section 288.06.8.B, we suggest that the department broaden the phrase changes to the physical environment captured by the maps to changes to the physical environment captured by maps, sensors, or other sources of information. In summary, IIHS supports the California DMV s proposed regulations for the testing and deployment of autonomous vehicles on public roads and feels these regulations provide a strong model for other states to follow. The revised text will lead to more precise, consistent, and reliable gathering of information about the operation of automated driving systems on public roads. The extent to which the California DMV can collect similar information about crashes and safety-related disengagements and encourage the thorough and timely reporting of this information from testing companies will further enhance its value. We hope that the California DMV finds the attached list of variables helpful when considering a minimum set of variables to collect using an autonomous vehicle data recorder following a crash. Finally, collecting a Safety Assessment Letter from manufacturers testing and deploying level 2 systems will help the DMV monitor the deployment of driving automation, assess if companies have implemented appropriate safeguards to reduce driver misuse and abuse, and discourage companies from self-certifying systems as level 2 driving automation to evade regulations. Sincerely, David Kidd, Ph.D. Senior Research Scientist Attachment Insurance Institute for Highway Safety. undated. Suggested variables to include in event-based electronic data recording for vehicles equipped with one or more driving automation systems. Arlington, VA.
Page 4 Suggested variables to include in event-based electronic data recording for vehicles equipped with one or more driving automation systems IIHS recommends that the following variables be recorded by an event data recorder or autonomous vehicle data recorder when an autonomous vehicle is involved in a crash. At a minimum, each variable below should be recorded every second during the period beginning 30 seconds before a crash and ending 5 seconds after, or until the vehicle comes to a stop. Some variables are currently recorded by event data recorders in conventional vehicles. It may be appropriate to record some variables more frequently. Definitions: - State - a categorical variable indicating if a vehicle system is off or on, its current setting (e.g., standby mode, low beam, high beam), or if the system is not functioning (e.g., failure mode). - Action - a categorical variable indicating when a restraint system, advanced driver assistance system, driver monitoring system, or driving automation system is warning, intervening, deploying, or responding to a safety-critical event. Category Time and history Location and path Vehicle state and kinematics Crash prevention, driver assistance, and restraint systems Variable Timestamp Ignition cycle count since being manufactured Latitude Longitude Elevation Heading Speed Steering input (torque or wheel angle) (overall, amount applied by driver, amount applied by automation) Brake position/input (overall, amount applied by driver, amount applied by automation) Throttle position/input (overall, amount applied by driver, amount applied by automation) Lateral acceleration Longitudinal acceleration Roll angle Transmission state (P (park); R (reverse); N (neutral); D,L (forward/drive)) Windshield wiper state Exterior lights state Engine RPM Antilock brake system state and action Electronic stability control state and action Front crash prevention system (e.g., forward collision warning, automatic emergency braking) state and action Rear crash prevention system (e.g., parking sensor, rear automatic emergency braking) state and action Lane change crash prevention (e.g., blind spot warning, blind spot intervention) state and action Lane maintenance system (e.g., lane departure warning, lane departure prevention, active lane keeping) state and action Frontal airbag state and action
Page 5 Vehicle occupant state Automated driving systems (e.g., self-parking, highway autopilot, traffic jam assistant) * these variables are collected for each equipped level 2-5 driving automation system even if the system is not in use for any reason (e.g., outside the operational design domain, driver choice) V2V basic safety message data for each message broadcasted and received V2I safety message data for each message broadcasted and received Side airbag state and action Safety belt pretensioner state and action for each occupied seating position Driver fatigue monitoring system state and action Hands-on wheel detection state and action Driver monitoring system (e.g., eyes on or off road) state and action Occupant presence for each seating position Safety belt state for each occupied seating position Occupant size classification for each occupied seating position OEM defined SAE level of automation for each equipped system Vehicle within or outside intended or specified operational design domain for each equipped system State of each equipped system Transition of control/take-over message action for each equipped system Time Message count Temporary ID Position data (latitude, longitude, elevation) Positional accuracy (semi-major axis accuracy, semi-minor axis accuracy, semi-major axis orientation) Transmission state Speed Heading Steering wheel angle Acceleration (longitudinal, lateral, vertical, yaw rate) Brake system state Vehicle size (width, length) Signal phase and timing message data Signal request message data Signal state message data Map message data Emergency vehicle alert message data Intersection collision avoidance message data Personal safety message data (vulnerable road user data) Road side alert message data Traveler information message data