Deliverable D53.3 Temporary Auto Pilot: 1 st System Functionality

Transcription

1 Highly automated vehicles for intelligent transport 7th Framework programme ICT ICT for intelligent vehicles and mobility services Grant agreement no.: The future of driving. Deliverable D53.3 Temporary Auto Pilot: 1 st System Functionality Version number Version 0.4 Dissemination level Lead contractor CO Continental Automotive GmbH Due date Date of preparation

2 Authors Name Thanh Binh To Arne Bartels Company VW VW Project Managers Prof. Dr. Alfred Hoess Continental Automotive GmbH Siemensstrasse Regensburg, Germany Phone Telefax Holger Zeng Continental Automotive GmbH Siemensstrasse Regensburg, Germany Phone Fax Project Co-ordinator Dr. Reiner Hoeger Continental Automotive GmbH Siemensstrasse Regensburg, Germany Phone Fax Copyright: HAVEit Consortium st System Functionality 2

3 Revision and History Chart Version Date Reason Template and table of contents Adaptation to TAP Corrected version after reviewing Final editing and submission to EC Generation of reduced deliverable version intended for publication 1st System Functionality 3

4 Table of Contents Table of Contents... 4 List of Figures... 6 List of Tables... 7 Executive Summary Introduction Temporary Auto Pilot Functionalities TAP preconditions for activation Pilot function activation by the driver Pilot function deactivation by driver Automatic deactivation of the pilot function caused by freeway Automatic deactivation of the pilot function caused by lane markings Automatic deactivation of pilot function caused by driver in the loop assessment Stop & Go in a traffic jam Architecture Generic HAVEit architecture TAP specific architecture extensions E/E architecture and bus structure Demonstrator configuration Perception layer Camera for the lane detection Lane curvature functionality Ego lateral offset and the lane width estimation Laser scanner Object detection Free area detection Long-range radar sensor ehorizon and Map Matching HandsOn detection Sensor Data Fusion Object Data Fusion Lane Data Fusion Grid Fusion Command layer: Joint System Target Selection Unit Driver State Assessment Mode Selection Unit Longitudinal Controller Lateral Controller Execution layer: actuators Longitudinal control actuators Lateral control actuators Driver interface Driver Monitoring System Human Machine Interface st System Functionality 4

5 3.4.3 Function control buttons System integration Preliminary TAP system validation Relevant use cases and scenarios Scenario 1: ACC activation by the driver Description of scenario Test results Scenario 2: Activation of the pilot function by the driver Description of scenario Test results Scenario 2: Automatic deactivation of the pilot function caused by driver in the loop assessment (Minimum risk manoeuvre) Description of scenario Test results Scenario 3: Deactivation of the pilot function by the driver Description of scenario Test results Full driving test with complex scenarios Description of the full driving test Free mode Follow to Stop and Stop & Go mode Minimum Risk State Emergency Stop Additional Software-in-the-Loop test cases Operator control action in ACC for free driving Operator control action in the pilot function for free driving Free driving in the pilot mode on a curve Follow a moving object in ACC mode on straight-forward road Follow a moving object in the pilot mode on straight-forward road Follow a moving object in ACC mode on a curve Follow a moving object in the pilot mode on a curve Automatic Go in the pilot mode Lane changes of a preceding object in the pilot mode Object driving past on the left lane in the pilot mode Ego driving past on the left lane in the pilot mode Shortfall of a lane marking in the pilot mode Ego lane changes in the pilot mode Slowly overtaking on the right in the pilot mode Conclusion References Annex 1: Keywords Annex 2: Symbols st System Functionality 5

6 List of Figures Figure 1: TAP functionalities... 9 Figure 2: Architecture overview of the HAVEit system Figure 3: TAP specific architecture extensions Figure 4: Components connection Figure 5: Lane curvature and curvature change at the Volvo high-speed track Figure 6: Lane offset and lane width estimation Figure 7: Object detection by SICK laser scanner Figure 8: Free area detection by laser scanners Figure 9: Object detection by the long range radar sensor Figure 10: Stationary and moving object detection by radar Figure 11: ehorizon and Map Matching Figure 12: HandsOn Detection and its confidence Figure 13: Sensor Data Fusion architecture Figure 14: Object and Lane Data Fusion Figure 15: Grid Fusion Figure 16: Target Selection in the cut-in and cut-out situation Figure 17: Mode Selection Unit Figure 18: Longitudinal Controller Figure 19: Time gap in the follow to stop situation Figure 20: Lateral Controller Figure 21: Ego lateral distance to the lane middle for the driving in the straight-forward road Figure 22: Lateral behavior on the curve Figure 23: A snapshot of the GUI for Driver Monitoring System Figure 24: Head position tracking Figure 25: Driver drowsiness and distraction detected by the camera Figure 26: HMI concept Figure 27: Take-Over Request or Take HandsOn in the Pilot mode Figure 28: ACC lever Figure 29: ACC activation by driver Figure 30: Pilot mode activated by driver and the corresponding HMI display Figure 31: Automatic deactivation of the pilot function by MSM Figure 32: Manual deactivation of the pilot function by driver Figure 33: Driving scenarios for a full test Figure 34: TAP states and ego vehicle data for a full test Figure 35: Object and lane data for a full test st System Functionality 6

7 List of Tables Table 1: Input data for the Object Data Fusion Table 2: Input data for Lane Data Fusion Table 3: Explanation to Figure 26 (a) (e) Table 4: ACC-lever manipulation and ACC system response st System Functionality 7

8 Executive Summary The overall objective of the HAVEit project is to develop technical systems and solutions that improve automotive safety and efficiency. Volkswagen Group Research contributes to the overall objective by developing the safety and comfort focused Temporary Auto Pilot application. The Temporary Auto Pilot (TAP, WP5300 in the HAVEit project structure) is fundamentally intended to support the driver in monotonous traffic situations like traffic jams or monotonous long distance driving from A to B where the driver can experience work underload which can lead to a lack of focus and increased accident risk. The TAP is a passenger car application which will support the driver on motorways and motorway similar roads with different levels of automation in longitudinal and lateral control of the vehicle at speeds between 0 and 130 km/h. This guarantees that the driver gets the best possible support available. This will contribute to traffic safety. This document summarizes the 1 st overall system functionality of the complete TAP system after previously installing all sensors and components in D53.1 [4] and basically testing them separately as documented in D53.2 [5]. Firstly, this deliverable recalls the general TAP functionalities and typical use cases including activation, deactivation, the minimum risk state and Stop & Go in traffic jams as well as the overall generic HAVEit architecture and the TAP specific extensions. The third chapter describes the configuration of the TAP demonstrator in detail following the commonly used layer structure with perception layer, command layer, execution layer and driver interface. All relevant components of the above mentioned layers are described in more detail than in previous deliverables by considering and estimating the functional performance and contribution of each component to the overall TAP system functionality. Finally, a first preliminary TAP system validation and achieved validation results are described. The validation is done with reference to the previously defined use cases and scenarios as defined in deliverable D11.1 [1]. Use cases involve and cover all the described system layers and thus give a good overview about the currently achieved 1 st overall system functionality of the TAP system in the demonstrator. 1st System Functionality 8

9 1 Introduction The intention of this document is to report intermediate results from testing and validating of the system functionality in the HAVEit WP5300 demonstrator the Temporary Auto Pilot (TAP) vehicle. The purpose of the Temporary Auto Pilot is to support the driver in monotonous traffic situations like traffic jams or monotonous long distance driving from A to B where the driver can experience work underload which can lead to a lack of focus and increased accident risk. The testing of the full system includes both individual testing of each component in the system as well as of the complete system. The complete system tests aim to evaluate the system performance in the relevant use cases described below. A more detailed description can also be found in section 2.1. System functionality testing (see Figure 1, D53.2 [5]): Driver-Assisted (Assistance Function) o assistance in lateral control (Lane Keeping System) o hands-on driving Semi-Automated (ACC - Adaptive Cruise Control) o assistance in longitudinal control only Highly Automated (Pilot Function) o automated driving on the motorway (automated longitudinal and lateral control) o hands-off driving Minimum Risk Manoeuvre (Safety Function) o emergency brake if the driver does not respond to Take-Over Request (TOR) Figure 1: TAP functionalities 1st System Functionality 9

10 System functionality means the whole chain from system inputs through the algorithm, to the vehicle control and the Human-Machine-Interface (HMI). System inputs are received from the environment and vehicle sensors as well as from driver input. Environmental sensor components are described in deliverable D53.1 [4]. A description of all other components in the system can be found in deliverable D53.2 [5]. The TAP demonstrator is built on the common HAVEit architecture which is described in deliverable D12.1 [3]. The functionality and architecture of the TAP system is described in section 2. Section 3 describes the demonstrator configuration and results of component testing while section 4 covers the TAP intermediate system results. Conclusions are summarized in section 5. 1st System Functionality 10

11 2 Temporary Auto Pilot The Temporary Auto Pilot will support the driver when driving in monotonous traffic situations like traffic jams or monotonous long distance driving on motorways by offering integrated longitudinal and lateral control. This means that the TAP system can continuously support the driver by automatically steering, accelerating and braking the vehicle when the function is activated. The aim of TAP is to increase safety by combining the strengths of the driver and the strengths of the TAP system. 2.1 Functionalities The TAP system follows the HAVEit generic key functionality definition described in deliverable D11.1 [1]. The level of automated control will be continuously adapted based on the states of driver, vehicle and environment. At the highest level of automation, the system will automatically handle steering, accelerating and braking to keep the vehicle in the middle of the lane and at a safe distance to the preceding vehicle. The subsections below describe the most important use cases and the functionality of the TAP system. Each subsection begins with one use case and the requirement connected to that specific use case. A more detailed and thorough description of the use cases is found in deliverable D11.1. These use cases are the basis for the test scenarios evaluated in section 4. In this section, some symbols are used for the TAP use case description (see Annex 2) TAP preconditions for activation The highest level of automation (pilot function) can be activated, if at least several preconditions have to be fulfilled: a) subject vehicle is driving b) driver on a motorway with V subject V max, pilot in the correct direction between the lane markings is not drowsy is not sleeping, passed out or dead is sitting on his seat has his seat belt fastened has not opened any door of the vehicle does not press the brake pedal 1st System Functionality 11

12 c) lane markings exist and are detected by the environment sensors d) motorway (in the next few kilometers) motorway does not terminate lane does not terminate no construction sites no exit proposed by the navigation system e) system no failures Pilot function activation by the driver Preconditions see Preconditions for activation above Driver in the loop with hands-on System status: ACC active and pilot function ready Triggers Manual transition to the pilot function by the driver via TAP switch Driver takes his hands from the steering wheel Reaction If TAP system recognizes hands-off then the subject vehicle is controlled by the TAP system Subject vehicle is longitudinally controlled with a desired velocity (free mode) or follows a target vehicle Subject vehicle is laterally controlled by keeping in the subject vehicle s lane 1st System Functionality 12

13 2.1.3 Pilot function deactivation by driver Preconditions System status: pilot function active Driver in the loop with hands-off Triggers Reaction Deactivation of pilot function by the driver via o applying a big enough steering momentum to the steering wheel, or o pressing the On/Off switch, or o pressing the brake pedal, or o unfastening the seat belt (in case of standstill), or o leaving the driver seat, or opening a door TAP initiates a Take-Over Request to the driver If TAP recognizes hands-on then TAP abandons the longitudinal and lateral control of the subject vehicle. TAP state is changed to Driver only Automatic deactivation of the pilot function caused by freeway Preconditions System status: pilot function active Driver in the loop with hands-off Triggers In the next few kilometers o motorway terminates o lane of subject vehicle terminates o construction site ahead o navigation system proposes to exit the motorway Reaction TAP initiates a Take-Over Request to the driver If TAP recognizes hands-on then it abandons the longitudinal and lateral control of the subject vehicle TAP state is changed to Driver only 1st System Functionality 13

14 2.1.5 Automatic deactivation of the pilot function caused by lane markings Precondition System status: pilot function active Driver in the loop with hands-off Triggers The visibility of the lane markings is insufficient because o no lane markings are on the road o sensors for the lane marking detection are blinded by light, or defect o sensors for the lane marking detection are blind because of rain, fog, etc. o sensors for the lane marking detection or the lane markings themselves are covered with dirt Reaction TAP initiates a Take-Over Request to the driver If TAP recognizes hands-on then TAP abandons the longitudinal and lateral control of the subject vehicle. TAP state is changed to Driver only Automatic deactivation of pilot function caused by driver in the loop assessment Precondition System status: pilot function active Driver in the loop with hands-off Triggers Driver monitoring system detects that the driver is o drowsy o sleeping Reaction TAP initiates a Take-Over Request to the driver If the driver does not respond to TOR then TAP is going to comfortably stop the subject vehicle on the current lane (Minimum Risk Manoeuvre). 1st System Functionality 14

15 2.1.7 Stop & Go in a traffic jam Precondition System status: pilot function active Driver in the loop with hands-off Triggers Target vehicle decelerates and accelerates between 0 and 40 km/h including standstill Reaction TAP adapts the velocity of the subject vehicle to maintain the time gap which was adjusted by the driver while keeping the subject vehicle in its lane 2.2 Architecture In this section, the HAVEit generic and TAP specific architecture will be described shortly. In the next section the architecture and the flexible software framework used for implementation of the functions in the TAP demonstrator will be presented in detail. The third section will describe the implementation of components on physical processing units as well as the connection between them Generic HAVEit architecture As described in the deliverable D53.2, the generic HAVEit architecture consists of four layers: perception layer, command layer, execution layer and driver interface component layer (see Figure 2). This modular architecture allows the developers to easily implement the highly automated vehicles and gives a possibility to extent or to exchange individual modules with other project partners. In the HAVEit architecture, three types of sensors are available: The environment sensors for observing the driving environment around the demonstrator like objects, obstacles and lane markings etc. The vehicle sensors, that measure the vehicle states such as vehicle speed and yaw rate etc. and The driver state monitoring sensor. 1st System Functionality 15

16 joint system HAVEit The input data from the two first sensors are further sent to the Sensor Data Fusion (SDF) in the perception layer, where they are fused and combined in an adequate way into one unambiguous view of the vehicle movement and its surroundings. However, the inputs from the driver are collected by the driver interface component layer via Driver Monitoring System (DMS) or via Human-Machine-Interface and further interpreted by the Driver State Assessment (DSA). The driver information together with the observed vehicle and driving environment states is used by the Mode Selection and Arbitration Unit (MSU) to select a suitable level of automation. Based on the information about the current level of automation from the MSU and data from Co- Pilot, the command layer decides either no, only longitudinal or both longitudinal and lateral support is activated and generates suitable control commands. Those control commands are sent to the different actuators controlling the powertrain, braking and steering of the vehicle. Driver interface components Driver Driver monitoring Environment sensors Vehicle sensors Sensor data fusion Perception layer Driver states assessment HMI Co-Pilot Command layer Mode selection unit automation level Command generation and plausiblization motion control vector Drivetrain control Execution layer Steering actuator Braking actuator Engine actuator Gearbox actuator Figure 2: Architecture overview of the HAVEit system TAP specific architecture extensions This section describes some specific architecture extensions for providing the best TAP functionalities while the number of physical control units is limited. To adapt the vehicle to HAVEit architecture, the different blocks of the generic HAVEit architecture must be clustered in a meaningful way (see the legend in Figure 3). The TAP architecture contains three main component types: ECUs of sensors or actuators or HMI Four EKF-PCs: o o o one for the Driver Monitoring System (CAF), one for the Sensor Data Fusion (VW) and Driver State Assessment (WIVW), Two for data logging and visualization (optional) 1st System Functionality 16

17 dspace prototype Autobox (Co-Pilot, Mode Selection Unit and Command generation and plausibilization). Moreover, there are some additional hardware components like Gateway (for transfer of data between different CAN bus systems), VGA-Switch (for monitoring) and Ethernet-Switch etc. Please note that in the current implementation the Driver State Assessment Module is integrated in our Sensor Data Fusion framework for debugging purposes, but it will be implemented in the CSC in the final version. Figure 3: TAP specific architecture extensions E/E architecture and bus structure All hardware components in the TAP architecture are connected and communicate together via the CAN bus systems with two different bus rates: 500kBaud/sec and 1MBaud/sec: Two Sensor-CANs with different bus rates for receiving all sensor data (the high-speed CAN-bus for laser scanners and the low-speed for other sensors) FusionCAN with 1MBaud/sec for the high-speed communication with the Autobox Figure 4 shows an overview about the connection between all components, from the sensors through PCs and prototype hardware components to actuators. EKF-PC1 contains the DMS software from CAF, which gets an image from a DMS camera, performs the image processing and provides a driver state evaluation (drowsiness and distraction). 1st System Functionality 17

18 Figure 4: Components connection All sensors are connected to the Sensor-CAN buses. Sensor Data Fusion algorithms are integrated in the EKF-PC2, which deliver as results a robust perception about the driving environment like ego-vehicle, lane and object data over the high-speed Fusion-CAN to the applications (Autobox). As mentioned in the previous subsection, the Driver State Assessment from WIVW will do fusion of the camera-based driver states (so-called direct method) with driver states estimated by an indirect method. That means, DSA needs not only driver information from the direct method, but some additional information of ego vehicle (steering wheel angle, lateral offset to the middle of lane, velocity etc.) and the information of the driving environment (state of the relevant object, road type, driving duration etc.) for the indirect estimation. Because this additional information is already available in Sensor Data Fusion, it is meaningful to integrate the Driver State Assessment in a so-called VW-DSA-Module of the Sensor Data Fusion Framework. The VW-DSA-Module has the two following tasks: Collecting all necessary input data for Driver State Assessment and sending them into CSC module over CAN in the final version Perform the full functionality of Driver State Assessment (e.g. it delivers the driver s drowsiness and distraction information). MSU and longitudinal and lateral controller are implemented in a prototype control unit Autobox, which is directly connected to the actuators over a CAN bus system. EKF-PC3 serves for providing ehorizon data and for visualization purpose. Further than the above mentioned important components, the TAP architecture includes also an additional computer EKF-PC4, which is connected to all CAN buses and serves for data logging only. The measuring of data recorded by the Data Logging PC can be further used for the software development and debugging purposes. 1st System Functionality 18

19 5 Conclusion This report shows that the TAP algorithms and the fully integrated system work well as intended. All units are implemented according to the HAVEit generic standard with just minor TAP specific adaptations. The communication and interaction between all the components work adequately together. In all scenarios derived from the TAP use cases the system reacted to the triggers in a satisfying way. Generally, the work package is progressing according to the time plan. The technical concept and the designed architecture are proved as valid. Most of the remaining work will be further optimization and validation of the system. 1st System Functionality 63

20 References [1] HAVEit deliverable D11.1 Function description and requirements, September 2008 [2] HAVEit deliverable D11.2 Specification, February 2009 [3] HAVEit deliverable D12.1 Architecture, February 2009 [4] HAVEit deliverable D53.1 Sensors installed in vehicle (1 st SW version), April 2009 [5] HAVEit deliverable D53.2 Temporary Auto Pilot: Components installed, working and tested, January 2010 [6] HAVEit deliverable D31.1 Co-driver command vector available (1 st version), 2009 [7] HAVEit deliverable D31.2 Co-driver command vector available (2 nd version), 2009 [8] HAVEit deliverable D32.1 Report on driver assessment methodology, 2009 [9] HAVEit deliverable D32.2 Model of driver behaviour for the assessment of driver s state, 2009 [10] HAVEit deliverable D33.2 Preliminary concept task Repartition, 2009 [11] HAVEit deliverable D33.3 Validation by simulation, 2009 [12] HAVEit deliverable D33.4 Algorithm (C-code, 1 st version) available for partners, 2009 [13] M. Mitschke, Dynamik der Kraftfahrzeuge, Springer-Verlag 2004; ISBN [14] O. Föllinger, Regelungstechnik, Dr. Alfred Hüthig Verlag 1985; ISBN [15] S. Steinmeyer, Handling contradictory sensor data in environment maps for advanced driver assistance systems, 4 th German Workshop Sensor Data Fusion, Lübeck, 2009 [16] The European ADASIS Forum consists of currently 30 members from the automotive industry and is coordinated by ERTICO. Further information is available on the internet: approach/ [17] A. Weiser, A probabilistic lane change prediction module for highly automated driving, 7 th International Workshop on Intelligent Transportation, Hamburg, 2010 (to be published) [18] MATLAB and Simulink, The Mathworks, Inc. [19] dspace Company, 1st System Functionality 64

21 Annex 1: Keywords ACC ADAS ADTF CAN Co-System CSC DA DDM DIM DMS DSA ECU GPS / DGPS HA HAVEit HMI I2V / V2V IMU MRM MSU NIR RAM RTK SA TOR UML Active/Adaptive Cruise Control Advanced Driver Assistance Systems Automotive Data and Time-Triggered Framework Controller Area Network Software components within the Joint System Chassis Safety Controller Automation level Driver Assisted Driver's Drowsiness Monitoring Driver s Inattentiveness Monitoring Driver Monitoring System Driver State Assessment Electronic Control Unit Differential Global Positioning System Automation level Highly Automated Highly Automated Vehicles for Intelligent Transport Human Machine Interface Infrastructure to Vehicle / Vehicle to Vehicle communication Inertial Measurement Unit Automation level Minimum Risk Manoeuvre Mode Selection and Arbitration Unit Near-Infrared Random Access Memory Real Time Kinematics Automation level Semi Automated Take-over Request Unified Modelling Language 1st System Functionality 65

22 Annex 2: Symbols V subject V target V set V max, pilot c d max subject vehicle speed target vehicle speed subject vehicle set speed maximum speed allowed for the pilot function clearance maximum detection range of the sensors for target vehicles 1st System Functionality 66