A Simulation Environment for Developing Intelligent Headlight Systems

Size: px

Start display at page:

Download "A Simulation Environment for Developing Intelligent Headlight Systems"

Avis Arnold
5 years ago
Views:

1 A Simulation Environment for Developing Intelligent Headlight Systems Martijn Tideman, Simon J. Janssen, Member, IEEE Abstract This paper presents a simulation environment for developing Intelligent Headlight Systems (IHS). These are systems that actively adapt the headlights based on events in the traffic environment such as curves, oncoming vehicles and pedestrians. The results of applying the simulation environment to two IHS research projects of an OEM are presented. One of the findings is that the behaviour and performance of the IHS concepts could be evaluated without using road tests. The paper concludes with discussing how the simulation environment can also be applied for developing intelligent vehicle systems such as ACC, FCW, CMB, LDW and LKA. A I. INTRODUCTION DAPTIVE Front-lighting Systems (AFS) adapt the vehicle s headlights based on inputs from the steering system, the suspension or other in-vehicle systems. Over the years, various different vehicles with AFSs have been introduced. A well-known example is the 1967 Citroën DS which was equipped with a system that adjusts the headlights based on mechanical inputs from the steering system. More recent AFSs mostly rely on electronic rather than mechanical inputs. For example, rather than reacting to curves based on steering angle, some concepts use GPS and map data to anticipate changes in road curvature [1]. Modern AFSs may also include auxiliary optical systems that are switched on and off as the vehicle and operating conditions call for light or darkness at the angles covered by the light beams. When the system also responds to events in the traffic environment rather than it only reacts to inputs from in-car systems, it is usually called an Intelligent Headlight System (IHS). The ultimate goal of an IHS is to fulfil all the lighting demands of the driver, but, at the same time, not bother or distract other road users. Hella has developed a headlight module which can emit up to five different beam patterns (i.e. town light, country light, motorway light, high beam light and adverse weather light) [2]. Gentex has developed an IHS system which automatically switches the vehicle s high beams on and off depending on detected headlamps and tail lamps from other Manuscript received February 15, M. Tideman is with TNO Automotive, Steenovenweg 1, 5708 HN Helmond, The Netherlands (phone: ; fax: ; martijn.tideman@tno.nl). S. J. Janssen is with Fontys University of Applied Sciences, P.O. Box 347, 5600 AH Eindhoven, The Netherlands ( simon.janssen@student.fontys.nl). vehicles [3]. IHSs with LED matrices provide even more freedom in light beam patterns, as the LEDs in the array of the headlamp module can be individually activated. For example, the Mercedes ESF 2009 safety research vehicle is equipped with such a system [4]. It darkens the part of the beam in which oncoming vehicles are located, but the system is also capable of directing the light beam such that potential hazards are highlighted. Developing an Intelligent Headlight System is extremely challenging, costly and time-consuming. The performance of an IHS depends on many parameters of which the values can vary widely. For example, traffic participants that should be detected can vary from large oncoming trucks to small children crossing the road. Environmental conditions in which the system has to operate can vary between a well lit busy town to a dark, rainy and deserted country road. These many parameters make that developing an IHS by means of real-world testing only is not a feasible option. This is especially true when the performance of the IHS during crash or near-crash scenarios has to be assessed as well. Simulation software may help to meet the stringent money-, time- and safety-constraints that play a role while developing an IHS. This paper shows how the software package PreScan can be applied in the development process of Intelligent Headlight Systems. Section II gives a general introduction to PreScan. In section III, a case is presented that is based on two research projects of an OEM. Section IV describes the results of applying PreScan to this case. Finally, section V contains concluding remarks. II. PRESCAN PreScan was created by TNO as a development environment for Intelligent Vehicle (IV) systems [5]. These are systems with sensors that monitor the vehicle s surroundings and use this information to warn the driver or to intervene in the control of the vehicle. PreScan can be used to develop IV systems based on sensor technologies such as radar, laser, camera, GPS and antennas for car-to-car (C2C) and car-to-infrastructure (C2I) communication. PreScan comprises several modules that together provide everything an IV system developer needs. It works using four steps (see Fig. 1). The dedicated PreScan graphical user interface (GUI) allows users to build traffic scenarios, using a database of road sections, infrastructure components (trees, buildings, traffic signs) and road users (cars, trucks, cyclists,

pedestrians). Representations of real environments can be made by reading in information from Google Earth [6], Google 3D Warehouse [7] and/or a GPS navigation device.

The interface with Matlab/Simulink [8] enables users to design and verify algorithms for data processing, sensor fusion, decision making and control.

Finally, interfaces with dspace ControlDesk [11] and NI LabVIEW [12] can be used to automatically run a batch of scenarios as well as to run Hardware-in-the-Loop (HiL) simulations. Fig. 1.

2 pedestrians). Representations of real environments can be made by reading in information from Google Earth [6], Google 3D Warehouse [7] and/or a GPS navigation device. Weather conditions (rain, snow, fog) and light circumstances can also be specified. Modelling sensors is done by filling PreScan s generic sensor models with supplier-specific specifications. The interface with Matlab/Simulink [8] enables users to design and verify algorithms for data processing, sensor fusion, decision making and control. This interface also allows for re-using existing Simulink algorithms or vehicle dynamics models from CarSim [9] or vedyna [10]. Finally, interfaces with dspace ControlDesk [11] and NI LabVIEW [12] can be used to automatically run a batch of scenarios as well as to run Hardware-in-the-Loop (HiL) simulations. Fig. 1. The four stages of PreScan. PreScan s latest release (R4.0) allows vehicles and lampposts to be equipped with lights. This functionality enables testing of vision systems in the dark as well as development of Intelligent Headlight Systems (see Fig. 2). Lights in PreScan can be assigned an intensity distribution map, and their position and orientation can be changed in runtime (i.e. based on the output of the algorithms that the user has defined in Matlab/Simulink). Also the light s overall intensity and its colour temperature can be adapted in runtime. A. Introduction III. CASE To show how PreScan can be applied in the development process of Intelligent Headlight Systems, it was applied to two IHS research projects of an OEM. The first IHS application is a High Beam Assistant that automatically pitches the high beam down when other vehicles are detected in the beam area. This should be done such that the beam ends just in front of or just behind the other vehicle. The system works based on input from a camera sensor that monitors the distance to other vehicles that have their headlights or taillights switched on. When another vehicle gets so close that the high beam cannot pitch down any further, the high beam is automatically switched off. When the curve radius becomes so small that it is impossible for the camera to detect vehicles further down the road, the high beam is deactivated as well. Finally, the high beam is also turned off when the vehicle s speed drops below 60 km/h, which indicates that the vehicle may drive in an urban area. The second IHS application is a Light Barrier Assistant that projects a red light barrier (i.e. a virtual stop-line) on the road in front of the vehicle. This light barrier should warn the driver for a possible collision at an intersection. The system operates based on input delivered by a C2C communication system. From the gathered data, the paths of other road users as well as the times-to-collision (TTC) to them are calculated. If the paths and TTCs are such that the chances of a collision exceed a certain threshold value, the headlamps will project a light barrier on the road in front of the vehicle. B. Modelling the lights in the PreScan GUI Fig. 3 shows the part of the PreScan GUI where the lights can be modelled. Fig. 2. Developing Intelligent Headlight Systems with PreScan. Fig. 3. Modelling the lights in the PreScan GUI.

The IES file format is widely used by many lighting manufacturers and is one of the industry standards in photometric data distribution.

3 For the High Beam Assistant, one of the OEM s demands was that the headlights (both the low beams and the high beams) were modelled after IES files. These IES files contain the lights intensity distribution maps stored in ASCII format. The IES file format is widely used by many lighting manufacturers and is one of the industry standards in photometric data distribution. For the Light Barrier Assistant, two different bar-shaped auxiliary lights had to be modelled. The first light should be able to project a horizontal light barrier to warn for collisions with crossing road users. The second light should project a vertical light barrier to warn for road users that are travelling straight while the vehicle is preparing to turn. C. Modelling the IHS algorithms in Simulink After modelling the lights in the PreScan GUI, the next step was to create a Simulink model that contains the algorithm for both IHS applications (see Fig. 4). The relevant parameters and threshold values were specified by the OEM. Fig. 4. Modelling the IHS algorithms in Simulink. The Simulink model of the High Beam Assistant contained the following main elements: 1) A relevant object selection algorithm. 2) An algorithm that pitches down the high beam based on the distance to the most relevant object. 3) An algorithm that deactivates the high beam when the vehicle is moving too slow, when the curve radius is too small, and/or when the selected relevant object is so close so that the high beam completely overlaps the low beam. The Simulink model of the Light Barrier Assistant had similar characteristics. It consisted of: 1) A relevant object selection algorithm. 2) A risk estimation algorithm. 3) An algorithm that pitches down the light barrier based on the distance to the most relevant object. 4) An algorithm that deactivates the light barrier when it is too close to the vehicle for the driver to see it or when the speed of the vehicle gets above a certain threshold. D. Relevant object selection algorithm For the High Beam Assistant, a relevant object is defined as a vehicle with activated headlights and/or taillights. The relevant object selection algorithm can process a maximum of fifteen detected vehicles. Initially, the algorithm assigns the highest priority to the closest vehicle with activated headlights/taillights, the second highest priority to the second closest vehicle, etc. Subsequently, all detected vehicles are compared with a heading criterion which checks if the detected vehicle is oncoming or preceding. If the closest detected vehicle is not considered to be either oncoming or preceding (for example, in case of parked vehicles or crossing vehicles), the second closest vehicle automatically becomes the most relevant object, etc. The relevant object selection algorithm of the Light Barrier Assistant works similarly. However, in contrast to the High Beam Assistant, this algorithm can also assign a high relevancy to crossing road users or to road users in the blind spot. E. Risk estimation algorithm The risk estimation algorithm uses the paths of other road users as well as the times-to-collision (TTC) to them to determine the chances of a collision. This algorithm is only present in the Light Barrier Assistant and not on the High Beam Assistant. F. Pitching algorithm For the High Beam Assistant, the distance to the closest relevant vehicle is used as an input for the pitching algorithm so that the beam ends just in front of or just behind that vehicle. For the Light Barrier Assistant, the pitching angle is also based on the distance to the other road user. The shorter the distance, the further the light barrier is pitched down. G. Deactivation algorithm For the High Beam Assistant, the high beams are switched off when they are pitched such that they completely overlap the low beams. The high beams are also switched off when the curve radius and/or the vehicle speed comply with

specific values. A similar deactivation algorithm was used for the Light Barrier Assistant.

Validation To validate the High Beam Assistant, a traffic scenario was created that would test all the OEM s requirements on the IHS s performance.

Some examples of situations within this traffic scenario are: 1) Oncoming vehicles (at various speeds and at various locations within the scenario) 2) Preceding vehicles (at various speeds and at

4 specific values. A similar deactivation algorithm was used for the Light Barrier Assistant. However, the criteria for deactivation were different as these depend on whether or not there is still a significant chance of collision. H. Validation To validate the High Beam Assistant, a traffic scenario was created that would test all the OEM s requirements on the IHS s performance. Different road types, environments, traffic situations and events were integrated into this scenario. Some examples of situations within this traffic scenario are: 1) Oncoming vehicles (at various speeds and at various locations within the scenario) 2) Preceding vehicles (at various speeds and at various locations within the scenario) 3) Parked vehicles (some with their lights switched on and some with their lights switched off) 4) Different curve radii (some exceeding and some not exceeding the deactivation threshold) 5) Different vehicle speeds (some exceeding and some not exceeding the deactivation threshold) The scenarios for validating the Light Barrier Assistant were based on two possible collision situations that were specified by the OEM. Fig. 5 shows a situation where the light barrier should warn for crossing road users, whereas Fig. 6 shows a situation where the IHS should warn for road users that are travelling straight while the vehicle is preparing to turn. A. High Beam Assistant IV. RESULTS Fig. 7 and Fig. 8 show the results of loading the IES files that were provided by the OEM into PreScan. Fig. 7 shows the low beam, whereas Fig. 8 shows both the low beam and the high beam projected on the road. Fig. 7. Low beam (pan view from behind the vehicle). Fig. 5. Light Barrier Assistant Scenario A: crossing road users. Fig. 6. Light Barrier Assistant Scenario B: road users travelling straight while the vehicle is preparing to turn. Fig. 8. Low beam and high beam (pan view from behind the vehicle). The first step in the IHS s algorithm is to determine the relevancy of a detected vehicle. In case the detected vehicle is not considered to be relevant, the IHS will not respond to this vehicle and move on to the next vehicle. Fig. 9 shows a situation where the car that is equipped with the IHS drives past two parked cars. Both parked cars are detected but lack the required relevancy to make the IHS respond. As can be seen, the high beam remains switched on and there is no pitch.

Fig. 9. Passing parked cars which are considered irrelevant by the IHS. When a relevant vehicle is detected, the distance to this vehicle is sent to the pitching system.

The first step creates an initial pitch angle as soon as a relevant vehicle is detected: as quickly as possible, the high beams are moved down under the lights of the oncoming vehicle.

10 gives a schematic overview of this concept. Fig. 11. Pitching behaviour of the IHS while an oncoming vehicle approaches with constant speed.

If the preceding vehicle starts to accelerate and the inter-vehicle distance increases, the high beam will start to pitch up.

Pitching the headlights under the lights of the oncoming vehicle. Fig. 11 shows the pitching behaviour of the IHS while an oncoming vehicle approaches with a constant speed.

The second scope shows that, at the moment of first detection, there is a small initial pitching movement. As the oncoming vehicle further approaches, the pitching movement is smoothly continued.

5 Fig. 9. Passing parked cars which are considered irrelevant by the IHS. When a relevant vehicle is detected, the distance to this vehicle is sent to the pitching system. Pitching the high beams is done in two steps. The first step creates an initial pitch angle as soon as a relevant vehicle is detected: as quickly as possible, the high beams are moved down under the lights of the oncoming vehicle. The second step is active during the rest of the oncoming vehicle s approach: the pitch angle and pitch rate keep increasing while the vehicle gets closer. Fig. 10 gives a schematic overview of this concept. Fig. 11. Pitching behaviour of the IHS while an oncoming vehicle approaches with constant speed. When the most relevant detected vehicle is preceding the vehicle that is equipped with the IHS, the high beam will be pitched down to a constant angle. If the preceding vehicle starts to accelerate and the inter-vehicle distance increases, the high beam will start to pitch up. Similarly, if the intervehicle distance decreases, the high beam will pitch down further. Fig. 12 shows a partially pitched down high beam due to a preceding vehicle. Fig. 10. Pitching the headlights under the lights of the oncoming vehicle. Fig. 11 shows the pitching behaviour of the IHS while an oncoming vehicle approaches with a constant speed. The horizontal axes show the elapsed time. The first scope shows that the oncoming vehicle is first detected at a range of approximately 380m (t = 16,5s). The second scope shows that, at the moment of first detection, there is a small initial pitching movement. As the oncoming vehicle further approaches, the pitching movement is smoothly continued. Just before the oncoming vehicle passes, the high beam is switched off because it completely overlaps the low beam. After the oncoming vehicle has passed, the high beams are switched on again and pitched back to their nominal position. Fig. 12. A partially pitched down high beam due to a preceding vehicle. Another feature of the High Beam Assistant is the deactivation of the high beam when the curve radius becomes too small for the camera to properly detect a vehicle. Fig. 13 shows the vehicle that is equipped with the IHS moving into a bend with a small radius. It can be seen that the high beams are switched off. And this is with good reason, as ahead in the bend there is an oncoming car which is not within the sensor s field of view.

Between approximately 6 and 8 seconds, this threshold value is exceeded and the high beam is therefore deactivated.

6 Fig. 13. High beam deactivated due to a too small curve radius. Fig. 14 shows a few scopes that display the IHS s behaviour during this situation. The first scope shows the actual bend radius as well as the threshold value for switching off the high beam. Between approximately 6 and 8 seconds, this threshold value is exceeded and the high beam is therefore deactivated. The second scope shows that the oncoming car was not in the sensor s view for a long time. Just after deactivating the high beam, the oncoming vehicle enters the sensor s field of view. The last scope shows the pitching angle of the high beams if they would have been activated. It can be seen that the initial pitching step, which would pitch the high beams under the oncoming vehicle s headlights as fast as possible, would be almost 3 degrees. B. Light Barrier Assistant Fig. 15, Fig. 16 and Fig. 17 show the results of evaluating the Light Barrier Assistant in the scenario that was schematically depicted in Fig. 5. The vehicle approaches an intersection whereas a bicycle is crossing this intersection and a collision is likely to occur. The two traffic participants communicate their GPS locations, velocities and headings to each other by means of C2C communication (in this case, bicycle-to-car communication). Based on the received data, a horizontal red light barrier is projected on the road in front of the vehicle. As the car moves closer to the intersection, the light barrier will pitch down thereby getting closer to the car. When the car gets dangerously close to the bicycle, the light barrier will start to flicker as an ultimate attempt to draw the driver s attention. Fig. 15. Results of Light Barrier Assistant Scenario A (TTC = 7s) Fig. 16. Results of Light Barrier Assistant Scenario A (TTC = 3s) Fig. 17. Results of Light Barrier Assistant Scenario A (TTC = 2s) Fig. 14. Behaviour of the IHS in a curve with a small radius.

7 Fig. 18, Fig. 19 and Fig. 20 show the results of the scenario that was schematically depicted in Fig. 6. The vehicle is preparing to turn while a bicycle that wants to go straight is in its blind spot. The IHS projects a vertical light barrier on the road in order to warn the driver for the hazard. Fig. 18. Results of Light Barrier Assistant Scenario B (TTC = 7s) Fig. 19. Results of Light Barrier Assistant Scenario B (TTC = 3s) V. CONCLUSIONS & OUTLOOK Developing an Intelligent Headlights System (IHS) is extremely challenging, costly and time-consuming. This is especially true when the performance of an IHS during crash or near-crash scenarios has to be assessed as well. Simulation software may help to meet the stringent money-, time- and safety-constraints that are associated with developing an IHS. Based on two concrete research cases of an OEM, it was described how the software package PreScan can be applied within the development process of IHSs. The first application presented in this paper is a High Beam Assistant that automatically pitches the high beam down when other vehicles are detected. The second IHS application is a Light Barrier Assistant that warns the driver for possible collisions at an intersection by projecting a virtual stop-line on the road in front of the vehicle. By modelling and simulating both IHS applications in PreScan, the behaviour and performance of the system concepts could be assessed without using road tests. PreScan s use and benefits are certainly not limited to the field of IHS development. As PreScan is a generic platform for developing Intelligent Vehicle (IV) systems, it can also be used in the development process of systems such as ACC, FCW, CMB, LDW and LKA. The main benefit of PreScan is that it facilitates for fast and cost-effective design iterations. Furthermore, PreScan allows for both controlling and quantifying the test conditions. Finally, PreScan enables for testing IV systems in dangerous situations. In future, developing intelligent vehicle systems is going to be an even more challenging and complex task than it is today. Reasons for this are that: 1) Individual IV systems will be integrated with each other. 2) IV systems will be required to handle increasingly complex scenarios. 3) More parts of the driving task will be taken away from the human driver and delegated to the IV system. Simulation environments such as PreScan help IV system developers to deal with those challenges and complexities. Fig. 20. Results of Light Barrier Assistant Scenario B (TTC = 2s) One of the things that was found about the behaviour and performance of the Light Barrier Assistant was that the system works better in cars that have short motor hoods. This is because long motor hoods will block the driver s view of the light barrier, especially in highly critical situations (i.e. situations in which the light barrier is projected very close to the vehicle). Another interesting and valuable finding from the PreScan simulations is that the visibility of the light barrier is significantly increased when the headlights are dimmed at the moment the light barrier is activated. REFERENCES [1] Wikipedia (2010, February 14). Headlamp [Online]. Available: [2] J. Bedi et al., The Technology Roadmap Report, Stamford, UK: Supplier Business Ltd., [3] K. Soezen et al., Active Safety Systems Report Volume II: Technology, Stamford, UK: Supplier Business Ltd., [4] M. Fehring, Das Mercedes Benz Experimentier-Sicherheits-Fahrzeug ESF 2009, in 7. VDI Tagung Fahrzeugsicherheit, Berlin, [5] M. Tideman, Scenario Based Simulation Environment for Assistance-Systems, in ATZ Magazine, vol. 112, 2010 [Online]. Available: [6] Google Earth: [7] Google 3D Warehouse: [8] Matlab/Simulink: [9] CarSim: [10] vedyna: [11] dspace ControlDesk: [12] NI LabVIEW:

Regulatory Impacts of Advanced Lighting Systems. Stephan Berlitz, AUDI AG

Regulatory Impacts of Advanced Lighting Systems Stephan Berlitz, AUDI AG 1 Technology Development of Frontlighting New LED Functions (Audi) 201x LED DRL (Audi) 2004 Full LED Head Lamp (Audi) R8-2006 Adaptive