Dipl.-Inform. Günter Ehmen Dipl.-Inform. Stefan Puch

Transcription

1 Dipl.-Inform. Günter Ehmen Dipl.-Inform. Stefan Puch Carl von Ossietzky University of Oldenburg Research Group Safety Critical Embedded Systems Research Group Hybrid Systems

2 Who are we? Dipl.-Inform. Günter Ehmen Dipl.-Inform. Stefan Puch Research Associate Group of Safety Critical Embedded Systems lead by Prof. Dr. Werner Damm Research Associate Group of Hybrid Systems lead by Prof. Dr. Martin Fränzle SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 2

3 Where are we from? CvO University of Oldenburg (Oldb.) Founded in 1973 About students All images Carl von Ossietzky University of Oldenburg SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 3

4 What are we doing? We currently supervise a project group of master students developing an autonomous racing system. In this course the students have the opportunity to gain experience in modeling control and steering algorithms in SCADE. In our talk we present first results. Carl von Ossietzky University of Oldenburg SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 4

5 Have you ever heard about Autonomous Racing? Audi RS7 Hockenheimring ETH Zurich ORCA Racing Source: Source: SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 5

6 Where did we get the idea? CPS WEEK 2015, HSCC John Lygeros and Alex Lininger from ETH Zurich presented ORCA Racer They talked about: A Viability Approach for Fast Recursive Feasible Finite Horizon Path Planning of Autonomous RC Cars Autonomous Driving is a very topical issue and therefore exciting for student work during master studies A real setting is too expensive, we need a smaller setup SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 6

7 dnano Platform Developed by Kyosho, Japan Racing car in 1:43 scale Multiple tuning options Chassis, engines, Suspension, tires, Modular Racing track SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 7

8 ORCA - Obstacle Avoidance Source: SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 8

9 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 9

10 Project Group during Master Studies In general Mandatory lecture course Duration of two semesters Six to twelve students Expiry of a project group The project group discovers a design process on an industrial scale in the field of embedded systems Independent project and budget planning Collaborative refinement of the task Regular reviews do support the development process and assist in quality assurance SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 10

11 Project Group during Master Studies Learning objectives Getting to know a continuous design process Solving complex tasks in different domains Project management and teamwork Estimation of costs and risk assessment Presentation and documentation of results SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 11

12 Members of our Project Group RCCARS Nikolai Bräuer Anatolij Fandrich Michael Bukowski Tom Reske SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 12

13 Jobs within the Project Group Course of study: Embedded systems and micro robotics (M.Sc.) Tom Reske (B.Eng.): Group Director Anatolij Fandrich (B.Sc.): Quality management Course of study: Computer Science (M.Sc.) Nikolai Bräuer (B.Sc.): Test management and document management Michael Bukowski (B.Sc.): Representative for hard- and software development and public relations SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 13

14 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 14

15 Related Work Autonomous RC Racing started in 2009 with a student thesis Features until 2015: Real-time Control of 1:43 Scale race cars Model Predictive Contouring Control for RC Race Cars Static and dynamic obstacle avoidance Design and Implementation of an Embedded Sensing and Control Platform for 1:43 Scale R/C Race Cars Autonomous Drift Control Racing Backwards with 1:43 RC cars Fast Viable Path Planning for 1:43 RC Cars State and friction estimation for 1:43 RC Cars Source: SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 15

16 Related Work Started with a master thesis in 2014 Cooperation with LMS International NV (Siemens) Design and implementation of a timeoptimal controller for model race cars Research context Path planning Model predictive control Development of a demonstrator Source: -of-a-time-optimal-controller-for-model-race-cars.html SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 16

17 Related Work The project CARS started in 2014 Three projects were preformed by students I. Camera-based Autonomous Racing System (Hardware setup, autonomous control software, lap time better than a human driver) II. III. Time optimal control strategy (Optimization of control algorithm, integration of a second car) Replacing electronics and extending control (Microcontroller, Bluetooth LE communication, Collision avoidance algorithm) SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 17

18 Related Work Project started 2014 with a bachelor thesis of six students Advanced vehicle control systems A multi-vehicle experimental platform for automotive algorithms Speed optimizations compared to an existing platform (from 30cm/s to 200cm/s) Proof of concept algorithms for collision avoidance & overtaking and platooning have been implemented SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 18

19 Schematic Structure of the System Source: CARS - Camera-based Autonomous Racing System, Universität Uppsala: SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 19

20 Information Flow in the System SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 20

21 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 21

22 Long Term Vision of RCCARS The Long Term Vision of the project RCCARS for our two research groups Safety Critical Embedded Systems and Hybrid Systems is the construction of a test bed to develop and analyze strategies, algorithms etc. in context of autonomous driving. This involves: The development of real-time-capable algorithms to the longitudinal and lateral control of vehicles Using a portable setup Test bed as a demonstrator at exhibitions and project reviews SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 22

23 Long Term Vision Development Steps Ongoing development by project groups and master or bachelor theses Step 1 will be implemented by the project group RCCARS SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 23

24 Scenario: Accident-free driving The scenario accident-free driving is fulfilled, if and only if a single car can drive autonomously at least five laps with an average speed of 1.5m/s on a closed racetrack without a collision of the lane boundary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 24

25 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 25

26 System Overview Inspired by the related work Four subsystems and a system control software Modular structure focusing on extensibility SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 26

27 Subsystems Racetrack & Car Kyosho Mini-Z Grand Prix Circuit (48 Parts) The racetrack has to be free of obstacles Lane boundaries prevent unintentionally departing from the racetrack Kyosho dnano RC Car, scale1:43 Every car has got an unique identifier (IR marker) Regular maintenance is recommended Sand in the steering gear SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 27

28 Subsystem Positioning Point Grey Flea3 Camera Framerate up to 150Hz USB 3.0 connection Wide-angle lens: 60 (V), 75 (H) Infrared Filter Infrared LEDs with heat sink In our scenario the car covers a distance of 1.5cm in 10ms The positioning requires a high frame rate to detect the race car The combination of infrared light and infrared filter reduces the processing load SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 28

29 Subsystem Control-Unit Represents the vehicle control system Real-time capable control algorithms necessary To cooperate with the positioning an execution time of 10ms would be advisable The black-box remote control requires some clever modifications to get different control and operation modes (e.g. assisted driving) SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 29

30 System Control In order to administrate the system some kind of control software is essential It should provide A graphical user-interface for the administrator Functionality to start, stop and emergency stop the race Logging capabilities for debugging purpose Virtual representation of the racing scenario SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 30

31 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 31

32 System Architecture and Interfaces SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 32

33 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 33

34 Reflection Marker Encoding of the car identities schematic view and realistic image SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 34

35 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 35

36 Subsystem Racetrack SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 36

37 Racetrack Marker SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 37

38 Racetrack Model CSV file Resolution of the racetrack: 1x1cm Data representation 0: impassable (red) 1: accessible (green) 2: target trajectory (blue) SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 38

39 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 39

40 Subsystem Positioning The local GPS in our project Image capturing Image processing Object detection Position calculation Error handling Developed in C++ using Eclipse IDE OpenCV Library PointGrey FlyCapture SDK SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 40

41 Subsystem Positioning SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 41

42 Subsystem Positioning SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 42

43 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 43

44 Subsystem Control-Unit SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 44

45 Subsystem Control-Unit SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 45

46 Subsystem Control-Unit SCADE Model Path planning Longitudinal and lateral control Collision detection and avoidance Plausibility checks Error handling SCADE Suite offers us simulation based analysis in early design stages and automatic code generation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 46

47 Subsystem Control-Unit Wrapper Code Developed in C using the Eclipse IDE Data preparation Imports the racetrack Triggers the SCADE Model Provides inputs Executes the model Reads outputs Error handling Interacts with the CarControl SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 47

48 Subsystem Control-Unit SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 48

49 Subsystem Control-Unit: CarControl SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 49

50 Subsystem Control-Unit: CarControl STM32F4 Discovery Board Perfex KT18 Remote Control CooCox CoIDE development environment GCC-ARM-EMBEDDED Toolchain FTDI and USBCDC SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 50

51 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 51

52 System Control The system behavior is described by a state machine Implemented in Java using the Eclipse development environment SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 52

53 Implementation SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 53

54 Network Communication Distributed system with network communication Implemented via UDP-Sockets SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 54

55 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 55

56 Quality Assurance Project reviews Test planning Integration tests Equivalence class tests Test management Code generation Simulation Test documentation Test automation Black- & white-box tests Test database Code review Hardware evaluation Cross tests SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 56

57 Test Database SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 57

58 Test Database SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 58

59 Quality Assurance Hardware evaluation RC Car Acceleration and braking distance Turning characteristics Racetrack and lane boundary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 59

60 Quality Assurance Hardware evaluation Camera Filter Infrared LED Infrared Marker SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 60

61 Outline Introduction and Motivation Lecture course Project Group during master studies at the department of computing science in Oldenburg Autonomous Racing - Related Work RCCARS - Long Term Vision und Project Group assignment RCCARS - System Overview RCCARS - Architecture and Implementation RCCARS - Quality Assurance Current State and Summary SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 61

62 Current State join our live demonstration during lunch break SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 62

63 Summary We introduced autonomous racing and presented our approach at the University of Oldenburg The project is mainly driven by students during their master and bachelor studies First use of SCADE as model-based development tool in the department of computer science at the University of Oldenburg Special thanks goes to Lothar Pfeifer and Chloe Dasse for getting started with the SCADE Academic Program Tobias Knostmann and Christian Schrader for their local support and training SACC 2016 Günter Ehmen & Stefan Puch Realtime Controlled Cooperative Autonomous Racing System 63

64 Thank you for your attention! Visit our web site for further information