SELF DRIVING VEHICLE WITH CONTROL SYSTEM USING STEREOVISION TECHNIQUE Kekan S M*, Dr. Mittal S K Department of Electrical Engineering, G.H. Raisoni Institute of Engineering and Technology, Wagholi, Pune-412207, MS-India ABSTRACT: This paper presents a stereovision technique using in the field of self-driving vehicle. the key component of a self-driving vehicle is the ability to localize itself accurately in an unknown environment and simultaneously build the map of the environment. Majority of the existing navigation systems are based on laser range finders, LIDAR technology, lanelevel localization, sonar sensors or artificial landmarks. Navigation systems using stereo vision are rapidly developing technique in the field of self-driving vehicle. This paper describes an experimental approach to build a cost- effective stereo vision system for selfdriving vehicle that avoid obstacles and navigate through indoor and outdoor environments. This paper proposes a stereovision system, which is low-cost, yet also able to achieve high accuracy and consistency. The mechanical as well as the programming aspects of stereo vision system are documented in this paper. Stereo vision system adjunctively with ultrasound sensors was implemented on the self-driving vehicle, which successfully navigated through different types of cluttered environments with static and dynamic obstacles. KEYWORDS: stereovision, lane-level localization, LIDAR, GPS, Mat lab. INTRODUCTION The future will see the deployment of self-driving vehicle in the areas of indoor automation, transportation and unknown environment exploration. Implementation of self-driving vehicle systems in such tasks is widely appreciated technique as they handle these tasks more efficiently and reliably. Currently, a growing community of researchers is focusing on the scientific and engineering challenges of these kinds of self-driving vehicle systems. This project tries to address the main challenges in the field of self-driving vehicle autonomous navigation. There are several techniques for effective autonomous navigation, among which vision based navigation is the most significant and popular technique which experiences rapid developments. Other techniques include navigation using ultrasound sensors, lidar (light detection and ranging) systems, preloaded maps, landmarks etc. Navigation which uses ultrasound sensors will not detect narrow obstacles such as legs of tables and chairs properly, and hence leads to collision. Lidar systems are perfect tools for indoor navigation because of their accuracy and speed but they are less impressive for large scale implementation due to their high cost. Navigation based on landmarks and preloaded maps become valid options only when there is prior information about the environment and thus, it does not give a generic solution to the IJSRE- June, Vol(1), Issue(6) www.ijsre.in 1
problem of autonomous navigation. Vision can detect objects just as in the case of human vision and it gives the sense of intelligence to the self-driving vehicle. Out of all vision based techniques, stereo vision is the most adoptable technique because of its ability to give the three dimensional information about how the environment looks like and decide how obstacles can be avoided to safely navigate through that environment. Commercially available stereo cameras are expensive and require special drivers and software to interface with processing platforms which again adds up the cost of implementation. In this scenario, building a cost-effective stereo vision system using regular webcams which is able to meet the performance of commercially available an alternative is highly appreciable and this fact makes the theme of this project. THE DEVELOPED SYSTEM ARCHITECTURE Our experimental platform is a four wheeled differential drive rover that can carry a portable personal computer. Two wheels are free rotating wheels with optical encoders attached for keeping track of the distance travelled. Other two wheels are powered by high torque geared motor. A. BLOCK DIAGRAM Figure-1 Block Diagram of Self Driving Vehicle with Control System Using Stereovision Technique IJSRE- June, Vol(1), Issue(6) www.ijsre.in 2
B. DESCRIPTION The system consists of microcontroller along with other peripherals like, ultrasonic sensors, cameras, PC with MATLAB, motor driver IC, DC motors, serial to USB converter and power supply. We will be using Arduino Uno microcontroller board, which has Atmega 328P microcontroller on it. Ultrasonic module HC-SR04 will be used to detect distance of an object from a vehicle. To cover both the sides of vehicle (left and right) we will be using two ultrasonic sensors. Ultrasonic sensor work on a principle similar to radar or sonar. The output of sensors will be read by microcontroller using digital I/O pins. This output is then sent to PC using Serial to USB converter to process it on MATLAB. Mat lab will take decision of whether the object on left or right side of the vehicle is nearer or far. Accordingly command is sent back to the microcontroller to change the direction of vehicle using DC motors. Also, to detect the object in front of the vehicle, two cameras are interfaced with PC. Both the cameras will act as eyes of a vehicle. We will be using stereovision technique to detect the object and to find distance of the object ahead from the vehicle. The resulted output will be then sent to microcontroller board through Serial to USB converter. If the distance of the object in front of vehicle is far enough, motors will be driven as per regular speed. But, if the distance is less than threshold distance set by microcontroller, the speed of the vehicle will be decreased and later it will be stopped. Thus, ultrasonic sensors are used for side wise detection of an object and cameras are used for front side object detection. STEREOVISION TECHNIQUE Computer stereo vision is the extraction of 3D information from digital images, such as obtained by a CCD camera. By comparing information about a scene from two vantage points, 3D information can be extracted by examination of the relative positions of objects in the two panels. This is similar to the biological process, stereopsis. Stereo vision is a technique for extracting the 3D position of objects from two or more simultaneous views of a scene. Stereo vision systems are extensively used in object classification and object grasping applications because of its ability to understand the three dimensional structure of objects. Mobile robots can use a stereo vision system as a reliable and effective primary sensor to extract range information from the environment. In ideal case, the two image sensors used in a stereo vision system has to be perfectly aligned along a horizontal or vertical straight line passes through the principle points of both images. Cameras are prone to lens distortions, which are responsible for introducing convexity or concavity to the image projections. The process called Stereo-pair rectification is adopted to remap distorted projection to undistorted plane. The obtained rectified images from both the sensors are passed to an algorithm which searches for the matches in the images along each pixel line. The difference in relative positions of an identified feature is called as the disparity associated with that feature. Disparity map of a scene is used to understand the depth of IJSRE- June, Vol(1), Issue(6) www.ijsre.in 3
objects in the scene with respect to the position of the image sensors through the process called Triangulation. Figure-2a. shows the arrangement of image planes and Figure 2b. describes the Pinhole model of two cameras to illustrate the projection of a real world object is formed in left and right image planes. The formation of disparity is shown in Figure 2c. Accuracy of depth perception from a robust stereo vision system is sufficient for segmenting out objects based on their depth, in order to avoid collisions during navigation in real time. The following sections describe the details of hardware and software implementations of stereo vision system in this project. Figure 2: Modelling of stereo rig and disparity formation using pinhole model of cameras THE SOFTWARE FOR STEREO VISION SYSTEM ARDUINO COMPILER: The open-source Arduino Software (IDE) makes it easy to write code and upload it to the board. MATLAB: MATLAB (matrix laboratory) is a multi-paradigm numerical computing environment and fourth-generation programming language. RESULTS Stereo vision based SLAM architecture is one of the least pondered but rapidly developing research area which has been dealt in this project and we have successfully implemented a cost effective prototype of the stereo camera and self-driving vehicle. This performance is adequate for safe indoor navigation for slowly moving robots. The overlapping of vision perception with other information from sensors ensures a nearly errorproof navigation for self-driving vehicle in indoor and outdoor environments. Vision can detect objects just as in the case of human vision and gives the sense of intelligence to the self-driving vehicle. The choice of mechanical parameters of stereo rig, stereo correspondence algorithm parameters and filter parameters used for reconstruction were proved to be sufficient for the successful accomplishment of tasks identified during project IJSRE- June, Vol(1), Issue(6) www.ijsre.in 4
proposal. The images of self-driving vehicle navigating in the indoor environment are shown in Figure-3. Figure-3a: images of self-driving vehicle navigating in the indoor Figure-3b: images of self-driving vehicle graphs CONCLUSION This paper outlines the implementation of a cost-effective stereo vision system for a slowly moving self-driving vehicle in an indoor environment. The detailed descriptions of algorithms used for stereo vision, obstacle avoidance, navigation and three dimensional map reconstructions are included in this paper. The self-driving vehicle described in this paper is able to navigate through a completely unknown environment without any manual control. The self-driving vehicle can be deployed to explore an unknown environment such as IJSRE- June, Vol(1), Issue(6) www.ijsre.in 5
collapsed buildings and inaccessible environments for soldiers during war. Vision based navigation allows robot to actively interact with the environment. Even though vision based navigation systems are having certain drawbacks when compared with other techniques. Stereo vision fails when it is being subjected to surfaces with less textures and features, such as single cooler walls and glass surfaces. The illumination level of environment is another factor which considerably affects the performance of stereo vision. REFERENCES 1. Xinxin Du, Kok Kiong Tan, Member, IEEE, Comprehensive and Practical Vision System for Self-Driving Vehicle Lane-Level Localization IEEE transactions on image processing, 2016 2. S. Hassan Hosseinnia, Inés Tejado, Vicente Milanés, Jorge Villagrá, and Blas M. Vinagre, Experimental application of hybrid fractional-order Adaptive cruise control at low speed, IEEE transactions on control systems technology, vol. 22, no. 6, november 2014 3. Vadim Butakov, Petros Ioannou, Fellow IEEE, Driving Autopilot with Personalization Feature for Improved Safety and Comfort, 2015 IEEE 18th International Conference on Intelligent Transportation Systems. 4. R. Danescu and S. Nedevschi, Probabilistic lane tracking in difficult road scenarios using stereovision, Intelligent Transportation Systems, IEEE Transactions on, vol. 10, no. 2, pp. 272 282, 2009. 5. L. T. Sach, K. Atsuta, K. Hamamoto, and S. Kondo, A robust road profile estimation method for low texture stereo images, in Image Processing (ICIP), 2009 16th IEEE International Conference on, pp. 4273 4276, IEEE, 2009. 6. C. Guo, S. Mita, and D. McAllester, Stereovision-based road boundary detection for intelligent vehicles in challenging scenarios, in Intelligent Robots and Systems, 2009, IROS 2009. IEEE / RSJ International Conference on, pp. 1723 1728, IEEE, 2009. 7. G. Silberg, R. Wallace, Self-driving cars: The next revolution, KPMG LLP and Center for Automotive Research, 2012. 8. M. Althoff, O. Stursberg, M. Buss, Safety assessment of driving behavior in multi-lane traffic for autonomous vehicles, in Proc. IEEE Intell. Vehicle Symp. (IV), St. Louis, MO, 2009, pp. 893 900. IJSRE- June, Vol(1), Issue(6) www.ijsre.in 6