Project Title: Wireless Hummer ECE 792 - Final Written Report Project Team Members: Justin Audley, Blake Brown, Christopher Dean, Andrew Russell, Andrew Saunders ECE Faculty Advisor: Dr. Richard A. Messner Submit Date: May 19, 2010
Abstract: The Wirelesss Hummer was a senior design project created by five electrical engineering students during the 2010-2011 school year. The goal of this project was to transform a remote control toy car into a laptop controlled reconnaissance robot. The Hummer was outfitted with a webcam, a microphone, four ultrasonic range finders, an Arduino Microcontroller and an Acer netbook. The research was completed and the project was presented at the Undergraduate Research Conference on April 27, 2011, where it functioned as expected. This report outlines the research, implementation, and testing that was performed on the Hummer from October 2010 to April 2011. It is written in chronological order of the goals that were completed during that period and an evaluation of the project and our performance is given at the end.
Table of Contents Abstract:... 2 Introduction... 4 Project Overview... 4 Initial Research:... 5 Wireless Steering and Control:... 6 Obstacle Detection:... 7 Relative Motion Detection:... 8 Video Transmission:... 8 Additional Goals Research:... 9 Elevator Accessibility:... 10 Evaluation:... 11
Introduction: This project idea was shaped by Dr. Richard A. Messner, and we accepted the challenge at the end of ECE 694 in the spring of 2010. We talked with Dr. Messner at the beginning of September 2010 about what his intentions were for the toy hummer and we wrote a proposal outlining our goals and expected budget for the project. The proposal was presented and accepted by Dr. Tom Miller and Dr. Richard A. Messner during October 2010. Research for the project started immediately, and we completed our first task of wireless control just in time for the project progress report in December 2010. At the start of second semester, we still needed to complete our secondary tasks, relative motion detection, obstacle detection and video transmission by April 2011. It is safe to say that we had our work cut out for us. These main goals were completed by the end of March, and we began working on our lofty additional goal of elevator accessibility in April. At the URC, all of our proposed deliverable goals were completed and in working order, but the elevator accessibility became a proof-of-concept idea because we weren't able to manipulate a solenoid from an infrared signal. The project was considered completed once the URC ended and further research was not conducted. Project Overview The purpose of this project was to create a device that will be capable of receiving and executing commands given from a remote location. For this project, the device was a remote control toy Hummer (shown in Figure 1) that was outfitted with several sensors, a netbook and an arduino microcontroller. The built-in remote control functionality was disabled, and the connection to the Hummer was made over an ad-hoc network which provided several advantages. The ad-hoc network allowed access directly to the hummer from a remote location without the use of wires and it avoided the heavy traffic on the UNH wireless network. This connection was used to send commands from the remote user to the hummer and to relay video from the Hummer back to the remote user. Figure 1: Remote Control Hummer for Senior Project (September 2010)
Initial Research: In October 2010, we disassembled the toy hummer and found that there were two DC motors, one for the front axle and one for the rear axle as shown in figure 2. We also found a servo motor with two wires for power and three wires for the potentiometer value shown in Figure 3. The toy was controlled by a single 12 volt lead acid battery. The motors were controlled by an h-bridge control circuit that interpreted the signals sent from the handheld remote transmitter. There was also another circuit board which we determined to be devoted to power distribution. The transmitter operated at 24 MHz. Initially, we left the antenna connected to the Hummer. This allowed us to observe how the hummer responded to the signals sent from the transmitter. We used this information to determine the functionality of the two circuits, because we could not find any information about this toy online. The antenna, headlights, taillights, and speaker were all connected to the control board shown in Figure 4. It was determined that a microcontroller was present on the control board with pins 8 and 9 being used to control the steering of the vehicle. Figure 2: The Undercarriage of the Hummer Figure 3: Hummer Taken Apart with Control Board, Power Board, Steering Cable and DC Motors Figure 4: Control Board interfaced to the Arduino (not shown)
Wireless Steering and Control: The communication medium between the user and the vehicle was the IEEE 802.11 wireless standard, otherwise known as WiFi. The Windows Sockets API, WinSock, was the highest level layer between the two communication devices. The client opened a connection that the server was listening for, creating a channel for the two-way communication between the user and the Hummer (see Figure 5). Custom code was written for both the server and client, which was constantly checking buffers for data to be received or transmitted. The server to client code was written using OGRE, an open source 3D graphics rendering system. However, a graphical user interface (GUI) was not created due to lack of time. Custom code was also written for the Arduino using the Arduino IDE. This code controlled the on board circuitry on the Hummer. We had specific code written for speed, steering, and the ultrasonic range finders. Below is a portion of some of the custom code used to control the Hummer. int minfrontdistance = 30; int minbackdistance = 20; int minleftdistance = 5; int minrightdistance =5; void ProcessSensors() { //if going Forward and front sensor sees something if ((SensorVal1 <minfrontdistance) && (Direction == 1)){ goforward(0); //stop } //if going Backward and back sensor sees something if ((SensorVal4 < minbackdistance) && (Direction == 0)){ goforward(0); //stop } //Front Left Sensor if(sensorval3 < minleftdistance){ goforward(0); } //Front Right Sensor if(sensorval2 < minrightdistance){ goforward(0); } return; } This code was used for obstacle detection and each SensorVal is a reading from one of the ultrasonic range finders. The minimum distances for each side is set by the user and are shown above. Looking at Front Left Sensor for example, if SensorVal3, the detected value from the ultrasonic range finder, is less than the set minleftdistance of 5, the Hummer will stop. The line goforward(0)is telling the Hummer to go forward at a speed of 0, which realistically means telling the Hummer to stop. In order for the Hummer to move, commands were sent from the user at the client computer, which were sent over WiFi to the server computer, which was the onboard netbook. The WASD keys and the arrow keys were used as the input for speed and steering. The up arrow was used for setting the maximum speed, and the down arrow decreases the maximum speed. These were chosen because they are typical movement keys for a computer game. The steering would automatically go back to the center when no turn key was pressed. These commands were
interpreted by the netbook and then sent over the COM port to the Arduino. The Arduino parsed the command and executed the required action, whether it be controlling the H-Bridge to drive the DC motors for forward and backward movement, or controlling the servo motor for turning. The ability to drive the hummer over the wireless network in Kingsbury was poor. There was a noticeable latency between when the user inputted a command and the hummer acted upon the command. This led to the decision to use an ad-hoc network, as this is not using any internet connected network at all, and is simply connecting the netbook to the remote computer. The netbook did not have the ability to host an ad-hoc network, but it could connect to one. Therefore, we used the remote laptop to setup the network. Figure 5: Flowchart for link between Client PC (user) and Hummer Obstacle Detection: For the obstacle detection portion of the project, the sensors chosen were the Maxbotix LV-MaxSonar-EZ0 ultrasonic range finders (Figure 6) capable of detecting distances accurate to the inch in the range of 5 inches to several feet. Four of these sensors were used, one on each side of the Hummer. These sensors all ran simultaneously to collect data from every direction. There are multiple ways these sensors can output the data. For this project, an analog voltage proportional to the distance measured was the best choice as it can be easily read by the A/D converter of the Arduino. This allowed the Arduino to override the user in the event an obstacle was in the path of the Hummer. The first problem encountered with the sensors was that while the code to process the sensor data and determine if the hummer needs to stop was running, the hummer moved with a stutter, stopping and starting constantly. Commenting out this function in the Arduino code allowed the hummer to move smoothly again. The problem was that the explicit delays in the code as specified by Maxbotix for proper sensor operation were making the main loop take too long to execute. These delays were shortened and the main loop was cleaned up, removing everything that was not absolutely necessary that had been created throughout the project for various debugging purposes. The hummer was then brought into the hall and tested again, where it could successfully stop itself before driving into a wall, even with instruction from the user to do so.
Figure 6: Maxbotix LV-MaxSonar-EZ0 Ultrasonic Range Finder Relative Motion Detection: The goal was to allow the hummer to track its movements relative to its starting position as well as speed and overall distance. We researched an optical mouse hack, where the data from the sensor is used to track movement similar to how a mouse on a computer works. We disassembled two optical mice and built two separate contraptions that were attached to each side of the undercarriage of the hummer. These sensors required a strong source of ambient light, and as such, LEDs were angled toward the ground directly under the sensors. We were able to move the hummer back and forth and see the mouse pointer on a computer screen move accordingly. We ran out of time to fully test and implement this idea. Video Transmission: The Hummer was implemented with a USB web camera connected to the netbook. It was placed on the top of the Hummer above the windshield in order to provide the best visibility. The webcam worked fine directly on the netbook with very minimal delay between the capture and the display. A problem arose when we tried to stream the video data from the netbook (server) to the client. We downloaded and used the software Broadcam (figure 6), which allowed us to stream the captured video as a series of images to a website. The Internet Protocol (IP) address of this website was never constant because it was dependent on the netbook which did not have a static IP address. Consequently, the client always needed to verify the server IP address in order to view the stream. This became a nuisance especially when the server would lose its internet connection. Eventually, when we realized the UNH wireless network would not cooperate with our needs, we switched to using an ad-hoc (or point to point) network. The Broadcam software was no longer usable because it required an active internet connection. Figure 7: Snapshot of the Broadcam software setup menu
Research into other methods of video streaming led us to VideoLan (VLC Media Player). This is an open source media player that is capable of streaming video over any type of network connection to a single IP address or to multiple IP addresses. We decided to stream to only one IP address, the client. Setting up the stream required selecting the active webcam and microphone and entering the IP address of the client. Once the stream was set up, the client simply needed to open VLC media player, open the port where the video was sent and then what the hummer was seeing could be viewed on the remote laptop (Figure 7). Figure 8: Snapshot of VLC Media Player showing the captured stream The video streaming speed was limited due to the processing power of the netbook and thus the delay from the camera to the client was about 5 seconds. We attempted to view the video while sending movement commands to the hummer, and this did not end as expected. Since WiFi is half-duplex, the movement controls for the Hummer we sent got delayed by several seconds and it was visibly obvious that the Hummer was not responding to some commands. We decided not to use the video streaming while we were controlling the Hummer and would only turn it on while the Hummer was standing still. This still completed our goal for this portion of the project, because we had achieved an easy way to stream video from the server to the client. Additional Goals Research: The additional goals were objectives that we wanted to accomplish once the deliverable goals were achieved. These goals included autonomous movement, audio streaming, elevator accessibility, a laptop battery back-up solution, and on and off device data storage. Research was completed on the elevator accessibility, audio streaming, and the laptop battery solution. The elevator accessibility is discussed in the next section. We made audio streaming an additional goal, because we thought that the microphone would only pick up the noise from the motor, and that we would have needed to design some sort of filter. This did not end up being the case and was easily included with the transmission of video. In regard to the netbook battery, we did not need to have a backup solution because the duration of the battery was over 6 hours. This was tested battery by connecting it to the wireless internet and reloading a webpage every 2 minutes. We did not have time to devote to autonomous movement, but did discuss how we would try to implement the idea. We figured we could set the Hummer on the ground, and give it an initial speed. Then the Arduino would interpret the data from the sensors to control the rest of the movement, such as stopping in front of obstacles, reversing and finding a separate way around. This would have been really cool to accomplish, but unfortunately we ran out of time and had other goals to accomplish. The on and off device storage was not feasible for this project. We wanted to be able to save the videos in order to go back and view them later, but again the processing power of the netbook prevented us from accomplishing this goal. It became
apparent that streaming the video and movement commands was too much for the Hummer to handle and adding the recording of the video was not going to be a possibility. Elevator Accessibility: The elevator accessibility was partially met. We designed a circuit (Figure 9) to allow the Hummer to push an elevator button with a solenoid. The Hummer had an infrared transmitter onboard and a receiving circuit was attached to the wall, leading to solenoids located over the elevator buttons. The idea was to send a binary code through infrared transmission, which would be interpreted by digital logic. Based on the code received, the logic would activate the corresponding solenoid. To implement this idea we had an infrared detector lead to a Schmidt Trigger which would define a logic high and low level. This output went to a ripple counter, which would count on every falling edge. The count was converted to a binary number, which was fed into logic gates to drive the corresponding solenoid. However, a major problem that was not predicted occurred. The output of the Schmidt Trigger had hysteresis along with noise on the falling edges of the clock counts, which in turn created incorrect counts. The inability to accurately count prevented us from using this concept. Figure 9: Original Circuit Design A proof of concept circuit (Figure 10) was created instead. The same photo detecting circuit was implemented and the output became the input of a voltage comparator that was constructed using an LM 311 operational amplifier. The voltage comparator had a threshold voltage of 350 mv. This ensured that when infrared was not sent the comparator would output 0 volts, or a logic low. When infrared was received and the voltage exceeded the threshold, a 5 volt logic high was outputted. This was sent to a 12 volt relay, which activated the solenoid simulating a button press on an elevator. The implement circuit is shown in Figure 11.
Evaluation: Our original problem statement outlined how first responders and safety personnel cannot enter an unknown area and survey it in a safe and timely manner. Knowing this, we designed a way to take out the human factor. The proposed problem solution was to be able to control a vehicle from a remote location and do the necessary work. Accomplishing our project goals allowed us to do just that. Wireless transmission between the Hummer used and the remote computer allowed us to safely survey an unknown area. For the purposes of this project, we used Kingsbury Hall to test the functionality of the Hummer. Video and audio transmission allowed the user to see and hear everything that the Hummer was able to see and hear. Obstacle detection was implemented so that the Hummer would automatically stop moving if the remote user was trying to drive it into an unseen wall or obstruction. Accomplishing these goals was a good way to simulate how the Hummer could actually be used in a real life scenario. There were a couple aspects of the project that we would have completed differently if given the opportunity. The first would have been to use a more powerful computer rather than use the netbook. A more powerful computer could have allowed for simultaneous movement and video feedback as we believe the netbook could not process the information fast enough. The second aspect would be to use a different camera. We feel as though the camera could have been better, but understand it was only for proof of concept. A superior camera may have yielded better picture quality at the receiving end. Given the opportunity to continue working on this project, we would finish implementing the relative motion detection, create a user friendly GUI, and continue debugging the original elevator accessibility design. If this project were to continue through another senior project team, several enhancements could be made. One enhancement could include controlling the Hummer from something other than a computer, for example a tablet, ipod, or Smartphone. A second possible enhancement could be to have the Hummer move autonomously. Moving autonomously removes the need for human interaction. If the Hummer was able to map out the interior of a building, it would know where to go and how to get there before a human did. This project was an excellent experience for us. We gained a lot of usable experience in designing and troubleshooting circuits that directly related to our project. We learned how to write a project proposal, stay within the proposed budget and how to use meetings effectively to keep everyone up to date and on track. We found out how difficult it is to stick to a week by week timeline that was created in the beginning of the year. Most of the goals were accomplished by the end of the year, but not everything matched perfectly to the timeline. Overall, the project was successful and it was a lot of fun.