Performance Test Results TEST REPORT In order to show compliance with specifications for bid submittal and verification purposes, several performance tests were conducted. The following sections contain a description of the test procedures, photos of the test area and test results. Frontage Road and I-15 in Lehi, Utah A performance test was performed on 10/31/06 across ten lanes of traffic at a point along the I-15 freeway where a frontage road paralleled the freeway. The sensor was placed on the west side of the freeway just south of Thanksgiving Point in Lehi, Utah. The frontage road consists of lanes 1 and 2 and the freeway consists of 4 lanes each direction in which the innermost lanes are high occupancy vehicle (HOV) lanes. The near boundary of the first lane was 6 feet from the sensor; the far boundary of the last lane was 252 feet from the sensor. Figure 1 shows a photo of the site from the perspective of the sensor and the geometry of the test site along with lane descriptions. East 32 ft 240 ft 228 ft 216 ft 204 ft 158 ft 146 ft 134 ft 122 ft 32 ft 6 ft 34 ft 3 ft Lane 1 Lane 2 Lane 3 Lane 4 Lane 5 Lane 6 Lane 7 Lane 8 Lane 9 Lane 10 (Frontage Road) Figure 1. Frontage Rd. and I-15 in Lehi, Utah, photograph and test dimensions A 30-minute dataset was collected, which included video from three different cameras and an event list from the SmartSensor HD. The last lane (Lane 10) was not automatically detected because it was not completely inside the 250-foot range of the sensor. Other than the manual addition of Lane 10, no other tuning or adjustments were made to the sensor. Other than the manual addition of Lane 10, no other tuning or adjustments were made to the sensor. WX-501-0009 2.9.16 1 of 7
1 1 0 68 69 1.47 With Lanes 3 6 2 0 1 56 55 1.79 With Lanes 7 10 3 7 0 471 478 1.49 4 10 4 528 534 1.14 5 8 3 549 554 0.91 6 2 4 148 146 1.35 7 1 8 155 148 4.52 8 0 16 551 535 2.9 9 9 19 561 551 1.78 10 8 21 418 405 3.11 I-15 in Springville, Utah Table 1. Results from Frontage Rd. and I-15 in Lehi, UT A performance test was conducted on I-15 at the 1500 North exit in Springville, Utah between 10:50 a.m. and 11:30 a.m. The freeway consists of three northbound lanes and three southbound lanes with a three-foot jersey barrier between the two directions of travel. The sensor was placed on the west side of the freeway looking across traffic (see Figure 2). 1.11 2.82 East 32 ft 127 ft 113 ft 100 ft 70 ft 52 ft 40 ft 28 ft 6 ft 30 ft 3 ft Figure 2. I-15 and Springville 1500 North, photograph and test dimensions Lane 1 Lane 2 Lane 3 Lane 4 Lane 5 Lane 6 A video camera was mounted just above the sensor and the recorded video was used as the truth data for comparison to the sensor event lists. The sensor was configured using the auto-configuration feature and no manual adjustments to the lanes or tuning of sensor parameters was performed. A 15-minute data set was taken at 10:50 am and a 10-minute data set was taken at 11:30 am (see Tables 2 and 3). WX-501-0009 2.9.16 2 of 7
1 3 1 193 195 1.04 2 3 6 246 243 1.22 1.29 3 5 3 125 127 1.6 4 0 1 146 145 0.68 5 3 1 236 238 0.85 0.99 6 3 1 140 142 1.43 Table 2. 15-minute data set from I-15 & 1500 North in Springville at 10:50 am 1 3 0 170 173 1.76 2 3 2 194 195 0.52 3 2 3 115 114 0.87 4 0 3 156 153 1.92 5 2 2 225 225 0 6 5 0 147 152 3.4 I-95 in Richmond, Virginia Table 3. 10-minute data set from I-15 & 1500 North in Springville at 11:30 am A per vehicle speed measurement performance test was conducted on 7/17/06 in Richmond, Virginia. The freeway (I-95) consists of two northbound lanes with fairly heavy truck traffic. A highly calibrated Piezo sensor system was used to generate the truth data for this test. Since the Piezo sensors were only installed in the two northbound lanes, these lanes were included in this test. No calibration, tuning or tweaking was performed on the sensor before performing this test. Individual speeds for vehicles were collected on both the Piezo system and the for a 15-minute period from 8:45 am to 9:00 am. Table 4 shows the number of cars detected by the and the number of detections for which per vehicle speeds were reported. The results show that, at this location, per vehicle speed measurements were reported on 99.50% of detections. 1.05 1.77 Total number of cars detected by Number with valid speed Percent with valid speed 212 211 99.50% Table 4. Number of cars detected by the with valid speeds Table 5 on the follwing page show the number of vehicles whose measured speed on the differed from the Piezo measurement. For example, there were 51 per vehicle speed measurements that were two miles per hour higher than the Piezo sensor measurements. Table 6 below shows the number of per vehicle speed measurements that were within the specified accuracy ranges. For example, there were 182 measurements (87.5%) that were within ± 4 miles per hour of the Piezo measurements. WX-501-0009 2.9.16 3 of 7
Error Number of Vehicles -10 0-9 0-8 1-7 0-6 0-5 0-4 1-3 0-2 5-1 11 0 21 1 37 2 51 3 38 4 18 5 10 6 7 7 6 8 1 9 0 10 0 Over 10 1 Total Vehicles: 208 60 50 40 30 20 10 0 Speed error histogram -10-9 -8-7 -6-5 -4-3 -2-1 0 1 2 3 4 5 6 7 8 9 10 Error (mph) Table 5. SmartSensorHD and Piezo speed measurement differences; histogram of same data Table 6 below shows the number of per vehicle speed measurements that were within the specified accuracy ranges. For example, there were 182 measurements (87.5%) that were within ± 4 miles per hour of the Piezo measurements. Range Number of Vehicles Total Vehicles % Within Range ± 3 mph 163 208 78.4 ± 4 mph 182 208 87.5 ± 5 mph 192 208 92.3 Table 6. Percent of vehicle speed measurements within each accuracy range Note. Upon examining the histogram plot in Figure 5, it becomes evident that the speed error from the sensor is biased 2 mph high for this particular data set. If the speeds were tuned to adjust all speeds down by two mph, which is possible using the SmartSensor Manager HD software, then the results shown in Table 7 would have been achieved. WX-501-0009 2.9.16 4 of 7
Range Number of Vehicles Total Vehicles % Within Range ± 3 mph 186 208 89.4 ± 4 mph 198 208 95.2 ± 5 mph 204 208 98 Center Street in Orem, Utah Table 7. Percent of vehicle speeds falling within each accuracy range after tuning A classification accuracy performance test was conducted on 12/7/06 at 1400 North Center Street in Orem, Utah at approximately 3:30 p.m. A video camera was mounted above the sensor to provide the truth data for comparison. This street consists of three lanes in each direction with a 14-foot painted gore area. Cones were placed close to the street 10 and 20 feet apart to aid in classification verification (see Figures 6 and 7). West 25 ft 108 ft 96 ft 84 ft 80 ft 67 ft 55 ft 43 ft 14 ft Lane 6 Lane 5 Lane 4 Lane 3 Lane 2 Lane 1 Figure 3. 1400 North and Center St. in Orem, Utah, photograph and test dimensions The was configured using the auto-configuration feature and no tuning, tweaking or manual adjustments were made to sensor parameters. Table 8 below shows the classifications boundaries that were set in the sensor. Class Number Class Boundaries 1 0 10 feet 2 10 25 feet 3 25 45 feet 4 Greater than 45 feet Table 8. Classification test settings WX-501-0009 2.9.16 5 of 7
Event data and video was collected for approximately 11 minutes to ensure at least 50 cars appeared in each lane. The classes reported in the event list were compared against the actual length of the vehicle as determined by the video (see Table 9). Lane Number Total Directions Number of Class Errors Error % 1 83 3 3.6 2 121 9 7.5 3 105 2 1.9 4 117 0 0 5 85 3 3.5 6 83 7 8.4 I-15 in Springville, Utah Table 9. Classification results A performance test was conducted on 10/20/06 in the presence of a barrier on I-15 at the Springville, Utah 1500 North exit from 10:50 am to 11:05 am. This location consists of three northbound lanes and three southbound lanes with a three-foot Jersey barrier between the two directions of travel. The sensor was placed on the east side of the freeway looking across traffic (see Figure 4). West 23 ft 123 ft 110 ft 96 ft 60 ft 47 ft 35 ft 22 ft 6 ft 30 ft 3 ft Lane 6 Lane 5 Lane 4 Lane 3 Lane 2 Lane 1 Figure 4. I-15 and 1500 North Springville, Utah, photograph and test dimensions The was configured using the auto-configuration feature and no tuning, tweaking or manual adjustments were made to sensor parameters. During the test, video was recorded to use as the truth data for comparison to the event lists collected from the sensor (see Table 10). WX-501-0009 2.9.16 6 of 7
1 1 10 205 196 4.39 2 3 6 271 268 1.11 3 6 2 218 222 1.83 4 0 0 180 180 0 5 1 3 295 293 0.68 6 5 2 225 228 1.33 Conclusion Table 10. I-15 and 1500 North Springville, Utah Test Results for 10:50 am As evidenced by the test results presented in this document, the provides a high level of performance in many different installation sites with a variety of offsets and mounting heights and in the presence of gore and barriers. 2.44 0.67 WX-501-0009 2.9.16 7 of 7