USF Libraries
USF Digital Collections

An un-cataloged item [C01-00446] from Center for Urban Transportation Research Publications [USF].


Material Information

An un-cataloged item C01-00446 from Center for Urban Transportation Research Publications USF.
Physical Description:


letter   ( marcgt )


General Note:
Full cataloging of this resource is underway and will replace this temporary record when complete.

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
usfldc doi - C01-00446
usfldc handle - c1.446
System ID:

This item is only available as the following downloads:

Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader ntm 22 Ka 4500
controlfield tag 008 s flunnn| ||||ineng
datafield ind1 8 ind2 024
subfield code a C01-00446
0 245
[An un-cataloged item [C01-00446] from Center for Urban Transportation Research Publications [USF].]
Full cataloging of this resource is underway and will replace this temporary record when complete.
1 773
t Center for Urban Transportation Research Publications [USF]
4 856


. .. Flelct-Perfotmance . . . Evaluation .' . . ... : . . .. . . . . ,. : . . . . : . AUTOSENSE Axle Counterna at leesburg Malnlne Toll Plaza Rna/ Repott January 2003


FIELD PERFORMANCE EVALUATION OF AUTOSENSE AXLE COUNTER AT LEESBURG MAINLINE PLAZA Prepared for Florida Department of Tra nsportation Florida's Turnp ike Enterprise Prepared by Center for Urban Transportation Research University of South Florida-College of Engineering 4202 E. Fowler Avenue, CUTl 00 Tampa FL 33620-5375 January 2003 The opinions and findings expressed in this report ar e those of the Center for Urban Transportation Research (CUTR) and the University of South Florida and not necessarily those of the Florida Department of Transportation-Florida's Turnpike Ente rprise or the project's technology partner (Schwartz Electro Optics, Inc.) This report has been prepared in cooperation with the FOOT, the Turnpike Data Center in Boca Raton and Schwartz Elect ro-Optics, Inc.


CUTR wou ld like to recognize the funding agency for t h is evaluation p r oject, the Florida Department of Transporta t ion-Florida's Turnpike Enter p rise (formerly, Office of Toll Operations). In particular, Gerald Coleman and Brett Massey who provided on-site coordination at the Leesburg Mainline To ll Plaza. Additi onally, Ms. Kathy Runk at the Turnp ike Data Center in Boca Raton, provided plaza transaction r eports to CUTR on a daily basis throughout the evaluatio n period. CUTR sincere l y appreciates the opportunity afforded by t he FOOT Florida's Turnpike E nterprise to conduct this independent f ield evaluation on their behalf. The technology vendor partner in this eva luat ion was Schwartz E l ectro-Optics, Inc. (Orlando, FL), represented by Eric Carr Director of Commercial Products. The CUTR research team consisted of M i chael Pietrzyk, Firoz Kabir, and Shireen Chada. The othe r primary s t aff was joan Derival (Graduate Research Assistant), and Patricia Bapt i ste (Program Assistant). Without the assistance and effort provided by the aforementioned individuals, t his evaluation could not have been conducted and documented -


EXECUTIVE SUMMARY ............. .. ...... .... .... .... .... .... .... .... .... .... ... .... .... .... ..... .. ......... v 1 PROJECT BACKGROUND AND OVERVIEW .. ....... .. ............................................ 1 AUTOSENSE AXLE COUNTER'"' DEVICE DESCRIPTION ...... ... .... .... ... .... ...... .... .... 2 Overvie\v ...... ............. .. .... .... .... ......... ....... ........ ...... ......................................... 2 Device Specifications .. ..... .... .... ..... ..... .... ..... ........ ...... .. ... ....... ....... .............. .... 2 Concept of Operations ............ ... . ...... .. .. ... .. .... ...... ... ... . ...... ... .. ...... ............... .. 3 Typical Insta llat ion .. ..... ........................................... ... .... ... ...... .. .... .... ..... ... .... .. 5 TES TING METHODOLOGY ......... .......................................... .............. ....... .. ..... 7 Descri p tion of Source Data .... .. .. ... ... . ... .. .. .. .......... ... ..................................... ... 7 Da ta Formatting ................... .... ............................................... ...... .. ................ 9 Ground Truth Comparison ............................................................................... 10 EVALUATION OF AUTOSENSE AXLE COUNTER'" ............................................ 12 Ease of Installation .............. .................. ................... ........ ................. ............ .. 12 Compatibility with Existi ng lan e Equipment/Operations ................................... 12 ANALYSIS OF AUTOSENSE AXLE COUNTER'"' .................................................. 13 General Error Appraisa l ...... ...... .... .... ........... .. ........................................ ......... 1 3 Discussion of Individual Error Types ................................ ...... ........... ...... .. ....... 13 SUMMA_RY OF FINDINGS .. ............... .. ...... .... ... ...... ................................. ... ...... 18 Evaluation Results ...... .... ...... : ............... ..................... ....... ............. ...... ......... 18 SEO Planned Improvements ... .... ........ ....... ... ............ ... ........... ........... ... .... .. ... .. 18 Next Steps ...... .. ....... . ...... .................... ............... .......... .... ......... ....... ......... 19 APPENDIX A ............. ............. .. .................... .. ..... ... .... .... .... .... .... ... .... .... ...... ...... A 1 APPENDIX 8 .... .. ...... ........................ .. .... ..... ... ..... .... ... ..... .... ... .... ...... ................. 81 APPENDIX ( .................................. .... ... ...... ... .... .... ...... .................................. ... C1 -


Figure 1: AutoSense Axle Counter Dev i ce ............. ....... ..... .. ...... .. .................... .. .. 3 Figure 2: Axle Counter Veh icl e Detect ion Sequence ................... ..................... .. .. 4 Figure 3: Installation of Axle Counter a t Leesburg Mainline Toll Plaza .............. .. ... 5 Figure 4: Schemat ic of Leesburg Toll Plaza System Operation ... ..... .... .. ... ............ .. 6 Figure 5: Axle Counter Field Test: Data Collection and Evaluation Process ............ 8 Figure 6: Example of Correct Transaction Recording .............................................. 9 -


Table 1: ASAC Device Specifica t ions ... ... . ...... .. .... ... .... ..... .. .. ... ..... ... ....... .. .. ....... 3 Table 2: Ground Truth Comparison Classification of Error Types .. .. ... ...... .. ...... . 10 Table 3 : Summary Results of ASAC'M Evaluation .. . .. .... .... . . ........... ... .... . . ...... .. 15 Tab l e 4: Summary of Axl e Counter Only Errors ................ ........ .... ... .. .... .... ...... .. 16 -


The Flo rida Department of Transportation (FDOn F lorida's Turnpike Enterprise retained CUTR to evaluate the functionality, capability and accuracy of Autosense Axle Counter"' (ASAC). ASAC is a diode-lase r -based vehicle detection and classification sensor manufactured by Schwartz Electro-Optics (SEQ) of Orlando. The device is ideally located on a pole alongside the highway or a longside a toll lane, pointing downward toward the center of the traffic lane. ASAC is mounted five feet above the road surface and emits two laser beams at a fixed angle separation of ten ( 1 0) degrees onto the pavement. As a vehicle passes the device, the laser beams are broken and the device is able to generate a series of two-dimensional scans of the vehicle. Once the vehicle has passed fully through the beams, a three dimensional image of vehicle i s developed. This image is used to count the axles using in-built device algorithms. This device had the multiple capabilities: to detect, to separate and to classify vehicles. The evaluat ion took place at Leesburg Mainline Toll Plaza on the Florida Turnpike The data used in this report included the informa tion dating form May 17july 14, 2002, with a sample size of 30,535 vehicles. The objective of the evaluation was to assess the accuracy of ASAC as a vehicle classifier a nd separato r Comparison of the ground truth videotape with the transaction records generated by the plaza computer, as well as by the SEO data files formed the basis for this eva luation. T his ground truth comparison allowed real time evaluation of any errors within the system. The ASAC device evaluated in this r eport was a prototype. Over th e course of the evaluation period, a total of 30,535 vehicle t ransactions were assessed. There were 370 errors attributed to the ASAC. The ASAC device accuracy at 95% confidence level is greater than 98. 6 7 % -


A Schwartz Electro-Optics (SEO) Autosense Axle Countef'M (ASAC) was loaned to the Florida's Turnpike Enterprise for the preliminary evaluation of th e unit's adaptabi lity and functionality with the existing Florida To ll s System. (For more information abou t SEO refer to Appendix A). The FOOT Florida's Turnpike Enterprise r etained CUTR to evalua t e the performance of ASAC installed and adjacent to l-a ne 3-B at the Leesburg Mainline To ll Plaza. T he performance eva luation is similar to that previously conducted by CUTR, to assess the accuracy of AutoSense as a vehicle separator. The evaluation of ASAC consisted of the follow ing : F lorida's Turnpike Enterpri s e installed one ASAC sideway unit i n Lane #3-B at the leesburg mainline plaza. The output signals for vehicle separation and classification were obtained exclusively from the ASAC unit. T his was compared directly with the corresponding video ground truth. ASAC was tested for the vehicle separation and classification A thirty-three day operation test for an 8-hour period of each day commenced immediately after ASAC: was installed and verified for operational performance CUTR prepared this performance evaluation report, including verification of statistical significance of the ASAC vehicle axle count performance. -


Overview AS AC is a diode-laser-based vehicle detection and classification sensor (VDC}. This device was designed by Schwartz Electro -Op tics (SEO} and was developed under an Intelligence Transportat ion Systems Ideas Deserving Exploratory Analysis (IT S -IDEA) program for the National Academy of Sciences. ASAC has the capabilities to detect th e presence of a vehicle, to separate vehicles and to count the number of axles per vehicle. It also provides other type of i n forma tion such as the ve hi cle's timing and speed ASAC is usually posit i oned on a pole alongside a toll lane, po i nting downward and toward the center of the lane. T h rough its field-of-view, ASAC f irst scans the road leading to range measurements across the width of the road at two locat ions in front of the device. These measureme nts are processed to generate messages that uniquely detect, separate, and quan tify each vehicle along with providing speed and camera trigger i nformation Device Specifications ASAC starts detecting vehicles automatically upon power up. ASAC is a user friendly device that doesn't require any f ie ld adjustment due to its self calibration. (See Tabl e 1 for specifications of the device ) The laser device is enclosed i n a solid case and requires l i ttle ma i ntenance T he only requ ireme nt for preventive maintenance is keeping the window clean. A d irty window can result in range errors. SEO recommends cleaning the window every six months with an optical lens tissue to prevent scratches. ASAC hous i ng is nitrogen purged, hermitically sealed prior to shipping, and should be opened only in a laboratory environment by SEO personnel. SEO staff set all calibration and alignment adjustments during the -


final test, with no field adjustments being requ ired. Initial installation of the device takes 1-2 hours and repair time i s 10-15 minutes. The device is designed to have an unlimited life span. After 35,000 hours o f use, which corresponds to a period of four years, the device should be returned to SEO for an overhaul servicing. F igure 1 displays t h e ASAC device. Table 1: ASAC Device Specif ication s Size Power Requirements 19:10 in x 9.65 inX5.90il) .a. 90-140V, 5Qc60 Hz, 1.5A b. 200-264 V, 50-60 HZ, 1.0A Figure 1: AutoSense Axle Counter Device Concept of Operations The ASAC dev ic e emits two narrow beams at a fixed angle separation of 1 0 degrees. T he first beam has a look down of 1 0 degrees and the second beam has a look down of zero degrees as shown in Figure 2. At a mounting height of 5 feet, this 1 0-degree separation equals the distance of one foot between th e two l aser beams. The laser beams continuously scan the road at a rate of 360 scans per seconds. Figure 2 shows the detection messages produced by ASAC as a vehicle passes through the ra nge of the device. A sideway height profile of the vehicle is generated each time the vehicle is scanned. Once the end of the vehicle has been detected, ASAC generates a three dimensional image of the vehicle by calculating the vehicle speed and combining the sequence of the two-dimensional scans. Once the three dimensional image is obtained, the ASAC device classif ies the vehicle -


using a built-in algo ri thm. In i ts configuration at leesburg Plaza, ASAC i s utilized as vehicle separator and classifier. Figure 2: Axle Counter Vehicle Detection Sequence Direction of Data Messages First Beam Sequence Message Number 1 o f 5 1 !Beam Vehicle Deted ion 10 ... 49 2 ofs 2n:s Beam Vehicle Detection 1 0 49 8egi n of Vehicle: 23 Speed in (lnph), 6 3 of s 1n8eam End 10 49 End of Vehicle, 29 4 of 5 2 "" Bea m End o f Veh i cle ID 49 Camera trigger for vehicle li<.:ensc tag s or s C l ass i fication 1 0 4 9 Length: 1 8.25 ft Axle count: If axles Exi t Speed(mph ) : 6 -


Typical Installation During the third week of May 200.2, FOOT staff helped with the installation of the ASAC as well as the video equipment in Lane #3-B at the L eesburg Mainline Toll Plaza. The installed ASAC device is shown in Figure 3. FOOT staff angled the device so that the laser beams contacted the paveme nt surface just in front of the treadle. Figure 4 shows a schematic of the Leesburg Toll Plaza system operation. The autom atic vehicle classification, AVC device, was responsible for i n form i ng the lane controller of the beginning and end of each vehicle that passed throug h the plaza, so that the vehicles could be correctly separated. AVC device was also responsible for signaling the end of the transaction to the system. ASAC's ability to separate and classify vehicles was utilized in the evaluation. Figure 3: Installation of Axle Counter at Leesburg Mainline Toll Plaza -


Figure 4: Schematic of Leesburg l oil Plaza System Operation Auto Sense AKI& Counter QNX Lane Co ntrol ler Aiming Loop CCTV System Computer CCTV I mage Overla y To il Collector Plaza Computer I Text File Toll Terminal Re<:eipt Printer Swipe Reader


The intent of the evaluation was to assess the accuracy of ASAC by comparing three sources of data: the ASAC data set provided by SEO, the ground truth videotape of Lane #3-B at Leesburg Toll Plaza, and the transact ion record generated by the to ll plaza computer. The basic process used to obtain the data requ ired for the eva lu ation is shown in Figure 5. Description of Source Data As is shown in Figure 5, there were three sources of data. Two of these sources, the ground t r uth videotapes and the transaction records were from T he Leesburg Mainline Toll Plaza and provided by the Florida Department of Transportation. The third source were the ASAC data provided by SEQ. CCTV Videotape T he first data source was the Closed Circuit Television (CCTV) videotape, obtained from Leesburg Toll Plaza. A CCTV camera was used to view the tol l booths during operational hours For each vehicle passing the tollbooth, a real time transaction information is generated on the scr een by the CCTV compu ter. This visual image was recorded onto videotape at the toll plaza and an example of the type of image obtained is shown in Figure 6. T he plaza clock was displayed within the overlay, allowing accurate real time transaction evaluation. Staff at Leesburg Toll Plaza sent t he videotapes to CUTR. Plaza Computer Text Files The second data source is the L eesburg Plaza computer text file. An example of this text file is shown in Table B 1, A ppe ndix B. T his file is the final transaction record of each day as collated by the p laza computer, and contains an extensive row of transact ion data for each vehicle proceeding throug h the plaza. The staff at Boca Raton Data C ollec tion Center was r esponsible for identify i ng the data required for the evaluation and sending it to CUTR via e-mail for the duration of the evaluation. ASAC Data from SEO The SEO team provided the electronic text files for the ASAC transactions via e-mail. -


Figure 5: Axle Counter F i eld Test: Data Collect ion and Evalua t ion Process CCTV videotaping o(Leesburg Malnl111e Toll Plaza Operation (Lane #5) SEO's Autosense Axle Count e r ASAC Data Files from SEO Videotane .

Figure 6: Example of Correct Transaction Recording Data F o rmatti ng frame 1 A tractor approaches the toll plaza. frame 2 It is registe red by the toll col l ector as a three-ax l e vehicle as shown by the "C13 code in Fw d section of over l ay. The toll collecto r receives payment. The tractor leaves the to ll booth An example of the toll plaza transaction record text files, comple t e with an exp l ana t ion of the column headings, is shown in Table Append i x B Table B-1 shows that these text files det ail each transact ion recorded at leesburg Plaza. T he files did conta in indepe ndent data related to the operation and o u tput from the ASAC dev i ce. Therefore, in order to assess the acc uracy of ASAC, t h e number of "forward axles" allocated to each vehicle is counted. The text files were imported i n to Microsoft EXCEL Spreadsheet, f o r matted t o iso_late the required data ( th e number of f orward axles allocated to each vehicle and time of transaction), and printed out for use in the ground truth comparison. There a r e three differe nt types of axle counts giv e n t o each tra n saction r ecord Appendix B-1 conta ins d ef i ni t ions of these three axle allocations. The f orward axle count i s the number of axles a lloca ted to the vehicle by the toll collector, and t h e obs erved axl e count is th e raw number -


5 of axles viewed on the CCTV videotapes. The SEO axle count is the number of axle determined by the ASAC. Ground Truth Comparison The evaluation commenced on May 17, 2002. FDOT staff at the plaza was asked to provide videotapes fo r three days, every week (Friday, Saturday, and Sunday), from 1 Oam to 6pm, for the observation period, which ended on July 28, 2002. Staff at Boca Raton Data Collection Center was asked to send daily output text files for the corresponding days. The SEO team was respon sible for sending electronically, the ASAC text f iles to CUTR, on a weekly basis. This third source of data was a daily summary of all the vehicles detected by the ASAC in lane 3-B. Information for every vel:licle was provided such as a time stamp, the axle count, their heigh t, width, length as well as their spee d. During the stage of the evaluation, CUTR analysts watched the videotape of the plaza operation and compared what they saw with the transaction data registered on CCTV overlay, in the text fi le, and the ASAC output files An example of a correctly recorded transact ion sequence was previously shown in Figure 6. If the transaction record did not correspond with the ground truth video, the transaction was recorded as an error. Pretest a na lysis of the videotapes indicate<;l that there were varieties of transaction error types that occurred at th e plaza. A careful analysis of the various errors indicated that these errors fell into six main categories. The utilization of these six categories provided the level of delineation required to accurately define all the errors that occurred at the plaza. Table 2 details the error codes and their unique definitions. Table 2: Ground Truth Compa rison Classification of Error Types I nco rrect numbe r of axles allocated by ASAC cto vetiicle due to error


Error Type E T his error is assi gned when a wrong axle numbe r was alloca ted to a vehicle by ASAC. This error type was d i fferentiated from a separation error by considering the trans action pre c eding and following th e er r o r transaction. I f the preced i ng and following vehicles were allocated the cor r ect number of axles in the transaction record, then the error was logged as an "E" type error. Tllis differentiates th i s e rror type from a separatioh error (S) when additiona l axles would be alloca ted to eithe r the preceding or fol lowing vehicle. This error type E can be attribu ted to ASAC. Error Type f Th i s erro r is assigned when the transact ion record appears correctly on t h e CCTV v i deotape, and in the ASAC text f ile, b u t was i ncorrect in t he FDOT text file. Clear l y, this error is a result of wrong number of axle reco r ded in t he FDOT tex t file Th i s error type is not attributed to ASAC. Error Type "C T his category describes t h e s i tuation when the transaction record appears correct l y on t he CCTV v i deotape and the ASAC text files but no transaction was recorded in the FDOT text file This error type i s no t attributed to ASAC. Error Type "/" This category of error was assigned to vehicles whose axl e s a ll ocation couldn' t be visually confirmed on th e CCTV videotape. T h e t ransaction r ecords of t h e FDOT text files as well as the ASAC t ext files we r e two sources of information for these veh i cles. Howeve r due to poor weather conditions or unusua l visua l effects, the axle allocation cou ldn't be determined. T herefore, those cases were put aside and weren't assigned to any potent i al source of errors This error type i s no t attr i bu ted t o ASAC. Error Type s T his category of error was assigned for incorrec t vehicle separation by the ASAC. This er r o r i s defined as an incorrect number of axles allocat i ons for consecutive veh i cles. Separation errors are often the result of an inco rr ect axle count of two or more consecutive vehicles. This type of error can on l y be attributed to the ASAC. Error Type "A" This error type is assi gned when the number of axles for a vehicle was incorrect in both t h e FDOT and the ASAC text files. Clearly, this describes the s i tuation when the transaction reco r d appears correctly on the CCTV overlay, but the transaction was incorrect l y r ecorded in the FDOT text file and t he ASAC text f iles. This error type can also be contributed to ASAC. -


Ease of Installation The device is usually mounted at a height ranging from 1.5 to 2.5 meters (4 and 6 feet) high and typical l y located on a pole alongside the highway or alongside a toll lane pointing downward and toward the center of th e lane. T he input power connector used for the device is a three-conductor power cab l e shielded with the requ irement size of 18 AWG (American Wire Gauge). SEO in c. provides an environmentally sealed cable. Ensuring proper installation, the three conductor power cable should be connected to the specified pow er and ground connections Initial insta llat ion of th e ASAC takes approximately one to two hours. It is a user friendly laser device. SEO Inc. sets the calibration and a l ignmen t adjustment. It doesn't require any additional adjustment by the end-user. Additionally, to i mprove the surface reflectivity of the roadbed, one or two stripes can be painted across the traffic lane to ensure the proper operation of the ASAC. Compatibility with Existing lane Equipment/Operations The ASAC is a stand-alone-device that requires little adjus tment to toll operations. It only requires the insta l lation of a pole alongside the edge of the toll lane and power source for the power cable of the ASAC.


The ground truth comparison phase for this eva I uation was performed from May 17 to july 28, 2002. Duri ng that period, a tota l o f 30,535 vehicles were reviewed T able 3 shows the results summarized fo r th i s eva lu ation G ene ra l E r r o r A p pra isal The six categories of errors (A, C, E, F, I, and S) show n in Table 3 help to identify the different occurrences in which vehicles a r e cl assif ied incorr e ctly. Table 4 identifies t he ASAC on l y errors. The occurrence rate for each type of error has f l uctuated from 1% to 4% ove r the duratio n of the evaluation period. Vehicles on june 7 to j un e 9, 2002 we r e not accounted f or because the ground t ruth video didn' t focus on lane 3 -B. D iscussi on o f Ind ividual Err o r T yp e s Error Type "E" This was the niost common error recorded with a freq u ency rate of 0.95% a n d describes the instances when there is a n i ncorrect axle allocat i on by t h e ASAC device. Th i s type o f error occurred especia ll y with vehicle pulling ano t her veh i cle or a t r ailer, was simply misclassi f ied by the device Othe r occurrences h appened when mu l t i -axle vehicles were a lso misclassified. These errors were attributed t o ASAC. Error Type f This category of error has the second highest frequency rate of 0.6 4 %. Th i s type of error describes the s i tuation when the transaction record appears corr ectly on the CCTV and in th e ASAC text files, b ut is inco rr ectly recorded in the FOOT text f ile. This error type represents th e eve n t o f the CCTV computer recording the transaction corr ect l y, which was t hen rendered incorrect by the action of the plaza computer Error Type c T his e r ro r has a rate o f 0.29%. It occurs when the CCTV ground truth v i deo and the ASAC device co rr ectly recorded vehicles, but no transaction was r ecorded in the FOOT computer text files. T he Toll collec t or is a majo r source o f this type of error


because during per iods of inconsistent transaction recordings, vehicles can be missed completely by the toll collector


Table 3: Summary Results of ASACTM Evaluation 5/18/02 Sat 1188 5/25/02 Sat 1815 6/01/02 Sat 1670 6/08/02 Sat 6/15/02 Sat 1708 .. 6/29/02 Sa t 1 180 7/13/02 Sat 1519 (C !.[I n1J.JI.-'-1' Q nwd \-., o 1 .-tcl'(n "''' 2,220 6,940 11,054 1 2 217 15,339 24,344 28,895 E f 4 5 8 5 3 9 22 12 l'' .E ,r,O r .. j c 1 s A --8 1 2 0 .... .. 7 0 0 0 .. 5 4 0 0 2 2 0 2 ---............ 11 6 3 1 0 1 ........ .......... 37 1 3 2 17 1 ........ 2 1 1 2 3 2 4


Table 4: Summary of Axle Counter Only Errors 5/18/02 Sat 1188 2220 4 2 0 . 0.51 5/25102 Sat 1815 6940 8 0 0 . 0;44 7/13/02 Sat 1519 28,895 3.7 1 7 1 3 : 62 .1.1> (Average)


Error Type t Th i s error type is assigned to vehicles whose number of axles couldn't be clearly identified on the videotapes. For error "1", only two sources of information were available such as the transaction record in the FOOT computer text f iles and the results f rom the ASAC device. In some inst ances due to weather conditions, the r esults couldn't be confirmed by visually classify ing those vehicles. T herefore, these errors weren't attributed to any potential sources of error. In our sample, 0.08% of the total numbe r o f vehicles fell i nto that category. Error Type s T his error type is related t o incorrect vehicle separation and theref ore was only attributed to ASAC. In each case, two or more consecutive vehicles were inco r rectly classified. T his was done to show that the total number of axles over the transa ction period was correct and hence the incorrect axle allocation inust be due to a separation error a nd not a typ e f error. The frequency rate for error type s was 0.21% of the total num ber of vehicles. These errors were attributed to ASAC. E rror Typ e 'A" The error type is defined for vehicles whose transaction records in th e FOOT computer text files as well as results from the ASAC text files were erro neous. T h i s FOOT error type has a frequency rate of 0.05%. This category of errors was attributed to vehicles that had both error types f and "F". These errors were attributed to ASAC.


Evaluation Results Over the course of the evaluation period, a tota l of 30,535 vehicles transactions were assessed. There were three potential sources of error attributed to the ASAC device, the error types "E", "S", and "A". The error type 'E", assigned to vehicle axle misallocation by the ASAC, has frequency ra te of 0.95"/o. The error type "S", defined as separation errors, has an error rate of 0.21 "'o. The error type "A", attributed to both the ASAC device and the FOOT computer text f iles, occurs with a rate of 0.05"/o. There were a total of 370 errors attributed to the ASAC (as shown in Table 4). Appendix C includes a description of the statistical analysis conducted for this evaluation. Statistically, there were 30,535 (N) potentially successfu l transactions from the ten data colleciion instances and 30,165 (X) successful transactions. This results in a 0.9879 successful proportion (P), or a 98.79"/o nominal ASAC d evice accuracy. Based on this value of P, the accuracy in terva l for a 95"/o confidence leve l is between 98.91 "'o and 98.67"/o. T his result is best expressed, "The ASAC device accuracy at the 95"/o confidence level is greater than 98.67"/o." SEO Planned Improvements The above results were presented to SEO staff. They indicated that the axle counter was a prototype and NOT the final version. SEO staff has indicated that the following changes were made to the axle counter since its inst allation at Leesburg: )> Improving the quality of the input data through the addition of a high voltage feedback control to attain the best intensity data. This change was implemented in the Open Road version of the Axle Counter and results improved by 0.5 percentage points, as reported by SEO. )> Increasing the laser pixel resol ution per scan while a vehicle is present by incorporating a dynamically changing spread of pulses within each scan using half angle steps for higher resolution at the base of the vehicle. Also, some of the planned changes (as per SEO) to the prototype axle counter before it is ready for sale are: -


)> Refinement of the "Shape Based" a lgor ithm to more efficiently utilize processor l oading )> Increase the size of the inpu t buffer for the laser (vehicle) data before t he a l gorithm can process it s6 that the long tractor trailers do no t fill the buffer. )> Incorporate the developed, but as yet impleme n ted "Range based" algorithm. The new hardware environment has greatly enhanced our ability to achieve less tha n 3 inch accuracy, thus the data quality will enable us to utilize a second a l gorithm to verify the number o f axles seen. Next Steps The ASAC device evaluated in the study was a prototype, which is not yet ready for fie l d deployment based on the accuracy level results presented above. The manufacturer (SEQ) has indicated that it is current l y working on additional improvements of the device. This study recommends that once the p l anned improvements are i n corporated, the ASAC device be evaluated again for field performance


APPENDIX A AutoSense Axle Counter (ASAC) Product Information A1


Company History Founded in 1984, Schwartz Electro-Opt ics, Inc. (SEO) has achieved worldwide r e c ognition as a leader in the design and manufacture of solid-s tate lasers. SEO produces a variety of leading laser systems for the commercial and governmenta l markets worldwide. Curren t product lines include laser -based weapon simulat i on systems, laser sensors for tra ffic m anageme nt, precision farming and conservation. I n addit i on, SEO is engaged annu a ll y in government and priva t e l y funded researc h a nd p r oduct develo pme nt p roj ects The company i s headquarte red in Orl ando, F lorida with a subsidiary in South P lai nfie l d, New jersey. Since 1997, SEO has developed the Autosense products, an advanced laser scanning p ro duct line fo r toll and traff i c management. The Autosense devices have the capabilit ies to track, analyze traff i c over a w i de range of applications. Some of those applicat ion s are vehicle d e t ect i on, classification for a multitude of applications including toll collections, t raffic flow analysis, bridge/tunnel clearance etc. To d ay, over o n e thousand Autosense units are installed on highways throughout the world. I n the past two years SEO had successfully obta ined differ e n t contracts. In 2001, SEO was awarded the United States A rmy's Multiple In tegrated Laser Engagement System XXI con tra ct to b ui l d laser tra in i n g systems. This con t ract was va l ued at more than $70 million over the nex t five years. In M ay 2002, SEO was awarded a $1rni l lion contract with Cande l a Corporation to provide diode laser modules. Last May, the company in troduced their conservation art restoration laser to the world at the Salone del Restauro in Italy The privately owned high tech laser manufacturer had yearly sales totaliz i ng $26 million A2


. APPENDIX B Sample Transaction Record Text Files B1


Table B 1: Sample o f Excel File o f Data T IME TRANSACT I O N FO RWARD Oil-FOOT OBSERVED 08-SEO SEQ ERROR TYPES 1 4 03 1 7 9019 05 0 5 1 4 E 14Q341 9020 02 0 2 0 2 140350 902 1 02 0 2 0 2 140422 9022 05 0 5 1 4 E 140430 9023 02 0 2 0 2 140441 9024 02 0 2 0 2 140449 9025 02 3 5 0 5 F 140458 9026 02 0 2 0 2 1 40508 9027 02 0 2 0 2 140522 9028 02 0 2 -1 3 E 140541 9029 02 0 2 0 2 140613 9030 05 0 5 3 2 s 140653 9031 05 0 5 3 2 s 140708 9032 04 0 4 2 2 s 140739 9033 05 0 5 0 5 140751 9034 02 0 2 -3 5 E 140758 9035 02 0 2 0 2 14081 2 9036 02 0 2 0 2 140840 9037 05 0 5 0 5 140854 9038 02 0 2 0 2 1 4094 4 9039 05 0 5 0 5 141020 9040 05 0 5 0 5 141026 9041 02 0 2 0 2 141044 90 4 2 0 2 0 2 0 2 141116 90 4 3 05 0 5 1 4 E Note: For the explanation of each column h e adin g see next page B2


Description of Column Headings of Table 8-1 TIME : Time when the transaction was recorded. The time is read as two digits each hour, minute, and second; i.e. TRANSACTION: Transaction number sequence FORWARD: Registered Axles. T he number of axles allocated to a vehicle as determined by the toll collector. OB-FDOT: This number represented the difference between the axles number i n the OBSERVED column a nd the axles number in the FORWARD column OBSERVED: Axles observed in the CCTV view. T he n umb er of axles allocated to the vehicle as determined by viewing the CCTV images. This number represented the difference between the axles number in the OBSERVED col umn and the axles number in the FORWARD column SEO: ASAC axles. The number of axles allocated to the vehicle as determined by the A SAC. ERROR TYPES: The error type as in Table 2 B3


APPENDIX C Determination of Statistical Significance of Test Results C1


If p i s the probabil ity of a success in a lane transaction and q =(1-p) is the p robabi lity of a failure (deviat ion) in a lane transaction then the p robab ility of X successes in N transactions is given by ( X) [ N! ] x(l)v-x p -X!(N-X)! p p (1) Equation (1 ) is usually called the binomial distribution". Since p(X) is a probability rather than a deterministic measure, solution for p(X) d irect ly from equation (1) is imposs i b le. We must tum to statistical methods for the solution. If a very l arge number of transactions (N) were observed and th e number of successes (X) were recorded, then we would expect that the proportion P = X I N (the success rate) would converge to a single value which would be numerically equa! to the probability of success, p. The issue at hand is how small can N be and still produce a n acceptable approximation to the probability of success, p. An examp le will help i n understanding this concept. Suppose that a perfect coin with probability of a head-toss equal to the probability of a tail-toss (both equal to 1/2) is used to conduct a series of tosses. At the first toss (N = 1) either a head or tail appears. Arbitrarily calling a head a success, the P value is either 1 or 0 depending on whether a head appeared. C lear ly 0 or 1 is no t a satisfactory approximation to the known probability o f a head. At the second toss(N = 2) again either a head or a tail appears. For N =2 there are fou r pos sibl e sequences of appearances: head-h e a d, head-tail, tail-head and tail-tail. Notice that two of the four possible seque nces agree with the k now n probabi l ity. Continued tosses will result in a greater and greater number of sequences where P agrees with the known probability. Although there wil l continue to be a difference between the known probability and the P value, that difference will decrease as N becomes very large. Now repeat the same procedure with a biased coin where the probabi l ity of a head-toss is not equal to the probability of a tail-toss. As the nu mber of tosses grows large the P value will be a better and better approximation to the unknown biased probability. The question of how l arge N s houl d be remains unanswered. It is shown in mathematical sampling theory tha t for N samples selected from an infinite (or very large) population, the mean and standard deviation are given respect ively, by (2) and, l.2


cr N (3) Since the standard deviation is the uncertainty in the value of p calcula ted from the N samples, equation (3) shows t ha t the uncertainty will decrease a s N becomes large We will see that tha t the standard deviatio n can be use d to calculate a confidence interval for t he P value. Given the mean and standard deviation of a samp ling d istribut ion with N > 30, we can expect to f i nd a sample stat istic lying i n the in terva l between 1!, -cr, and 1!, +cr., 68.27% of the time. If a greater certainty is desired th e interva l must be i ncreased. T he amount of increase can be found for spec i fic val ue s of th e confidence interval. In a sample of size N drawn f rom a binomial population i n w hich th e p is the p robability of success, the con fidenc e li mits for pare given by Pz,cr.,. The values for Zc d e t erm i ne the confidence in terval and are shown in Table 1 for selected confidence leve ls. Using equation (3) we have the conf i dence limits for the pro port ion as Pz p ). N (4) In practice for N?:. 30, the value for P may be substituted f o r p. Tab l e C-1. Selected Values For Confidence Coeff i cients Confidence L eve l 99% 98% 95% 90% 68.27% Z< 2.580 2.330 1.960 1 .645 1 000 T he confidence level for the axl e counter can now be determined fro m equa tion (4) and Table C-1. Stat istica l Determination for ASAC Accuracy N (Sample Size) Errors X (Successful Trans.) P (S ucces sfu l Proportion) p(1-p) 30,535 370 30,165 0 9879 0.0120 o(1-ol!N sort. (o(1-ol!N) z for95% z*sort. = C P+C 3.91472E-07 0.0006 1.96 0.0012 0.9891 C3 P-C 0.9867