USF Libraries
USF Digital Collections

Evaluation workbook for community transportation coordinators and providers in Florida

MISSING IMAGE

Material Information

Title:
Evaluation workbook for community transportation coordinators and providers in Florida
Physical Description:
1 v. (various pagings) : ill. ; 29 cm.
Language:
English
Creator:
Florida -- Commission for the Transportation Disadvantaged
University of South Florida -- Center for Urban Transportation Research
Publisher:
Center for Urban Transportation Research, University of South Florida
Place of Publication:
Tampa, Fla
Publication Date:
Frequency:
annual

Subjects

Subjects / Keywords:
Local transit -- Evaluation -- Florida   ( lcsh )
Local transit -- Personnel management -- Florida   ( lcsh )
Transportation -- Evaluation -- Florida   ( lcsh )
Genre:
government publication (state, provincial, terriorial, dependent)   ( marcgt )
bibliography   ( marcgt )

Notes

Bibliography:
Includes bibliographical references.
Additional Physical Form:
Also available online.
Statement of Responsibility:
prepared for Florida Transportation Disadvantaged Commission ; prepared by Center for Urban Transportation Research, University of South Florida.
General Note:
"April 1994."

Record Information

Source Institution:
University of South Florida Library
Holding Location:
University of South Florida
Rights Management:
All applicable rights reserved by the source institution and holding location.
Resource Identifier:
aleph - 026220107
oclc - 85773747
usfldc doi - C01-00005
usfldc handle - c1.5
System ID:
SFS0032128:00001


This item is only available as the following downloads:


Full Text
xml version 1.0 encoding UTF-8 standalone no
record xmlns http:www.loc.govMARC21slim xmlns:xsi http:www.w3.org2001XMLSchema-instance xsi:schemaLocation http:www.loc.govstandardsmarcxmlschemaMARC21slim.xsd
leader 00000nas 2200000Ka 4500
controlfield tag 001 026220107
005 20110819174812.0
008 070306s1994 flua b s000 0 eng d
datafield ind1 8 ind2 024
subfield code a C01-00005
035
(OCoLC)85773747
040
FHM
c FHM
043
n-us-fl
090
HE4487.F6
b E93 1994
0 245
Evaluation workbook for community transportation coordinators and providers in Florida /
prepared for Florida Transportation Disadvantaged Commission ; prepared by Center for Urban Transportation Research, University of South Florida.
260
Tampa, Fla. :
Center for Urban Transportation Research, University of South Florida,
[1994]
300
1 v. (various pagings) :
ill. ;
29 cm.
500
"April 1994."
5 FTS
504
Includes bibliographical references.
530
Also available online.
FTS
650
Local transit
z Florida
x Evaluation.
Local transit
Florida
Personnel management.
Transportation
Florida
Evaluation.
1 710
Florida.
Commission for the Transportation Disadvantaged.
2
University of South Florida.
Center for Urban Transportation Research.
773
t Center for Urban Transportation Research Publications [USF].
4 856
u http://digital.lib.usf.edu/?c1.5
FTS
y USF ONLINE ACCESS
910
tp3043
951
10
SFU01:001759780;
FTS



PAGE 1

. -' ;;_ . '' ... .. EVALUATION WO -RKBOOK ;: .,. ,-. : for < Community Trilh:spot:fillion Coordinators and Providers in Florida Aprill994 ---. . I_ .. 7 / I .. Florida Transportation Commiss ion

PAGE 2

EVALUATION WORKBOOK for Community Transportation Coordinators and Providers in Florida Apr-il 1994 Prepared For F l orida Tran s por t atio n Disadvantaged Co m mis s ion P repare d B y Cen ter for Urban Trans port a tion Research University of South Fl ori da, Tampa

PAGE 3

Tra.nsportation Disadvantaged Commission 605 Suwannee Street. MS-49 Tallahassee, Florida 32399-0450 (904) 488-6036 Executive Director: Jo Ann Hutchinson Project Managers: Floyd G. Webb, ni Dene Dixon Center for Urban Transportation Research University of South Florida 4202 E. Fowler Avenue, ENB 118 Tampa, Florida 33620 (813) 974-3120 Director: Project Director: Project Manager: Project Staff: Gary L. Brosch F Ron Jones Rosemary G. Mathias R. Benjamin Gribbon Rebecca Rahirni Jennife r Hardin ACKNOWLEDGEMENTS CUTR and the TD Commission extend their appreciation to Hillsborough County Department of Social Services and Good Wheels Inc., for contributing their time to field rest this Evaluation Workbook

PAGE 4

Table ofContems , EVALUATION WORKBOOK for Community Transportation Coordinators and Providers in Florida TABLE OF CONTENTS PART I: INTRODUCTION Overview . . . . . . . . . . . . . . . . . . . . . . . . . . I -1 The Florida Coordinated Transportation System . . . . . . . . . . . . I I The Evaluation Model . . . . . . . . . . . . . . . . . . . . . I 2 Uses of the Evaluation Model . . . . . . . . . . . . . . . . . . I 2 Purpose of Conducting the Evaluation .' . . . . . . . . . . . . . . I 2 What the Evaluation Model Will Do . . . . . . . . . . . . . . . I 3 Evaluation Goals . . . . . . . . . . . . . . . . . . . . . . I 3 Appropriate Impetus . . . . . . . . . . . . . . . . . . . . I 3 Aspects to Evaluat e . . . . . . . . . . . . . . . . . . . I 4 The Reviewer . . . . . . . . . . . . . . . . . . . . . . . . I 4 The Evaluation Period . . . . . . . . . . . . . . . . . . . . . I 5 Timing of the Evaluation . . . . . . . . . . . . . . . . . . . . I 5 Other Evaluations . . . . . . . . . . . . . . . . . . . . . . I 6 How To Use This Workbook . . . . . . . . . . . . . . . . . I 7 The Module Concept . . . . . . . . . . . . . . . . . . . . . . I 7 Table J.J List of Modules and _uses . . . . . . . . . . . . . . . I 8 Parts of the Modules . . . . . . . . . . . . . . . . . . . . . I 9 How to Complete the Worksheets . . . . . . . . . . . . . . . . . I 9 Table /-2 Completing the Worksheets . ........................... I II Interpreting the Worksheets ....... ............................. I II Summary ................. .................................... I 12 I

PAGE 5

Table of Contents (Continued) PART II: GETTING STARTED Overview . . . . . . . . . . . . . . . . . . . . . . . . . II 1 Worksheet 1: Defining the Evaluation .............................. II 2 Completing the Work s h ee t . . . . . . . . . . . . . . . . . . . II -2 Worksheet I Defining the Evaluation .............................. II -4 Worksheet 2: Basic Information about the Community Transport atio n Coordinator . . . . . . . . . . . . . . . . . . . . . . . . II -5 Completing the Worksheet . . . . . . . . . . . . . . . . . . . . II -5 Worksheet 2 Community Transportation Coordinator .......... ........ II -6 Worksheet 3: Inventory of Local T ransportation Services ................. II 7 Completing the Worksheet . . . . . . . . . . . . . . . . . . . . II 7 Worksheet 3 Inventory of Local Transportation Services ................. II -9 Worksheet 4: Organizations to be Included In Eva luation ...... .......... II 10 Who Should be E valua ted? . . . . . . . . . . . . . . . . . . . II 10 Local CTC . . . . . . . . . . . . . . . . . . . . . . . II -I 0 CTC Peer Gro u p . . . . . . . . . . . . . . . . . . . . II 10 Single CTC Peer . . . . . . . . . . . . . . . . . . . II -11 Included T ransportation Providers . . . . . . . . . . . . . ll 11 Potential Transportation Operators . . . . . . . . . . . . . II -12 Standards . . . . . . . . . . . . . . . . . . . . . . . II 12 Completing the Work s heet . . . . . . . . . . . . . . . . . . . II 12 Worksheet 4-Agencies Included in Evaluation ... . .................. II 14 PART Ill: EVALUATION MODULES Overview ..................... ......................... ..... Worksheet S: Competition ........... ......... ................... Completing the Worksheet ...... ...... .. . . ................... Interpreting the Worksheet .... ............ ... ..... ...... ....... Worksheet 5 Level of Competition ........ .... . ............. ... Worksheet 6: Coordination .... ........... . . . .............. ... Wh Co d" ? at ts or m alio n . . . . . . . . . . . . . . . . . . . . F lorida Requirements ............ ........................... Lev e l of Coordination ...... . ........ ... ..... . ........... Elements of Coord in ation ....... ..... . ..... . ......... ........ Plan.ning .......... ............. ..... ..... ....... ...... Public Information ... . . ......... ................ . ..... Call Intake ........ . . . .......................... ..... Certification . . . . . . . . . . . . . . . . . . . . .. II III I III 2 III 2 Ill 2 III 3 III 6 III 6 Ill 6 Ill 6 Ill 7 III 7 III 8 III 8 III 9

PAGE 6

Table of Contents (Continued) Eligibility Re<:ords . . . . . . . . . . . . . . . . . . . . . Reservations . . . . . . . . . . . . . . . . . . . . . . . Trip Allocation .............. ...... . . ........... ....... Scheduling .............................................. Transport ..... ...... ... .. ............ .......... ....... Dispatching . . . . . . . . . . . . . . . . . . . . . . . General Service Moni toring .................................. Daily (RealT ime) Serv i ce Monitoring ........ .... ..... ..... ... T R 1 np econct tattoo . . . . . . . . . . . . . . . . . . . Billing .................... .............. .............. Reporting . . . . . . . . . . . . . . . . . . . . . . . . Other Coordination ... .... .... ........................ ..... Cost Sharing . . . . . . . . . . . . . . . . . . . . . . Inf o rmation Sharing .... ........ .... .... ..... ......... Completing the Worksheet ..................... ............ ... Interpreting the Workshee t ......... . . ............ ............ Worksheet 6 Level of Coordination .... ...... . . ........ ..... Worksheets 7 9: Cost Effectiveness and Efficiency ................... Evaluating Co s ts of the CTC and Operators .................. .. ... Completing the Worksheets . . . . . . . . . . . . . . . . . . Interpreting the Works h eets . ............. ....... ... ....... .... Worksheet 7 -Cost Effectiveness and Efficiency of the CTC System ..... Worksheet 8a Costs by Expense Category ...................... Worksheet 8b Detailed Cost s by Expense Category .............. Worksheet 8c Co s ts by Ca1egory ......... ............. . ... Worksheet 9Costs of Transportation Operators ........ . ... ... Worksheet 10 : Aspects of Quality ........ ...... ........ ....... . . OnTime Performance ..... ............ ...... ..... ......... Pick-Up Procedures ... . ................ .................. Trip Timeliness C a tegories ... .............................. Trip Length ................ ....... ................... Other Measures of Serv i ce Quality ........ ............. .......... System Reliability ..... . ..... .... ....... ............. ... .. Call Hold Time .... ..... ............... . ........... ... Public Infor m ation and Communications .... .... ........... ... Comp l aints and Compliments ..... . .... ........ ........... . Grievan.ces . . . . . . . . . . . . . . . . . . . . . . . Safety . . . . . . . . . . . . . . . . . . . . . . . . Vehicles .................... . ..... ...... .... ..... Prev e ntive Maintenance .......... ...... .... . ........... .. Admin i stra t ive Management ....... : ....................... ... .. Risk Management ............... .... .... .... ...... ..... Goal Setting and Plans tor I mprovement ...... ........... ...... Quality Management ............ ......... ..... ...... ... .... Ill Ill 9 m10 mro III -10 lll-11 lll -11 III-11 III 12 IU-12 III 12 I1I 13 III 13 lll-13 l1l 13 I1I 14 III IS Ill 16 III-19 Ill20 III 20 III-21 III 23 Ill 24 HI25 III 32 III 33 III 34 III 34 Ill 35 Ill 36 Ill 38 III 38 m-38 Ill 39 Ill 39 Ill 39 Ill 40 III 40 III -41 Ill 42 Ill 42 III 42 III 43 III-44

PAGE 7

Table of Contents (Continued) Special Qualities or Services ................ ............. ... Training .................. a Completing the Wo rksheets . . . . . . . . . . . . . . . . . . Interpreting the Worksheets ..... ... .......... ............... ... Worksheet lOa Evalua tion and Comparison of Aspects of Quality ..... Worksheet JOb -Calculation of Quality ........................ Worksheet JOeConclusions Regarding Aspects of Quality ........... lll44 lll-44 Ill 45 Ill. 46 Ill. 47 III 5 1 Ill 52 Worksheet 11: Training as a Measure of Quality ........ ............ Ill 54 The Value of Training . . . . . . . . . . . . . . . . . . . . Ill -54 Training Standards . . . . . . . . . . . . . . . . . . . . III 54 Means of T raining . . . . . . . . . . . . . . . . . . . . . III 55 Major Categories of Training . . . . . . . . . . . . . . . . . . III 55 New Em ployee . . . . . . . . . . . . . . . . . . . . III 55 On-the-Job . . . . . . . . . . . . . . . . . . . . . . III 55 Refresher . . . . . . . . . . . . . . . . . . . . . . III 55 Retraining . . . . . . . . . . . . . . . . . . . . . . . III 56 Skills Enhancement . . . . . . . . . . . . . . . . . . III 56 Types of Training . . . . . . . . . . . . . . . . . . . . III 56 Special Licensing . . . . . . . . . . . . . . . . . . . III 56 Ve hicle Operation . . . . . . . . . . . . . . . . . . . . III 57 Defe n sive Driving . . . . . . . . . . . . . . . . . . . . III 57 Emergency a nd Accident Procedures . . . . . . . . . . . . . . III 57 Passenger Relations . . . . . . . . . . . . . . . . . . . III 58 Passenger Assistance Techniques and Sensitivity . . . . . . . . . . III 58 Stress Management . . . . . . . . . . . . . . . . . . . . III 58 Substance Abuse Awarene ss . . . . . . . . . . . . . . . . III 58 Map Reading . . . . . . . . . . . . . . . . . . . . III -59 Radio Usage . . . . . . . . . . . . . . . . . . . . . . III -59 Scheduling/Dispatching . . . . . . . . . . . . . . . . . . III 59 ASE Certifications . . . . . . . . . . . . . . . . . . . . III 59 Completing the Workshe ets . . . . . . . . . . . . . . . . . . III 60. Interpreting the Wo rksheets . . . . . . . . . . . . . . . . . . III 61 Worksheet lla Evaluation and Comparison of Trai ning as a Measure of Quality . . . . . . . . . . . . . . . . . . III 62 Worksheet lib Conclusions Regarding Training as a Measure of Quality . . . . . . . . . . . . . . . . . . . III 64 Worksheet 12: Availability of Service . . . . . . . . . . . . . . . III 66 Measures of Availability . . . . . . . . . . . . . . . . . . . III 67 TD Population and Demand . . . . . . . . . . . . . . . . . III 67 Serv ice Provided . . . . . . . . . . . . . . . . . . . . . III 68 Reach of the Coordinated Program ....... . . . . . . . . . III 68 Public Information . . . . . . . . . . . . . . . . . . . . III -69 Access . . . . . . . . . . . . . . . . . . . . . . . Ill 69 Capacity . . . . . . . . . . . . . . . . . . . . . . . Ill -70 tV

PAGE 8

Table of Contents (Continued) Completing the Worksheets . . . . . . . . . . . . . . . . . Ill 70 Interpreting the Worksheets . . . . . . . . . . . . . . . . . . Ill 72 Worksheet 12a-Evaluation of AvailabilitY . . . . . . . . . . . Ill 73 Worksheet 12b Conclusions Regarding Availability . . . . . . . . Ill 75 Worksheets 13 and 14: Funding and Accountability . . . . . . . . . . III -76 Accountability of Operators . . . . . . . . . . . . . . . . . III -76 Completing tbe Worksheets . . . . . . . . . . . . . . . . . . III -77 Interpreting tbe Worksheets . . . . . . . . . . . . . . . . . III -77 Worksheet /3 Evaluation of Funding . . . . . . . . . . . III -78 Worksheet /4a Evaluation of Accountability . . . . . . . . . . . III -79 Worksheet 1 4b Accountability of Operators . . . . . . . . . . III 80 Worksheet 14c-Conclusions Regarding Accountability . . . . . . . li1 -81 Worksheet 15: Comparison of CTC to Peers . . . . . . . . . . . . . III -82 Completing the Worksheets . . . . . . . . . . . . . . . . . . Ill -82 Interpreting the Worksheets . . . . . . . . . . . . . . . . . . III -83 Worksheet I 5a Comparison of CTC' to Peers by Structure . . . . . . . Ill -85 Worksheet 1 5b Conclusions Regarding Comparison of CTC to Peers . . . III -86 PART IV: CONCLUSIONS Overview . . . . . . . . . . . . . . . . . . . . . . . . . IV I Worksheet 16: General Findings . . . . . . . . . . . . . . . . IV -4 Completing the Worksheet . . . . . . . . . . . . . . . . . . . IV -4 Worksheet 16-General Findings . . . . . . . . . . . . . . . . IV5 Worksheet 17: Comparative Concl u sions . . . . . . . . . . . . . IV -13 Completing the Worksheet . . . . . . . . . . . . . . . . . . . IV -13 Worksheet 17-Compa rative Conclusions . . . . . . . . . . . IV -14 Worksheet 18: Summary of Evaluation . . . . . . . . . . . . . . IV 19 Completing the Worksheet . . . . . . . . . . . . . . . . . . . IV19 Worksheet 18Summary of Evaluation . . . . . . . . . . . . . IV -20 APPENDICES A Resource List . . . . . . . . . . . . . . . . . . . . . . . A-I B List or Abbreviations . . . . . . . . . . . . . . . . . . . B-1 C Glossary of Terms . . . . . . . . . . . . . . . . . . . . C-I D CTC Peer Group Data .............. ....... ................. D-1 v

PAGE 9

PART 1: INTRODUCTION

PAGE 10

Part 1: Introduction INTRODUCTION Overview This workbook, prepared by the Center for Urban Transportation Research (CUTR) for the Florida Transportation Disadvantaged (TD) Commission, presents a framework for evaluating designated commllllity transportation coordinators and transportation operators. The workbook may be used by local coordinating boards or commllllity transportation coordinators. Part I describes the Florida Coordinated Transportation System and presents the evaluation model. Part II leads an evaluator through the steps necessary to begin an evaluation. Part lii is divided into several modules, each examining a diffe.rent aspect of local coordinated TD transportation. Part IV contains modules designed to draw evaluation conclusions based on the individual modules completed The appendices contain a resource list, list of abbreviations, and a glossary of terms. All references made in the text are listed in Appendix A. I t is highly recommended that all of Parts I and II be read before any of the modules are used The Florida Coordinated Transportation System The TD Commission oversees a coordinated system of many local TD transportation service providers in the state. At the loca l level, commllllity transportation coordinators (also referred to as coordinators or CTCs ) are responsible for the provision of service. The service area for which the CTC is responsible is, at a minimwn, one county, but can include more than one county. The coordinator can be a transportation operator and actually provide TO transportation service or it can form a network of providers by brokering all or some of the service to other transportation operators (also referred to as transportation providers). All entities that expend federal, state, or local government funds to transport persons who are transportation disad v antaged are mandated by Chapter 427 of the Florida Statutes to contract with the local CTC for TO transportation services. The Florida Coordinated Transportation System (FCTS) is fully described 1 A list of abbreviations is included in Appendix B. I I

PAGE 11

Parr I: Introduction in the "Florida Five-Year Transportation Disadvantaged Plan." The Statute (427 F.S.), and rule (Rule Chapter 41) that implements it, outline the duties and responsibilities of the CTC Technical Memoranda No.3 and No.4 of the "Florida Five-Year Transportation Disadvantaged Plan" also eStablishes specific goals, objectives, and measures for CTCs. Each CTC contracts annually with the TO Commission and is advised by a local coordinating board (LCB). By law and by rule the LCB evaluates the performance of the CTC, approves the CTC s annual service plan, which includes an evaluation clement, and provides recommendations to the TD Commission regarding the renewal of the CTC's contract with the TO Commission. This evaluation workbook was created to provide a formal proces s for evaluating the performance of the CTC (and its opera to rs). In addition, ongoing concern exists throughout F lo rida about the use of transportation operators or, more specifically, how transportation providers are selected for incl usi on in each coordinated s ystem and how trips are allocated to each. This evaluation workbook provides a means for CTCs to evaluate transportation opera tors that are already included in the local coordinated system and those being considered for inclusion. The Evaluation Model Uses of the Evaluation Model The evaluation model i s to be used to evaluate CTCs and/or potential or existin g transportation operators. It may be used either by the CTC; the LCB, and/or the TO C01runission. Purpose of Conducting th e Evaluation Although th e catalyst for conducting the eva luation and its specific application may vary, the underlying purpose o f cond ucting an evaluation with this model is to ensure that the most cost effective, efficient, unfragmented, unduplicated, appropriate, quality and accountable transportation service is being planned and provided for the TO popu l at i on. The intent of this model is to determine h ow well the CTC (and operators) are doing and whether the costs (or rates) are reasonable. I 2

PAGE 12

Part I: Introduction What the Evaluation Model Will Do The evaluation model requires participation and thought from the reviewer(s) and, therefore, has the potential to accomplish many things. The model can be used to evaluate the base service and compare that service to a number of service options. The evaluation model is designed to: (a) help evaluate fully allocated costs; (b) compare the rates of operators; (c) identify what qualities and advantages are included in costs; (d) determine how well the CTC is doing according to prescribed goals and objectives ; (e) identify areas to be targeted for improvement; and (f) show costs in proper perspective by looking at quality and other factors in order to determine reasonableness. Evaluation Goals Each evaluator's goals and objectives for conducting an evaluation will vary depending on: (a) the motivation for the evaluation; (b) the organization and network type of the local CTC; (c) the number of transportation operators; (d) any constraints p l aced on the coordinator; (e) the local environment; (f) the number of service options available; (g) local priorities and concerns; and (h) the scope or elements and level of detail the CTC, LCB, or TD Commission wishes to include. Appropriate Impetus Many situations can trigger the use of this evaluation model. For example, the evaluation is extremely well suited for use as part of an annual evaluation of community transportation coordinators by their local coordinating boards. The CTC, LCB, or the TD Commission may suggest that the evaluation workbook be referenced in the CTC Service Plan as the criteria for evaluation. The TD Commission may initiate or require an evaluation, using this model, of any I 3

PAGE 13

Part 1: Introduction particular CTC, whether selected rando mly or based on criteria for spot checks. The CTC may choose to use the evaluation workbook to conduct a complete self-evaluation or to look at a particular aspect of its service using one or more modules. The evaluation also may be applied as requested by the local board or at the initiative of the CTC to evaluate the transportation operators for the purposes of renewing contracts or allocating trips, or to consider additional operators. The CTC may apply the model at regular intervals (e.g., annually), as needed or over time (e.g., one module every month). Aspects to Evaluate The evaluation model may be used to evaluate any or a ll of the following major aspects of coordinated transportation: (a) competition; (b) coordination ; (c) cost-effectiveness and efficiency; (d) quality of service; (e) training as a measure of quality; (f) availability of service; (g) funding and accountability; (h) comparison of CTC to peers; and (i) conclusions regarding th e CTC and its operators overall. A separate module and set of worksheets is provided for each of these major evaluation areas. All of these aspects should be included if an LCB is conducting an annual or comprehensive evaluation. The Reviewer The reviewer also referred t o as th e evaluator, is the person or team designated to lead or conduct th e evaluation. Who should conduct the evaluation will vary according to who or what organization initiated the evaluation and the goal of the evaluation. If, for example, the TD Commission or a local coordinating board wishes to conduct a formal evaluation of a CTC for the purpose of making a determination regarding renewal of the CTC' s contract with the TD Commission, it would be appropriate to designate a party outside of the CTC to lead the I -4

PAGE 14

Part /: Introduction evaluation to reduce the perception of bias The TO Corrunission or LCB might ask the staff of the designated official planning agency (DOPA) to cortduct the evaluation. If, for example, the CTC has simply decided to take a look at how training, as a measure of quality, varies between itself and other transportation operators the manager of the CTC may informally request a supervisor to apply the workbook using the training quality module. Those conducting the evaluation may be a team selected from one or several organizations, led by a lead reviewer. In any case, the reviewer(s) always will need to work closely with CTC staff. If a large team is conducting a comprehensive evaluation, the reviewers may form smaller teams responsible for different modules. The Evaluation Period At the outset of the evaluation, the reporting period (the period from which to collect and count information) must be determined. The worksheets require some of the same information reported in the annual operating reports (AOR) required and collected from CTCs by the TO Corrunission. The same reporting period (i.e. July through June), therefore, is easiest to use for evaluation, if the timing of the eva luatio n permits. Bear in mind that longer reporting periods provide more stable infonnation than shorter periods. In many cases, the entity that required the evaluation also will establish the reporting period that will be used. Some worksh eets examine trend information, requiring data from both the current and comparable previous reporting periods. Note that to conduct peer comparisons, the reporting periods m ust be selected carefully. A specific peer may have a different reporting period. Peer group information relies on the Statewide Operations Report (SOR), which may not be available as early as information for the local coordinator Timing of the Evaluation A complete evaluation of this type should normally be conducted at least annually. Areas of special concern might be examined on an as-needed basis. One of the most important factors influencing when the evaluation is conducted is the availability of data. One-year periods are usually the most useful to examine, so it will be most appropriate to conduct an evaluation immediately following the preparation of annual reports. The evaluation relies on mu ch of the same data required and c o llected by the TD Commission in the AOR, so it may be easiest to begin the evaluation as soon as this information is available. Peer comparisons, however, would require waiting until peer information is available fr om a similar system or until the SOR is available. Conductin g an evaluation as soon as AOR information is prepared can allow the I -5

PAGE 15

Parr 1: Introduction evaluation results to help shape a new service plan and help with the memorandum of agreement (MOA) renewal process. Although the model is flexible as to when evaluations are conducted, timing is influenced by several factors. Among the most important of these is how long the evaluation will take to complete, which is in fluenced by the following factors: number of aspects evaluated; time commitment available from reviewers; number of organizations in v olved; availability of information; and deadlines. A complete evaluation can be conducted in a week if a review team of about half a dozen people can devote full-time attention to the review, if all needed infonnation is available, and if few outside organizations are included in the review. Writing the conclusions may take some additional time. Few systems, howev er, actually will be able to devote their full attention to the evaluation or will have all information readily ava ilable. If the evaluation is to be conducted over a long period of time, complete the modules one at a time. A coordinator who is conducting occasional self-evaluations might wish to complete one module every month or so. Other Evaluations This evaluation model does not preclude the usefulness of other evaluations and performance reviews. Other types of evaluations include: o compliance audits; o management performance audits; financial audits; and special purpose evaluations. The evaluation model is tailored to the objectives of the Florida coordinated system (although the modules are usable by others as well). This model includes assessments of coordination, quality, availability, funding, and accountability in relation to cost, and looks at a number of general performance indicators; however, the evaluation model is not intended as a substitute for all types I 6

PAGE 16

Part I: Introduction of evaluations. A management infonnation system (MIS) is the mechanism that collects and reports key operating and financial inforniation foi' lnilhagers on a continuing and regular basis. A compliance audit checks t o see that the organization complies with all federal, state, and local Jaws. A management performance audit is a comprehensive evalua tion of the management planning and control processes within eac h of the system's functional areas.' Efforts that attempt to ensure that record keeping is accurate, that accounts balance, an d that specific expenditures are appropriate fall under the scope of a financial audit, which is frequently conducted by a third party. Several publications that discuss other evaluations are listed in Appendix A of this workbook. How to Use This Workbook This workbook contains a complete evaluation model. The remai nder of the workbook is divided into 16 modules, each designed to address a particular issue or aspect of the local coordinated system. Part II of the workbook presents four modules that help the evaluator focus and begin the evaluation. Part III consists of nine modules, designed to evaluate transportation competition coordination costs quality training, availability, and funding and accountability. Part IV concludes with three mo du les that look at the local system overall and assists the reviewer in drawing conclusion s based o n the other modules. The Module Concept This evaluation workbook has been designed to satisfy a number of evaluation purposes. The division into modules con tr ibu tes to the flexibility of the evaluation workbook; evaluators can use only those modules th at apply to the specific purpose of their evaluation. For example, some modules are only appropriate tor evaluating the CTC, while others may be used to evaluate both the CTC and transportation operators. If a complete evaluation is being conducted, most -if not all-modules will be used; however, the modules are designed to allow the evaluation to be conducted in steps. The module concept also allows the model to be used to examine a single aspect of service. The modules are arranged in the order most appropriate for conducting a 2 Institute for Urb:.m Tnmsportatio n "Handbook for Management Performance Audits," DOT-T-88-21 (Bloomington, Indiana: Urban Mass Transportation Administration and U.S. Oepanment ofTransportation. J 988). 1. I 7

PAGE 17

Part I: Introduction complete evaluation. The difterent modules to 1?e used for evaluation of a CTC and for evaluation of transportation operators are shown in Table I-I. As shown, worksheets on coordination, availability, funding, and peer comparisons are not used for reviewing transportation operators. TABLE 1-1 List of Modules and Uses Evaluation Use Modules Worksheet(s) Page Transportation No. CTC Operators I'; .'.'1:' '.0' : .. s 'tai1ed ' :'' > .. "fl':' ; ... . . . ".: : "':/ V4 ___ ___ "" .' ...;1:. b* !. ,,,.,);, '1 H*S\ : Defining the Evaluation 1 11-2 , Basic Informat io n about the CTC 2 11-5 , Inventory of Loca l Transportation Services 3 11-7 , Organizations to be Included i n the Evaluation 4 11-10 , III: EY'a)!'atlon Modules . ... ,!:. '\' '" ', > ; : .,(.,, ....... Competition in Use of Opera t ors 5 111-2 , Evaluation .of Coordination 6 111.0 Cost-Effectiveness and Efficiency 7-9 111-19 , Aspects of Quality 10 111-34 , Training as a Measure of Qua l i ty 11 111-54 , Availability of Service 12 111.06 Funding 13 111-76 Accountability 14 111-76 , Comparison of CTC to Peers 15 111.02 . . ; c -' Part IV: Conclusions .. ,, 'h." .... .. ,;, -,, General Findings 16 IV-4 , Comparative Conc l usions 17 IV-13 , Summary of Evaluation 18 IV-19 , I 8

PAGE 18

Part 1: Introduction Worksheets 1-4 always should be completed in an evaluation. The conclusion worksheets ( 16-18) require that some combination of modules from Part III be completed. Some worksheets refer to other modules that require similar information. Parts of the Modules Each module varies in length and consists of discussion, worksheet instructions, and worksheet(s). The discussion part of each module presents key concepts to be considered in the evaluation of certain aspects of service. For aspects that are more difficult to quantify, the modules contain significantly more discussion. The instructions discuss preparations for complet ing the worksheets and provide clarification for some worksheet questions. The instruc tion s also provide some assistance in interpreting the completed worksheets. The worksheets themselves lead the reviewer through a series of questions about the ere, coordinated service, and/or transportation providers. Some modules contain multiple worksheets. How to Complete the Worksheets Reading the discussion text in each module before completing the worksheets will provide greater understanding of the aspects of service that the worksheets examine. All instructions should be read thoroughly befo r e attempting to complete each worksheet. The worksheets will be more useful if they are tilled out completely and accurately. The discussion section at the beginning of each module does not need to be read every time the evaluation is conducted if the concepts are clear to the reviewers. However, the instructions always should be referenced. Return to the discussion if the instructions and worksheets are unclear if the reviewers have difficulty drawing conclusions, or when the evaluation reveals a lack of information or areas that need improvement. The discussion may provide new information or approaches for problem areas. The information needed to complete the worksheets may come from several sources. The AOR, prepared by each ere for the TD Commission, is one of these sources. Depending on the record keeping practices of the ere, much information may come from other regular reports. The reviewers may need to use existing information to calculate the answer required for the worksheet. Some special reports may need to be generated and, i n some cases, new information may need to be collected. For example, the quality module may require that a physical count of I -9

PAGE 19

Part /: Introduction vehicles and their charact e ristics be conducted. In some cases, accounting departments might need some time to compile information. Information from third parties will require some phone calls or correspondence, so be sure to determine at the beginning of the process what information will be required from those outside of the organization. If information is unavailable a portion of the worksheet may be left blank. Do not become frustrated if your data do not perfe ctly fit the format of the question. First, try to collect or adapt information to fit the question exactly. If a question is not applicable, or not enough room is given to provide a use f ul answer, mark the worksheet space with an asterisk () or number in a circle, and include a footnote at the bottom or end of the worksheet with whatever information may be necessary to draw conclusions about that aspect of service. Feel free to write in the margins of the worksheet but only include information necessary to evaluate the particular aspects of service being examined. These are not reporting forms and need not detail every related piece of information about a coordinator or operator. Enough discussion and explanation should be provided to interpret each worksh e et question; however, because operations vary, slight differences in how each question is answered may arise. Refer to the instruction s discus s ion text, or glossary to help clarify each question, and recall frequently the specific intent for conducting the evaluation Use your best judgment. Most importantly, be sure to apply each question the same way for each time period, or the same way to each operator evaluated. The thorough understanding of each aspect of the organization evaluated that the reviewer will gain in collecting and evaluating information is more important than the response in e ac h box on the worksheet. This understanding will help the reviewer to complete the worksheet page s that require the judgment of the reviewer in drawing conclusions about particular aspects o f s e r vice The entire worksheet should be completed by one reviewer; however, the input of others may be sought on the worksheet pages that draw conclusions. Each worksheet should be marked with the reporting period the preparer's initia ls, and the date prepared. Remember to photocopy the worksheets before writin g on them Key points for completing worksheets are summarized in Table 1-2. I -10

PAGE 20

TABLE 1-2 Completing the. Worksheets Part/: lmroduction Refer to the discussion text at the beginning of each module to clarify important concepts. Refer to the i nstruct ions before completing each worksheet. Some worksheet questions will be simpler tha n others to answer. Some questions simply require numbers to be i nserted some require consideration, and j udgment and others will require research and calculations. How easily eaclh question can be answered depends on the availability of information. The evaluation worksheets will be usef u l even if some quest i ons have to be left blank. but will be much more useful if r esearc h is i nvested as needed to answer all questions. The forms are worksheets, so use the margins to write in clarifications Keep the goals of the evaluation in mind at all times Before drawing conclusio n s . study the completed worksheets. Copy additional workshee t s as needed. Interpreting the Worksheets The evaluation worksheets are designed so that the reviewers should gain significant insight into the organization included as information is collected, worked with, and recorded on the worksheets. Before the reviewer gets t o the concluding questions on the worksheet a number of discoveries will be made. The last questiops on the worksheet, included as a separate page on longer worksheets, prompt the reviewer to look back and analyze the worksheet results in different ways. The interpretation usually includes comparing organizations in specific areas. Some of the interpretation questions are quantitative, but most require the judgment of the reviewers, based on the specific information summarized by the worksheet. The interpretation questions are designed to identify strengths and weaknesses i n each organization and identify areas that, in either ca se, could be impr oved The judgment of the reviewers should be informed by the discussion text which each module begins with, preceding the worksheets. Each module in Part Ill includes a brief section, titled "Interpreting the Worksheet." Information from the interpreted worksheets also is used in the conclusion of the evaluation, in a worksheet that summarizes each module in order to compare organizations across several l II

PAGE 21

Part 1: Introduction evaluated aspects, such as cost and quality. Some conclusion information will therefore, need to be copied onto the last worksheet. Summary Although this evaluation model is designed to be flexible, there are typical uses for which it is especially well suited. In Florida, each coordinator's annual serv i ce plan is required to have a CTC evaluation element. This evaluation model may be used to help meet that requirement, in which case, all modules should be completed, including the conclusions section. In this scenario, the review should be led and concluded by one or more designees of the local coordinating board, with assistance from CTC staff. The evaluation would need to be completed before the service plan is developed, before the LCB's annual review of the CTC is completed, and before the MOA is submitted to the TO Commission for renewal. The LCB would need to be responsible for drawing conclusions and establishing measurab le criteria to be met in the following year. The CTC should actively be invo lved in iden tifying, for their own information, areas that can be improved Many coordinators may use the evaluation workbook for a full or partial self-evaluation. In these evaluations o nly the modules the coordinator is specifically interested in need to be used; however, all are appropria te. T he only exception is that a review of quality should i n clude both the "Aspects of Quality" and "Train ing as a Measure of Quality" modules The "Conclusions" module should be used if multiple modules are completed as part of one evaluation process because they consider several areas at once. The timing is less important in the self-evaluation scenario, except for two impor tant c ons iderations. First, many worksheets use AOR data, so evaluations conducted afte r AOR data is compiled will be easiest and most timely. Second in order to consider the re l ationsh i p between two areas (e.g cost and quality), the modules need to be completed at about the same time. Thi s will allow the reviewers to consider both areas when draw in g conclusions. I 12

PAGE 22

PART II: GETTING STARTED Worksheets 1-4

PAGE 23

Pan II: Getting Started GETTING STARTED Overview This section of the workbook assists the reviewer in setting up the evaluation. As such, it should be completed at the start of every evaluation. The worksheets require the reviewers to make some fundamental decisions about the evaluation process. Therefore, all reviewers should be familiar with the sections titled, "The Evaluation Model" and "How to Use this Workbook," in Part I. The evaluation model does not rely entirely on existing data. The evaluation model does, however, use a lot of data already collected for other purposes. The following reports and information for the appropriate periods (usually the most recent and previous period) should be gathered at the beginning of the evaluation, as they will probably be needed for more than one module (depending on which aspects are evaluated) 0 TO Annual Operating Report (AOR) 0 Other operating reports 0 TO Statewide Operations Report (SOR) 0 Annual Memorandum of Agreement (MOA)/Service Plan 0 Coordinated Transportation Development Plan (CTOP) 0 Monthly reports and/or MIS reports 0 System Safety Program Plan (SSPP) 0 Existing inventory of local transportation services 0 Previous evaluations 0 Any related reports What other informati o n will be required and how it will be collected depends on the evaluation conducted and how the coordinator collects and reports information. If information will be included from other organizations, review ihe worksheets well in advance so the organizations can be told what information may be required. II -I

PAGE 24

Part II: Getting Starred Worksheet I Defining the Evaluation Why an evaluation is being undertaken, which entities are being evaluated, who will conduct the evaluation, and what is expected to result from the evaluation should be defined clearly as the first step of the evaluation Otherwise, the reviewers may make some false assumptions about the purpose and scope of the evaluation. Worksheet 1, shown on page II -4, helps to focus the evaluation and should be completed even if only one module is being used in the e valuat ion. Completing the Worksheet Worksheet 1 s hould be completed carefully, with the assistance of those requiring and those leading the evaluation. The worksheet determines how the evaluation will be conducted and may prompt important decisions. First, record the current date, the county or counties for which the ere is responsible, and the name of the coordinator Next, record the name of the person or organization that requested or required that the evaluation be conducted. This often will be the coordinator itself (or a ere manager), or the LCB. T his question, a l ong with a question below about what triggered the evaluation, helps to establish why an evaluation is being conducted. Check all the types of entities that will be evaluated. In most cases this process will evaluate the coordinator, and often it wil.l evaluate "incl uded" transportation operators. The model also could be used to consider potential transportation operators (although not in place of formal bid or proposal evaluations). Occasionally an evaluator may wish to include a peer CTC or a CTC standard for comparative purposes. Check which level of effort will be expected in conducting the evaluation, as this will affect what s te ps to take and not take and how long the evaluation will take. This also is valuable information for reporting results. The quickest (but least useful) evaluation would be one based solely on the information the coordinator already has available Other information could be computed from the information available. Allowing the reviewers to research and uncover all the information reques t ed on the worksheets will provide the most valuable evaluation results. 11-2

PAGE 25

Part II: Getting Started Check which aspects of the service or organizations will be evaluated. Each of the areas corresponds to a module in this workbook.. The for "other" recognizes that the evaluation might be customized, possibly incorporating some other element or future modules. See "Aspects to Evaluate" and "The Module Concept" in Part 1 for more information. Record the specific periods (month, day, year) that the evaluation will cover. The data used must reflect these dates. The previous period must be appropriately comparable to the recent period; for example if the recent period is one year then the previous period also should be one year. Refer to "The Evaluation Period" in Part I for more information. Briefly describe the event or situation that triggered the evaluation, which may be a regular or unusual occurrence. For example, the impetus may be "required annual evaluation," or "increased complaints related to quality." Write exactly what the evaluation is expected to accomplish. Describe whether it should result in a report, who the results should go to, and what kind of information the reviewers desire (e.g., whether they are looking for recommendations for improvements or an assessment of the past year's performance). Describe whether any particular decisions should result from the evaluation process. Read "The Evaluation Model" in Part I for more information. Indicate the deadline for completing the evaluation. When this information is completed, the individual or a representative of the organization that requested or required the evaluation should s ign this worksheet, indicating that the evaluation to be co nducted matches his/her expectations II 3

PAGE 26

WORKSHEET I DEFINING THE EVALUATION See discussion and instructions beginning on page II 2. Date: I I County(ies): mo day yr Community Transportation Coordinator. This evaluation is being conducted at the request or direction ot. The following person(s) and organization(s) will be responsible tor conducting the evaluation: Entlty(ies) to be eva l uated: (.1 all that apply) 0 Community transportation coordinator 0 Coordinated transportation operator(s) 0 Transportation provider(s) not coordinated 0 Other (specify) Research level expected : (.1 all that apply) 0 Existing available data 0 Information that can be computed 0 All researc h as needed Reporting period to be examined: __ mo day yr mo day yr Event or situation that t ri ggered the evaluation: Area(s) to be evaluated : (.1 all that apply) 0 Competition 0 Level of coord ina tion 0 Cost effectiveness and efficiency 0 Quality 0 Training 0 Availability 0 Funding and accountability 0 Comparison to peers 0 Other: (specify) Previous period: to mo day yr mo day yr Refined goal of the eval u at i on and what s hould be accomplished: Expected completion date: I I mo day yr Authorized s i gnature repre sent in g entity requesting the evaluation: Reporting Period: _______ Preparefs Initia ls: ______ Date : ____ Il-4

PAGE 27

Part II: Getting Started Worksheet2 Basic Information about the Community Transportation Coordinator AI:ty use of this evaluation workbook requires that certain basic information about the coordinator be documented. Certain worksheets and modules draw upon this basic information in forming conclusions. Furthermore, fundamental i nformation may be required by those less familiar with the CTC or those reviewing the evaluation at a later date. The basic information required is contact information operating environment, the organization and network types of the coordinator, and si.ze. Definitions are the same as those used for completing the Annual Operating Report (AOR) and are included in the glossary (see Appendix C). Worksheet 2 should be used to document this basic information . Completing the Worksheet Using Worksheet 2, fill in the name, address and phone numbers of the CTC, and the name and title of the manager or director of the CTC. The contact is the person that should be called if someone outside of the evaluation process has questions about the evaluation in the future, or the name of the person responsible for conducting the evaluation. Describe how many years (or months if less than a year) the organization has been the CTC. This information is noted only to help understand the evaluation results. T he local operating environment, organization, network and size information is used partly to understand the system and partly to establish peer groups for comparisons. This information can be taken from the AOR or SOR. The environment is "urban" if the coordinator's service area includes a population center of 50,000 or more, "rural" if it does not. Indicate whether tb.ere is a fixed-route transit system that receives Section 9 funding from the Federal Transit Adm inistration operating in the service area Also, describe any special characteristics of the service area that may have a major impact on the coordinator. Examples include major bodies of water that may result in exceptionally long trips, a mix of urban and rural areas, a recent b.urricane or other disaster or a major change in public policy. This information simply helps others interpret the evaluation. Indicate one organization and one network type, referring to the glossary or AOR in s tructions if clarification is required. The size of the system will be measured by the number of tr i ps coordinated in one year. The total reported in the AOR or SOR is appropriate in this case. I! 5

PAGE 28

WORKSHEET2 COMMUNITY TRANSPORTATION COORDINATOR See discussion and instructions beginning on page II -5. 1. Contact Info rma t io n: NameofCTC: County(ies) served: Address: Phone : Fax: Director/Manage r : Contact for this evaluation: 2 Local coordinator since: (date), ______________________ 3. Operating Environment: (see instructions. page II S) (./ one) 0 Urban (inc ludes a population center of 50,000+) 0 Rural 4. Organization Type: (./ one) 0 Private for-Profit 0 Private non -Profit 0 Governme nt 0 Transportation Agency 5. Networll Type : (./ one) 0 Sole Prov ider 0 Partial Brokerage 0 Complete Brokerage 6. Size: Annual One-Way Passenger Trips. ___ ________________ (indicate exact number reported and circle category) 1 = 09,999 Annual Tfl:ls 2 = 10,000-49,999 :t= 50 ,000.99,999 4 100 ,000. 199 999 5 = 200,000. 749,999 6 750,000 1 ,499, 999 7 = t ,500,000 and up 7. Briel description of any unique c ha racte ristics of the service area and/or significant events that may have occurred during the evaluation period: Reporting Period: _____ __ Prepa r er's Initials : ______ Date: ____ 11-6

PAGE 29

Part II: Getting Started Worksheet3 Inventory of Local Transportation Services Before evaluating certain aspects of coordinated TD transportation, the evaluation team should compile or obtain an inventory of all transportation services available in the service area including all providers, especially those not coordinated. This information is necessary for evaluating the structure of the CTC, evaluating the extent of or potential for competition, and determining which transportation providers should be included in an evaluation of operators. Given the responsibilities of a coordinator, it is likel y that such an inventory already exists A new inventory need not be compiled if the existing inventory is recent and accurate. Some investigation may be necessary to confirm or update the inventory. Otherwise a new inventory should be created. The inventory should include every organization in the service area that: (a) receives public funds and transports individuals in connection with those funds; and/or (b) provides transportation services to the general public (including transportation arranged pri va tely). The inventory will, therefore include social service agencies and taxi companies among other organizations The inventory should include contact information, an indication of the size of the transportation operation organization type (public, private, for -p rofit, non-profit), and criteria for passenger eligibility. Additional information may be sought now or as needed since such inventories can be valuable to coordinators in many ways. Worksheet 3 should be used to document the completion of an inventory of local transportation services. Completing the Worksheet Using Worksheet 3, shown on page II 9, record the date that the most recent inventory was conducted. Indicate whether the inventory is documented; that is, whether a hard copy with complete information was produced. If no inventory exists or several different inventories exist indicate such in the space provided. Indicate what sources were used to identify transportation providers in the inventory. As many sources as needed (see l i s t on Worksheet 3) should be checked. Indicate what types of systems the inventory includes; ideally the inventory shoul d include all operators in all the categories listed. Total the number of transportation providers in the inventory to indicate the total number of transportation providers in the service area. Looking at the Jist of sources consulted and t he information and organizations included indicate whether, 11-7

PAGE 30

Part II: Getting Started in the judgment of the lead reviewer a new inventory needs to be conducted or if the inventory needs to be updated or supplemented with additional research. If the inventory needs to be updated, that research should be begun immediately or prior to evaluating the use of competition and operators. If an inventory is not used, reviewers may have difficulty completing the competition and coordination modules. II-8

PAGE 31

WORKSHEET3 INVENTORY OF LOCAL TRANSPORTATION SERVICES See discussion snd instructions beginning on page II 7. 1. Date of last inven t ory: I mo day yr 2. Is this inventory documented i n writing? 0 Yes 0 No 3. The follow i ng sources were consulted to complete this inventory: (./ all that apply) 0 Directory( i es) of public services 0 Te l ephone directory(ies) 0 Business-to-business directory(ies) 0 Referra l (s) from other transportation provider(s) 0 Local social service agency(ies) 0 Prev i ous inventory of local services 0 List from local public transportation commission or authority 0 Local business/occupational license department 0 Request(s) for i nclus i on on i nventory or as an operator 0 CTC planning documents (e.g., CTDP) 0 Other (specify) 4. The inventory includes: (./ all that apply) 0 Community transportation coord i nator 0 Current contract operato r (s) 0 Not cont r acted: 0 Soc i al serv i ce agency(ies) 0 Chu r ches 0 Volunteer transportation provider(s) 0 For-hire taxi and/or van company(ies) 0 Public transit system 0 School bus provider(s) 0 Other : (specify) 5. Within the CTC s service a rea, there are a total of organization(s) that provide(s) any transportation service(s) to all or some of the general public 6. Does a new inventory need to be compiled? 0 Yes 0 No 7. Does the inventory n eed to be supplemented with additional information? 0 Yes 0 No 8. When w i ll the new i nforma t ion be available? I I mo day yr Reporting Period: ______ Preparer's Initials: ______ Date: ____ 11-9

PAGE 32

Part II: Getting Started Worksheet4 Organizations to be Included in Evaluation Who Should Be Evaluated? This evaluation workbook is suitable for a variety of purposes, and is flexible regarding which organizations are included in an evaluation. Another advantage of this evaluation model is its use of comparisons. Comparisons allow evaluators and planners to draw realistic conclusio ns, which tend to be more useful information locally than assigning a performance ranking, and comparisons allow reviewers to draw conclus ions without waiting for trend statistics. This aspect also alloWs the same model to be used for evaluating different organizations. Parts of this workbook can be used to compare and evaluate the CTC, its peers, included transportation operators, and transportation operators being considered for inclusion in the coordinated system. Local CTC This workbook is especially designed to evaluate a CTC. All of the discussion sections and worksheets assume that the reviewer is, first, looking at a particular CTC and, second looking at its peers or other transportation operators. Although this evaluation also may be used to assess onl y transportation operators, the CTC will need to be in c luded at some level. CTC Peer Group Comparing an organization such as a CTC to similar organizations is a common and useful practice. The set of similar CTCs is called a peer group, and because CTCs can be similar in a variety of ways, each CTC has several potential peer groups. CTCs can be grouped according to: (a) the environment (u rban or rural); (b) the organization type (private no!l-profit, private for-profit, transit agency, or government); (c) the network type (sol e-provider, partial brokerage, or complete brokerage); II10

PAGE 33

Part If: Getting Started (d) availability of fi>eed-route transit service (yes or no); and (e) system size (based on ridership budget or number of employees or vehicles). Any other similarity also could fonn a peer group, but those listed above are deemed to be the most useful for this model. The use of peer groups ensures that a CTC is being compared in a similar set of circumstances, increasing the useful ness and validity of a comparison. Comparing the cost per mile of a rural system to that of an urban system, for e>eample, would not provide useful information to either entity The last module in Part III compares the ere to its various peer groups which were identified in Worksheet 2. Single CTC Peer An evaluator also may select a single CTC to use as a peer for comparison. The single peer may be the ere with which the local ere being evaluated has the most in common. In some cases, it may be the only CTC with the same combination of organization type, network type, environment, and size. Select i ng a single peer can ensure that the CTC is compared to at least one very similar system and allows much more detailed information to be compared. While peer group comparison information must be compiled from annual statewide operating data, two eres can share more detailed information with each other if their evaluations are conducted simultaneously. If a single peer is included in the evaluation, the information for that coordinator can be included on worksheets as if it were another transportation operator Comparisons with a single peer requ i res a lot of cooperation between the two CTCs. Most importan tly, the reviewer must identify and select a CTC that has the same environment, organization type, network type, availability of fixed-route transit (or lack thereof), and approximate size or as many of these similarities as possible. Included Transportation Providers If the CTC is a brokerage, the reviewer may wish to evaluate the performance of the transportation operators who deliver the trips in the coordinated system. This approach may be used to enable system-wide improvement by identifying weaknesses and strengths, and also can be useful in p l anni ng how trips should be allocated among operators. Such reviews can suggest the cheapest or highest quality operator for trip ass ignm ent purposes. These individual operators also can be compared to the standards set for the local program. In the case of a partial brokerage, the transportation coordinator also is a tran sportatio n operator and should be included II II

PAGE 34

Part II: Getting Starred as one of the "operators" in the comparison of operators. In such a case, it is important to allocate costs appropriately, separating the costs of providing transportation from the costs of administering the brokerage. Potential Transportation Operators These are organizations that do not have a contract with the ere and are not part of the coordinated system, but are being considered as potential transportation operators to be brought i nto the coordinated system. They are referred to as "not-included," "unincluded," or "considered" transportation providers or operators. Including such an operator in the evaluation does not need to result in the official selection or denial of that operator. This model may include any number of operators, whether they have indicated interest in being included in the coordinated system or not, although obtaining information from a disinterested operator would be challenging. Elements of the model also could be used as one part of the formal selection criteria for competing operators. Standards A comparison also may be made with established performance standards. The specific compon ents of cost, quality, availability, and other areas can have specific minimum standards set for the coordina t or, the system, or the tranSportation providers. Standards might be required by an organization or be voluntary. A local coordinating board, might, for e)(ample, establish minimum service standards that it uses to measure the performance of the eTe. In other e)(amples the coordinator may have m inimum service standards that transportation providers under contract must meet or the reviewers may identify a suggested standard to compare with before recommending it for implementation. In any of these cases the reviewers may wish to identify a set of standards that will be recorded on the worksheets as if it were another "organization" for the sake of comparing the coordinator and other operators to the standard(s). A column also might be used to establish a new standard based on the results of the evaluation Completing the Worksheet Determine, with assistance from those initiating the evaluation, which entities will be included in the evaluation. Most worksheets are designed to evaluate either the ere or a set of operators, II 12

PAGE 35

Part ll: Getting Started but the same worksheet that compares operators can be used to compare the ere to selected peers by considering them as separate operilt6rs. Each organization or transportation operator other than the coordinator (but including the coordinator as a transportation operato r if applicable) will be assigned a letter designation. On Worksheet 4, beginning on page II 14, write the identifying information for each organization ("operator") in one of the boxes provided. Throughout the evaluation these organizations will be identified on the worksheets as "O perator A," "Operator B," etc. The text and worksheets that compare organizations ("operators"), will tend to refer to them all as transportation operators or providers. Several worksheets compare these "operators" by assigning each to a column designated by a letter and allow each evaluation question to be answered for each organization. Under "Role/Evaluated As," note the role of the operator in the evaluations; that is, how they are included in the evaluation (e.g., "included operator," "CTC transportation," "peer CTC," "bidder," "standard," or "considered transportation provider ") Briefly describe the type of service provided (e.g. stretcher, wheelchair, and/or ambulatory). Also record for each "operator" the address, phone numbers and a contact person to be used during the evaluation. The operating information collected for this evaluat i on applies only to existing service provision. Therefore this evaluation process should not be used as a substitute for selection criteria used during a proposal process. Including information on a potential operator can, however, give the reviewers a feel for how that operator might fit into the coordinated system. II -13

PAGE 36

WORKSHEET4 AGENCIES INCL U DED IN EVALUATION (Page 1 of2) Soo discussion beginning on page II 10 and instructions beginning on page II. 1 2. : "OPE,M}.OR : A{' ; (usually . the. I;TC unless setioticeljs fully, brokel&d) Name: Role/Evaluated as: SeNice{s) provided: Address : Phone: Fax: Contact : Role/Evaluated as: Service(s) prov ided: Address: Phone: Fax: Contact: "OPERATOR c . Name: Role/Evaluated as: SeNice(s) provided: Address: Phone: Fax: Contact Reporting Period : ______ Preparer's Initials: ______ Date: ____ II 14

PAGE 37

WORKSHEET4 AGENCIES INCLUDED IN EVALUATION (Continued; copy i2S needed} See discussion beginning on page II 10 and instructions beginning on page II -12 "OREI.t" TOR !;;. .... . ; .,;, "' _,; .. .. Name: Role/Evaluated as: Service(s) provided: Address: Phone: Fax: Contact: "OPE!tti;TOR .. u Name: Ro l e/Evaluated as : Service(s) prov ided: Address: P hone: Fax: Contact 'z?H':' ; -' ' t k .' ... < < Name: Role/Evaluated as: Service(s) provided: Address : P hone: Fax: Contact: ' .... .. -' ;t>11<< ;i <> """ Reporting Period: ___ ___ Prepare(s I n itials: ______ Date: ___ II -15

PAGE 38

PART III: EVALUATION MODULES Worksheets 5-15

PAGE 39

Part III: Evaluation Modules EVALUATION MODULES Overview Part I of this evaluation workbook introduces the evaluation process, its uses and purposes, what it can do, varying initiat i ves and goals for evaluations the aspects of service that can be evaluated, who to include, and the timing and scope of evaluation. Part I also explains how to use this workbook and how the modules are organized. Part II, titled "Getting Started," describes the key documents that should be assembled. Part II also requires the completion of four worksheets that outline the goals and scope of each evaluation, identify the ere and other organizations to be included, and prompt the collection of a complete inventozy of transportation providers. Part II must be completed before using any of the primary evaluation modules here in to evaluate various aspects of service. Part III contains all the evaluation modules except those that set up the evaluation or form the final conclusions. These primary evaluation modules include: (a) Competition; (b) Coordination; (c) Cost Effectiveness and Efficiency; (d) Aspects of Quality ; (e) Training as a Measure of Quality; (f) Availability of Service; (g) Funding and Accountability; and (h) Comparison of ere to Peers. Each of the modules in Part III includes its own discussion, instructions, and worksheets. Any one more or all of these modules may be used; the only exception is the module "Comparison of ere to Peers," which should nor be used alone Refer to Parts I and II for more information on their uses. III I

PAGE 40

Part Ill: Evaluation Modules (Competition) WorksheetS Competition Local policy and management wisdom should determine the organizational structure and service delivery network appropriate for each community transportation coordinator. This decision, however, should be based on an assessment of available options. Each CTC should be certain they have considered the resources available. Worksheet S in this module examines how local transportation providers are selected and used. Completing the Worksheet Before examining the use of transportation operators, an inventory of local transportation providers should be conducted, or updated if one has been compiled in the recent past This inventory should include all providers of transportation services to the public, whether included in the coordinated system or not, and regardless of whether transportation is the primary activity of the organization. Also include any known transportation operators that are not in the area but have expressed interest in acting as a local transportation provider. The inventory should be complete. The purpose here is to examine what options are available, before examining which options have been exercised. The CTC should be included as a transportation provider if it is a sole-source provider or partial brokerage. Be sure to include school buses, when applicable, under the "government" category. Refer to a complete inventory of local transportation services and to Worksheet 3. Use the information collected for Worksheet 3 to complete Question I on Worksheet S on page III 3. To complete the rest of Worksheet S, simply follow the directions provided for each question. Some research may be required Interpreting the Worksheet Questions 10-13 will require the revtewers to draw conclusions about the level and appropriateness of competition locally. In some cases, the reviewers may believe there are too few or too many providers. In other cases the number may appear to be just right. There is no "rule of thumb"; this judgment will be based on the characteristics of the local area and local priorities. III 2

PAGE 41

WORKSkEETS LEVEL OF COMPETITION (Page I of3) See discussion and instructions beginning on page Ill 2 1. a. Based on a local inventory of all transportation providers operating within the CTC's service area, how many of each type exist (including the CTC)? (The total shoUld match answer 5 on Worl
PAGE 42

WORKSHEETS LEVEL OF COMPETITION (Page !1 of 3) See discussion and instructions beginning on page Ill -2. 7a. In the past five years, how many times have the following methods been used in the selection of transportation providers? 0 low bid 0 request:-s-:lo,-r q u a""li-::lic-a..,ti:-o-n-s ----0 request for proposals-:-=,..------0 requests for interested parties ____ 0 negotiation only _______ 7b. Which of the above met h ods was used to select current providers? _________ 8. Which of the following items are inco rpo rated In the review and selection of transportation providers for i nclusio n as operators i n the coordinated system? Review ot. (check all that apply) 0 capabilities of the transportation provider 0 age of company 0 previous experience 0 management 0 qualifications of staff 0 resources 0 economies of scale 0 contract monitoring methods 0 reporting 0 fina ncial strength 0 pertormance bond 0 res pons iveness to solicitation 0 scope of work 0 safety program 0 training program 0 i nsura nce 0 accident history 0 qualify 0 knowledge of the community 0 the cost of the contracting process 0 price 0 distribution of costs 0 Other: (list) 9. a. If a competitive bid or request for proposals has been used to select transportation providers to how many potential operators was the request distributed i n the most recently completed process? ________________________ b. How many res po nded? _______________________ c The request for bids/proposals was distributed: (./ all that apply) 0 locally 0 statewide 0 natio na lly Reporting Period: _______ Preparer's Initials: ______ Date : ____ III -4

PAGE 43

. WORKSHEET S LEVEL OF COMPETITION (Page 3 of3) See discussion and instroctions beginning on page tit 2. 10. Has the CTC reviewed the possibil iti es of competitively contracting any services othe r than transportation provis ion (such as fuel, maintenance, etc.)? 11. Given the number of t ransportation p r ov i ders available i s a brokerage-type network feasible? 12 Based on the i nfo rmat ion provided i n Worksheet 5, has the CTC achieved an appropriate l evel of competitio n for its organization and community? 13. Based on the i nformation provided in Worksheet 5 what areas of competition need to be pursued or examined in more detai l ? Reporting Period : _______ Preparer's Initia ls: ______ Ill -5 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No Date: ____

PAGE 44

Part Ill: Evaluation Modules (Coordination) What is Coordination? Worksheet6 Coordination The Joint USDOT/USDHHS Coordinating Council on Human Services Transportation defmes coordination in the following way: Coordination is a process through which representatives of different agencies and client groups work together to achieve any one or al l of the following goals : more cost effec tive service delivery; increased capacity to serve 11111Jlet needs; improved quality of service; and, services which are more easily understood and accessed by riders. 3 Florida Requirements Chapter 427 Part I, F .S., defines "coordination" as follows: Coordination means the arrangement for the provision of transportation services to the transportation disadvantaged in a manner that is cost-effective, efficient, and reduces fragmentation and duplication of services. (427.011(11)] The statute furthermore charges the community transportation coordinator with ensuring that coordinated transportation services are provided to the TO population in a designated service area. Level of Coordination A Handbook for Coordinating Transportation Services suggests that coordination can occur at one of the following three basic levels: I. Cooperation; II. Joint use arrangements; and III. Consolidation. "Coordination of Community Transportation S.rvices (Washington D.C : Joint USDOT/USDH.HS Coordinating Council on Human Services Transportation, n.d. ) 4 Ohio Department of Transportation Division of Public Transportation, "A Handbook for Coordinating Transportation Servic e (Washington D.C.: U.S Department ofTransportation, Technology Sharing Program 1991). 111 -6

PAGE 45

Part III: Evaluation Modules (Coordination) The total level of coordination depends upon: (A) The number of elements (described below) that are coordinated and the degree to which duplication and fragmentation of service is eliminated as measured by: the proportion of transportation providers and sponsoring agencies involved in the coordinated system and the proportion of trips needed by the TD population that are met through the services of the coordinated system; and (B) The degree to which each of the coordination elements (described below) are each coordinated. Elements of Coordination Coordination is not necessarily consolidation or centralization, but does require that at l east one of the elements listed below be centralized. Planning Public information Call-intake Certification Eligibility records Reservations Trip allocation Scheduling Transport Disp atching General service monitoring Daily service monitoring Tr ip reconciliation Billing Reporting Cost sharing Information sharing However, if only one of these areas is centralized, the degree of coordination is minimal. Minimal coordination, such as only coordinating planning, wou ld not meet the Florida requirements for coordinated TD transportation When more of these areas are centralized, the degree of coordination increases. As long as at l east one area is centralized, however, it is possible to coordinate other areas without centralizing them. In the most highly coordinated system all of these areas are coordinated; if not centralized. Each coordination element is discussed below. Planning This includes the development of long and short-range plans for how the TD trips in a communit y will be provided. Even if all services are operated independently, coordinated planning helps to ensure that the services complement each other; that is, that services are not duplicated and that certain TO transportation needs are not omitted. Coordinated planning also III 7

PAGE 46

Part III: Evaluation Modules (Coordination) ensures that independent agencies are not working at cross purposes. Effective local coordinating boards are examples of coordinated planning. Florida's requirement that the CTC approve an agreement with each transportation provider receiving government funds in the community results in some level of coordinated planning. Public Information Marketing tactics need not aim to generate business but should coordinate the transportation information that is provided to TO transportation users. Coordinated marketing aims to match individuals and the transportation service(s) most appropriate for them. An example would include publishing a brochure that lists the phone numbeu to contact for the area's various transportation services. Centralized marketing means that the transportation providers market collectively. The more coordinated the service in the area, the fewer phone numbers such a brochure would need to include. This is a valuable step in coordination from the consumer's perspective. It also inevitably results in some level of coordinated planning. Call-Intake Coordinated call-intake seeks to reduce the number of calls that a user needs to make to have transportation questions answered, learn about transportation options, arrange for service, arrange for payment, and/or resolve ride problems. In a highly coordinated service, transportation disadvantaged clie nts would have a single phone number to call for any needs or questions related to transportation, from certification to trip prob lem resolution. Non-transportation entities could provide this phone number or transfer the call. Coordinated call intake can be achieved either by centralizing aU transportation-related or by having a network with a limited number of phone numbers and staff who are able to refer or transfer any misdirected calls to the correct office. Fewer phone numbers for the various aspects of transportation services results in a higher level of call-intake coordination. For the consumer, coordinated call-intake reflects a high level of overall transportation coordination. Coordinated call-intake is frequently combined with some level of coordinated public info rmation III 8

PAGE 47

Part Ill: Evaluation Modules (Coordination) Certification Certification is the stage at which an individual is initially registered as eligible for one or more ttansportation program(s). Coordinated certification means that a potential transportation user contacts, or is referred to, a single office to determine which transportation program or programs the individual is eligible for, if any. This effort usually requires carefully trained personnel and good communication with the included agencies in order to accurately determine eligibility. Centralized certification often results in an identification card for transportation services. At this time centralized certification for multiple transportation programs is rare. When certification is coordinated, eligibility, planning, marketing, and transportation services are usually coordinated as well. Coordinated certification aims to prevent users from needing to register separately with multiple uansportation services. Certification refers only to the stage of the eligibility process during which a potential rider s eligibility is verified and be/she is registered for a transportation service The process of determining what eligibility criteria for various transportation services will be is pan of the planning process. The maintenance of eligibility records is described below. Eligibility Records The goal of coordination of eligibility records is to organize and communicate wbich clients are eligible for which transportation services. That information needs to be conveyed to those that take trip reservations. In a system without coordinated certification it is especially important to coordinate eligibility records. Among other things, the coordination of eligibility records will aid reservationists in linking trips with different funding sources. In turn, this creates a much more usable reservation system for riders, who can reserve trips (or have trips reserved for them) with one call even if the sponsors vary Coordinated eligibility also helps reservationists to make sure sponsors only pay for eligible trips. The actual certification applications and paperwork do not need to be a part of coordinated eligibility, but essential information from each sponsor does need to be communicated to the coordinator. Such information includes: what trip purposes, days, and travel times a rider is eligible for; what special assistance the rider may require; and what the rider's limitations are. Ill -9

PAGE 48

Part Ill: Evaluation Modules (Coordination) Reservations Coordinated reservations means that the reservation process for service is 10 some way standardized or centralized. Call-intake for reservations usually is coordinated, and the reservations probably are placed through the same computer or manual reservation system. In a coordinated reservation system a reservationist s ho uld be able to confmn reservations. Coordinated reservations also means that a single source can provide reservation information to schedulers, which also means that the same source can ensure that only one reservation is placed for a single trip. Coordinated reservations prevent trips from being reserved through two separate servtces. Coordinated reservations does not necessarily require centralized call intake or centralized scheduling Standardizing the way in which all reservations are made also contributes to coordinated reservations. Establishing a standard system for requesting return trips (e.g., will calls), contributes to coordination. Trip A llocation Trip allocation, also called "brokering deals with the way each trip is assigned to a transportation operator. When trip allocation is coordinated, the assigrunent of trips to transportation providers is organized in some way. A broker takes each reservation and assigns it to a particular transportation provider based on predefined criteria, such as cost, capacity, rotation, match of service, or multi-loading. If aU trips are assigned by a single entity, trip allocation is centralized and highly coordinated. A system that randomly assigns all reservations in order to distribute a certain number or share of trips among carriers also has some level of coordination. A consumer option system, in which a rider chooses her/his own carrier, has decentralized trip allocation and is difficult to coordinate. Scheduling Scheduling is the process of assigning trip requests to a specific vehicle at a specific time in a particular sequence for the vehicle. Schedu lin g can be coordinated or centralized. Centralized scheduling the most coordinated form means that a sing le organization assigns all requested trips Ill 10

PAGE 49

Part Ill: Evaluation Modules (Coordination) to vehicles or drivers. An example of decentralized but coordinated scheduling would be where each transportation provider schedules the trips they are responsible for providing, but some allocation system ensures each provider does not receive too many trips to schedule. Another example of coordinated but decentralized scheduling would be a system in whi ch different providers schedule even reserve, trips independently but all store the schedules in a format which can be accessed by other agencies involved in coordinated transportation. Standards for pick-up windows and on-time definitions and standards for other scheduling practices also contribute to coordination. Transport Transport deals with the actual ride and the way in which passengers are picked up, transported, and dropped off. Transport is coordinated if the way passengers are picked up, transported, and dropped off is standardized. This requires all transportation operators to have similar service standards such as pick-up procedures, levels of assistance, training drop-off procedures, ride length time, safety, vehicle features and other transport practices. Coordinated transport also ensures that the appropriate modes are availabl e for the clients transported, and that any transfers are coordinated. Dispatching The primary feature o f dispatching is communication with the drivers during service provision. The coordination of dispatching is increased when drivers maintain radio contact with a dispatcher and when dispatchers communicate with each other. Because a dispatcher can realistically work with a limited number of drivers at one time, it is rare that all dispatching in a community would be centralized, but coordination can be increased through opening lines of communication among dispatchers or by placing all dispatchers in contact with a single entity during operations. Coordinated dispatching aids in the resolution of problems and contributes to coordinated service monitoring. General Service Monitoring General service monitoring tracks the performance of the transportation providers, transportation coordinator, and other ent ities that might be a major part of a centralized or coordinated system. Coordination is increased when operat i ng statistics and performance measures from entities in the 111-11

PAGE 50

Parr III: Evaluation Modules (Coordination) coordinated system are compared combined, and/or analyzed. Setting standards for reporting methods also increases coordination. General service monitoring is fully centralized when all transportation providers report standard information according to standard definitions to a single entity that analyzes this information and makes judgments about the performance of the providers. It is assumed, but not necessarily true, that this service monitoring results in improvements in the service in weak areas Coordinated service monitoring and trip reconciliation can help in coordinating general service monitoring. Daily (RealTime) Service Monitoring Daily or "real-time" service monitoring deals with how transportation problems are resolved as they occur (e.g., bow late rides and no shows are resolved). Central.ized service monitoring requires that all problems involving both the passenger and transportation provider are routed through a single entity. A common example is when passengers have a single telephone number to call when rides are late. Coordinated real-time service monitoring does not require that any other aspects of service be coordinated, but real-time service monitoring is made easier by coordinated call-intake, scheduling, dispatching, and general service monitoring. Trip Reconciliation Coordinated trip reconciliation usually is tied to coordinated billing. Trip reconciliation is the process of ensuring that the trips were delivered as reported by the transportation provider. This process checks the actuality of completed trips late trips, passenger no shows carrier no shows, fares, and other asp e cts of trips that require information for each trip, usually for billing, general service monitoring reimbursements, and determining incentive and penalty payments. The coordination 9f several other elements can contribute to coordinated trip reconciliation. Coordinated trip reconciliation may be centralized, or it may be decentralized with a standard method and coordinated reporting. Billing Billing is the process of collecting fares and reimbursement for trips provided. Coordinated billing can take a number of forms, depend ing on the payment processes, agencies involved, and the organization of the CTC At a minimum a coordinated billing system should be capable of ensuring that each trip is paid for (reimbursed) only once, based on accurate trip reconciliation. Ill -12

PAGE 51

Part Ill: EvalULZtion Modules (Coordination) A billing system that is centralized involves either all payments passing through a single entity, or all bills/invoices originating from a single entity, or lioth. The more billing functions handled by a single entity, the more highly centralized the billing element. Reporting Coordinated reporting is one of the most common elements of a coordinated system. Coordinated reporting means that information such as accounting, operating statistics, measures related to certification and billing, and other informat ion i s reported for all TD transportation service provision (from all transportation providers) according to similar definitions, and it is compiled and examined For funding entities, commissioners, and legislators, coordinated reporting is an important element in a coordinated system. Ot h er Coo r dination There are, in fact, other ways in which TD transportation services can be coordinated that cannot alone resul t in a system that is coordinated, even if the aspect is centralized. Examples include information clearinghouses for transportation providers, coordinated training programs, and certain cost sharing programs. These "other" coordination efforts can reduce costs or improve the quality of service, but would not result in coordination that is recognizable to consumers or funding sowces. Cost Sharing One of the aims of coordination is to reduce duplication and fragmentation of service and, related to this, to minimize costs as services increase. A number of transportation providers can coordinate and share certain costs, such as insurance. Note that coordination of cost sharing does little to reduce duplica t ion and fragmentation of service and, from the user's perspective, does not contribute significantly to coordination of transportation services. Information Sharing Coordination can be greatly enhanced by giving all entities the same good information with which to work. Information sharing can be coordinated in a number of ways, including the coordination elements listed above. Other examples include holding regular meetings among staff from all of 111-13

PAGE 52

Pan Ill: Evaluation Modules (Coordination) the organizations involved, circulating phone numbers and contacts, publishing newsletters, professional development of staff distributing coordinated reports, sharing databases, and the distributing some sort of reference sheets for the organizations involved. Coordinated information sharing is at once one of the strongest and weakest forms of coordination. Alone, it is unlikely to coordinate services, but it is the most important factor in coordinating other elements of transportation services for the transportation disadvantaged. Completing the Worksheet The worksheet that follows should be used to evaluate the degree of coordination in the CTC's system and the overall level of coordination in the CTC's service area. Questions 1 of Worksheet 6, beginning on page Ill 16, examines each of the elements (discussed above) of TO transportation that may be centralized or coordinated. The reviewer should be fully knowledgeable about both the coordinated system and the transportation needs and services in the service area. Therefore, an inventory of transportation services should be conducted first (see Worksheet 3 -Inventory of Local Transportation Services, page II 9). On Worksheet 6, each question should be answered as an "all" or "none" test; that is, each element must be very highly coordinated to score a "yes" response. Do not check "yes" if it seems that an element is only slightly coordinated. It is likely that each CTC will have few "yes" responses. Each question should be answered from the perspective of the ere considering all transportation for the transportation disadvantaged, whether included or not included in the coordinated system. This reflects the possibility that services under the umbrella of the CTC could be very coordinated while service may not be very coordinated within the community as a whole. On Worksheet 6, the reviewer may want to enter comments for each element, although comments are not required The comment areashould contain several types of notes. Comments may be included that: (I) indicate i n greater detail the degree or level of coordination, (2) indicate why coordination in this area is not being pursued, and/or (3) indicate specific improvements planned being pursued or potential. The comment area can be used to note the level of coordination using the USDOT/USDHHS method described above (i.e., Levell, II, or Ill), or to indicate "fully, partly, or not-coordinated." The comment area should not be used to describe the Ill 14

PAGE 53

Part III: Evaluation Modules (Coordination) existing coordinated system in any detail, but only the degree of coordination. A planning comment, for example, could read: "Two providers not included." Certain elements of coordination may be specifically outside the goals of the local community. An appropriate comment could be, for example, that "centralized certification is not intended." In considering the questions on each element of coordination, a knowledgeable reviewer will identify areas where coordination should be improved. Such comments should be noted here. If, for example, certification is intentionally decentralized but reservations are coordinated, an appropriate comment regarding the coordination of eligibility records might be: "Should be targeted." Interpreting the Worksheet To complete Question 21, sum the number of "yes" answers. This total gives an indication of how coordinated both the system and service area are, on a scale from 1-18 Because these questions are very broad, this ranking sho u ld not be used for comparison among coordinators. Furthermore, the "no" responses should be ignored, as this cannot be used as a measure of how uncoordina ted the system is. It is not the goal of TO coordination to centralize every aspect of TO transportation, nor is this possible. This evaluation will, however, suggest ways in which TO transportation may be more coordinated. The comment area is, therefore, especially important. Remember that the evaluation worksheet should not be used as a description of the CTC for readers who are not familiar with the CTC. The final questions and answers should communicate the conclusions of the reviewer. Review the answers to questions 1-21 before drawing additional conclusions in questions 22-2'1. Ill-15

PAGE 54

WORKSHEET 6 LEVEL OF COORDINATION {Page I of3) Evaluate the degree of coordination of TD transportation considering both the CTC and the community as a whole Read discussion beginning on page Ill 6 and instructions on page Ill -14. 1a. Within the CTC's service area there are a total of organizations that prov i de{s) any tra n sportation services to some or all of the general public. (See Worksheet 3 Inventory of Local Services, page II -9) 1b. How many operators are included in the coord i nated system? _________ (See Worksheet 5-Level of Competition, page Ill 3) 2. With how many agencies does the CTC have a coord i nation andfor service contract? __ ;>i 3. Planning Are p l ans for transporting the TO popu l ation coordinated? 4 Public Information Is the i nformation distributed about transportat i on services In the coordinated? 5 Certification Is the way In which ind i v i duals are certified and registered for local TO transportatio n services coord i nated? 6. Eligibility Records Does a system ex i st to coordi n ate wh i c h individuals are elig i ble for which special transportation services in the co m munity? 7 Call Intake Are t ransportation services coord i nated to t h e extent that users can reach a reservation i st o n the first call? 8 Reservations Is the way in which reservations are placed standardized and can a r eservation i st preve n t dup l icate reservations d i fferent carriers? 9. Trip Allocation Is the allocatio n of trip r equests to transportation providers coordinated? 10. Scheduling Is the assignmen t of trips to vehicles coord i nated i n some wdy 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No 0 Yes 0 No DYes 0 No Reporting Period : ______ P r eparer's ______ J[[ -16 Date : ____

PAGE 55

11. 12. 13. 14. 15. 16 17 18. 19. 20. 21. WORKSHEET6 LEVEL OF COORDINATION (Page 2 of J) Transport Are the actual transportation service and modes of transportation 0 Yes 0 No coordinated? Dispatching Is the rea l time communication and directio n of drivers 0 Yes 0 No coordinated? General Service Monitoring Is the over-see ing of the transporta tion 0 Yes 0 No providers coordinated? Dally Service Monitoring -Is the way in which t rip problems are resolved in rea l 0 Yes 0 No time coordinated? Trip Reconciliation Is the way In wh i ch all completed trips are confirmed 0 Yes 0 No for offic i al counts coordinated? Billing Is the process for requesting and processing fares, payments, and 0 Yes 0 No reimbursements coordinated? Reporting Is operat in g i nformat io n reported according to a set of standard 0 Yes 0 No definitions, compiled and examined? Costs Resources -Are costs, which must be in curred by. many or all of t he transportat i on providers, shared so as to 0 Yes 0 No reduce the overall costs of the coordinated program? lnfonnation Resources Is information shared regularly with all organizations invo lved so that they may have a similar D Yes 0 No base of knowledge which smoothes service provision and i ncreases the coordination of other elements? Overall Does the CTC have an approved contract or formal agreement 0 Yes 0 No with organizations which provide transportation in the community? c-moot Commtnt: Com"'*"! Com!Mtll: ComfMI'II: ComiMI'It Comment: Comment: """""" Commtnt: How many of the above eleme nt s of transportation (number of Yes's) are highly coordinated : Reporting Period: ______ Prepare(s Ini tials: ______ Date: ____ lll-17

PAGE 56

WORKSHEET6 LEVEL OF COORDINATION (Page 3 ofJ) 22. Which transporta t ion operators need to be better coord i nated when compared to the others? In what ways? 23. Of the agencies that the coordi nator has coordination or service contracts with which could be better coordinated, when compared to the others? In what ways? 24 What improvements in coord i nation could be pursued? Reporting Period: ______ Preparer's Initials : ______ Date: ____ Ill -18

PAGE 57

Part Ill: Evaluation Modules (Cost) Worksheets 7-9 Cost Effectiveness and Efficiency Regardless of any advantages or disadvantages of a particular organization involved in transportation, the financial resources available will be finite. What each community transportation coordinator or transportation operator is able to accomplish with those finite resources, therefore, becomes extremely important. The Florida Five-Year Transportation Disadvantaged Plan establishes a goal for CfCs that relates to costs: Ensure that service is delivered in the most effective and cost-efficient manner possible. Necessarily, the CTC also must ensure that the transportation operators included in the coordinated system do the same. The ere is given several specific objectives in support of this goal, including: (I) accomplish cost-effective service delivery; (2) ensure optimal utilization of services provided; and (3) ensure utilization of the most cost-effective transportation modes. A number of measures su pport these objectives: operating expense per passenger trip; operating expense per vehicle mile; percent administrative expense are of total expenses; passenger trips per vehicle; passenger trips per vehicle hour ; passenger trips per vehicle mile; percent volunteer-provided trips are of total trips; percent group trips are of total trips; and percent fixed-route trips are of total trips Many of these measures also are appropriate for evaluating the cost of transportation operators. 11119

PAGE 58

Part Ill: Evaluation Modules (Cosr) Evaluating Costs of the CTC and Operators Any assessment of program costs should work with costs that are fully allocated. All aspects of this module will be more valuable if the CTC and each ope rator are properly allocating costs. Fully allocating costs is a standard method of showing all costs, including hidden costs. It is a process that should be entrenched in all aspects of the coordinator's accounting, including recordkeeping, budgeting, and reporting. The TD Commission requires the use of fully allocated costs in annual operating reports. Financial audits are an important mechanism for checking that costs have been properly allocated, and can ensure that the information that is relied on in reports and evaluations, such as this model, is reliable. Most of these measures are based on fundamental operating statistics, collected in accordance with the AOR required of all CTCs. The measures, therefore, are not discussed at length here. The performance meas ures are not reported in the AOR but are easily computed with AOR data and are reported in the Statewide Operations Report. For additional information, the glossary defines many of these terms, and the bibliography lists several docwnents that discuss what performance measures may reflect about service. Note that "costs" are expenses. Several worksheets that follow are used to evaluate the costs of the coordinator. Completing the Worksheets Worksheet 7, shown on page III 23, reviews the performance measures outlined above and compares them to the previous reporting period. Questions l, 2, 4, 6, and 8 can be easily computed by dividing one figure from the AOR by another. Passenger trips per vehicle, for example, is simply the total one-way passenger trips provided in the period, divided by the nwnber of vehicles as defined in instructions for the AOR. The other questions may require some additional research; they provide valuable insight to cost effectiveness but are not crucial. Study the answers to the first eight questions. Answer quest io ns 9-12 based on the table and the judgment of the reviewer. Changes in service that may account for changes in cost-effectiveness should be noted to help interpret the table but shouldn't, themselves, be evaluated. Summary information on the l ast financial audit should be indicated in question 13. Worksheets Sa and 8b, shown on pages Ill 24 and III -25, break down the costs of the coordinator by standard expense categories in order to compare expenses per trip in each major expense area. T h i s information will be most useful when compared to previous periods or a peer III 20

PAGE 59

Part Ill: Evaluation Modules (Cost) CTC. Note: -Worksheet 8b needs to be completed only if more detailed, specific expense areas are of interest to the evaluators. Worksheet Sb is especially helpful if it is found that a particular expense category has jumped in total dollars per trip or compares unfavorably to a peer. It should be used much like Worksheet Sa, using the last column (1") to mark expenses that increased from the previous to the evaluation period. Not all parts of Worksheet Sb need to be completed; for example "Labor and Fringe Benefits" could be completed alone. The expense areas and code numbers shown in Worksheets 8a and Sb are those used in the Rural Transportation Accounting' system and the annual operating report that the TD Commission requires of CTCs. Worksheet 8c, shown on page III 32, must be completed if Worksheet 8a or Worksheet 8b are completed. Several of the measures above are valuable in evaluating the costs of each transportation provider and are used in Worksheet 9, shown on page III 33. These should be rhe providers identified in Parr II, and may also include peers, potential operators, and/or a standard. See page II II for discussion on using a single CTC peer in the evaluation. This worksheet also examines the rate(s) charged by each provider. T he costs of transportation providers should be compared to other providers on the worksheet; however, other factors should be considered as well. The worksheet includes a question that allows the reviewer to mark whether or not there are special factors associated with a provider that should be considered in comparing costs. For an operator that only provides trips for passengers that use wheelchairs, for example, a note such as "W/C Only" should be made. Some of these measures may not be obtainable from aU operators Interpreting the Worksheets As indicated above, the top part of the worksheets should be completed as best as the reviewers are able. Study the table, making comparisons, and answer the questions that conclude each worksheet. Identify areas to be targeted for improvement and the most cost-effective operator. Notes on Worksfleet 8c: This worksheet will require careful consideration of Worksheets Sa and 8b, and will require some judgment and possibly policy consideration. The review team may wish to meet to discuss expenses that seem high before drawing conclusions. The first question asks which costs are high; include expenses high in absolute terms, unusually high, higher than last year, and/or unexpectedly high Complete citation in Appendix A. Ill-21

PAGE 60

Part III : Evaluation Modules (Cost) The modules that follow will help to identify what benefits coordinated TD ttansportation is gaining from each operator. In Part IV the costs and special qualities of the CTC and operators will be considered together Ill 22

PAGE 61

WORKSHEET 7 COST EFFECTIVENESS AND EFFICIENCY O F T H E CTC SYS TEM Answer all questions as they peltain to the ere as a whole using AOR information and researching answers as necessary Leave blank any items that are not applicable, or not determinable. Read discussion beginning on page Ill -19 and instructions beginning on page Ill 20 Measure Eva l ua tion Previous PeerCTC Period P eriod (Optiona l ) 1. Op e ratin g Expense per Passe nger Trip $ $ $ 2. Operating Expense per Vehic l e Mile $ $ $ 3 Admi n istrative Expense as a Percentage of % % % T o t a l Operat i ng Expense 4. Passenger Tri p s per Vehicle 5. Passenger Trips per Vehic l e Hour 6 Passenger T rips per Vehicle M i l e 7. Volunteer provided Trips % % % as a P e rcentage of T otal Trips 8. F i xed ro u te Trans i t Trips % % % as a Percent of Total Trips 9 I n wh i c h c o st-effectiveness and eff i cie n cy areas has the coord i nator improved? 10. In wh i ch areas has the coo r dinato r worse n ed? 11. What changes i n serv ice may accou n t for this change? 12 What a r eas sho uld be targeted for im p rovement? 1 3a Date o f CTC s las t f i nanc i a l audit: b. Were the r e critic.af findings to be reso l ved? 0 Yes 0 No c If so. have they bee n fully r esolved? O Yes ONo Reporting Per iod: ______ Preparer's I n i tials: ______ Date: ____ III 23

PAGE 62

WORKS HEET Sa COSTS BY EXPENSE CATEGORY 1. Read discussion beginning on page Ill19 and instructions beginning on page Ill-20. 2. Enter the number of one-way passenger trips provided through the CTC's coorrJinated system for the evaluation period and the previous period. 3 Enter the transportation expenses according to expense category for the coordinator for each period 4 Divide each expense by the total trips ; en ter this number in the Ct:Jiumns 'CosVTrip' for each period. 5 Compare the Ct:JstArip for each expense category listed by marking a ./ in the last column ( t) wherever the Ct:Jst per trip exceeds t hat of the previous period. '.,. ,.;:h;.;;,,,:;,;, f!YACC" Q UN J (500-599) ;-.fiit.;;, "'" ,<'" '' ;, i}.. Ev al uation Per iod Pre v ious P eriod Expense Cat e gory No. of Tri ps: N o. o f Trips: t E xpe n se CostfT rlp Expense CostfT ri p Labor (501) $ $ $ $ Fringe Benefils (502) $ $ $ $ Services (503) $ $ $ $ Materia l s and Sup p lies $ $ $ $ Consumed (504) Util i ties (505) $ $ $ $ Casualty and Liabi li ty $ $ $ $ Costs (506) Taxes (507) $ $ $ $ Purchased Tra n sporta t ion $ $ $ $ Serv i ce (508) Miscellaneous Expenses $ $ $ $ (509) I nterest Expense (511) $ $ $ $ Leases and Renta l s (5 1 2) $ $ $ $ Annua l Depreciation a n d $ $ $ $ Amortizat ion (513) Contributed Serv i c e -$ s $ $ Allowable Expenses (530) Inel ig i ble Expenses (550) s $ $ $ A ll ocated I n direct s $ $ $ Expenses ( i f app l icab l e) SYSTEM TOTAL $ $ $ $ 6 Complete Worksheet 81:> (optional) with more detailed information for any or all expense categories of interest. 7 Complete Worksheet Be. Reporti n g Period: _ ___ Preparer's I n i tials: ______ Date: ____ Ill -24

PAGE 63

WORKSI;JEET Sb (OPTIONAL) DET AILED COSTS BY EXPENSE CATEGORY (Page 1 of7) Wo'*sheet 8b is optional. See Worksheet Ba. Read discussion beginning on page Ill 19 and instructions beginning on page Ill 20 Expens e Category 4 mi 'f1l< : ,., ;. ,,, 501.01 Operato r Sa la ries and Wages $ $ $ $ 501.02 Trai ni ng Salaries a nd Wages $ $ $ $ 501.03 Dispatchers' Salaries and Wages $ $ $ $ 501 .04 Admin i strative Salaries and Wages $ $ $ $ 50 1.99 Other Sal aries a nd Wages $ $ $ $ . ".,, : .... ,,. .; :. 1 ::: ,;' ''1;?, ;c;r, lt'::: .' r Fringe Benefits (502) ..,; 502 .01 FICA $ $ $ $ 502.03 Medical Hospital, a nd Surg ica l $ $ $ $ Insurance Pla n s 502.05 Life I nsur a n ce Plans $ $ $ $ 502.08 Worke r's Compensatio n Insura n ce $ $ $ $ 502.10 Hol i day Pay $ $ $ $ 502.11 Vacation $ $ $ $ 502 .99 Fringe B enefits O t he r $ $ $ $ $ $ $ $ $ $ $ $ $ s $ $ $ $ $ $ $ $ $ $ $ $ s $ s $ $ s HN ,. Report ing Period: ______ Prepare(s Ini t i als: ______ Date : ____ III 25

PAGE 64

WORKSHEET 8b (OPTIONAL) DETAILED COSTS BY EXPENSE CATEGORY (Page 2 of7) WOII Utilities (505) <' . 505.02 Uti l i t ies -Tel e p hone s $ $ $ 505.99 Uti lities Othe r $ $ $ $ Report i ng Per iod:. ______ Preparer's Ini t i als: ___ ___ Date: ____ III 26

PAGE 65

WORKSHEET 8b (OPTIONAL) DETAILED COSTS BY EXPENSE CATEGORY (Page J of 7) Worlcsheet Bb is optional. See Worksheet Ba. Read discussion beginning on page Ill 19 snd instructions Ill 20. Expense Category Liability d-I L """li';W->' .. __ ,-_'t"', .$'!:>.<';. I"_. < *"'',.;.{f '''"" ->>' . lt$\'f'<," : 506.01 Premiums and Physical Damage $ $ $ $ I nsurance 506.036 Premiums for Public Liability and $ $ $ $ Property Damage Settlements 506.99 Other Insura nce $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ '';i '! ,: TaJ<"!' ; (507) . , > . 507.03 Property Tax $ $ $ $ 507.99 Other Taxes $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ 6 Expense li ne item 506 .03 appeared as 505.02 in the 1992 Annual Operating Reporting fonns. Reporting Period: ______ Preparer's Initia ls: ______ Date: ____ III -27

PAGE 66

WORKSHEET 8b (OPTIONAL) DETAILED COSTS BY EXPENSE CATEGORY (Page 4 of7 ) Wori
PAGE 67

WORKSHEET 8b (OPTIONAL) DETAILED COSTS BY EXPENSE CATEGORY Expense Category 511.02 In t erest on S h ort-Term Deb t -Allowable J (Page 5 of7) $ $ i rt!li:b! its?'"A . 512.04 Passenger Revenue Vehicles $ 512.05 Service Veh i cles $ 512.06 -Vehicl e Dispatch and Storage $ Cente r 512.07 Maintenance Equ ipme nt Faci l ities $ 512. 1 0 Data Processin g Facil iti es $ $ $ $ $ s s s s $ $ $ .. ;' "_, $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ s $ $ $ $ $ $ $ $ $ $ $ $ $ s $ $ $ $ $ $ $ $ Reporting Period: ______ P reparers Initia ls: ______ Date: ____ lii -29 l! ..

PAGE 68

W O RKSHEET 8b (OPTION AL ) DETAI L ED C O STS BY E XP ENS E C A TEGORY {Page 6 of7) Worl "'& <<,\;>'( ::; -'.': Y.r
PAGE 69

W O RKSHEE T 8 b (OPTIONAL) DETAILED C O STS BY EXP ENSE C A TEGORY (Page 7of7) Worl
PAGE 70

WORKSHEET 8c COSTS BY CATEGORY Read diswssion beginning on page Ill -19 and instructions beginning on page Ill 20. 1 Which expense areas are especially h i gh? 2. Are these high expenses acceptab l e? Are they approved? 3. What strategies cou l d red u c e the u nacceptab l e costs? Reporting Period: ____ __ Preparer's Initia l s : ___ ___ Date: ____ lii -32

PAGE 71

WORKSHEET9 COSTS OF TRANSPORTATION OPERATORS Answer all questions. researching answers if necessal)l. Leave blank any items that are not applicable, or not determinable Read discussion beginning on page Ill -19 and instroctions on page Ill 20. ,,. .. ..... _,. . . ,1 :/ . .,: . :, : :; > . . :>:l 1a. Flat Contract Rate(s) ($amount I $ $ $ $ $ $ unit) Indicate the unit 1b. Detail other rates as needed: (examples: ambulatol)l wheelchair, stretcher, out-of-county, group) $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ $ 12. Special or unique considerations which costs? (Y or N) Exp lanatio n : 3 Operating Expense per Passenger s $ $ s $ $ Trip 14. Operating Expense per Vehicle Mile s $ $ $ $ $ 5. Trips per Vehicle 6. Trips per Vehicle Hour 7 Trips per Vehicle Mile 8 Compare each operator to their peers. Circle the areas where each operator excels. Which operators compare especially well? Reporting Period: ______ Preparer's Initia ls: ______ Date: ___ _ III -33

PAGE 72

Part III: Evaluation i'vfodules (Quality) Worksheet 10 Aspects of Quality A goal of any community transportation program should be to ensure the provision of quality service. The "Florida FiveYear Transportation Disadvantaged Plan" establishes the following goal for local CTCs: Ensure that quality service is attained This goal is supported by several specific objectives: (I) encourage courteous customer relations and passenger comfort; (2) provide service that minimizes customer travel and wait times; and (3) provide safe and reliable service At a minimum, the following five measures, which reflect the above objectives, should be examined: percent of on-time pick-ups; number of complaints and grievances filed; accidents per I 00,000 vehicle miles; vehicle miles between breakdowns; and number of persons completing training programs. Because many tasks can directly and indirectly support the above objectives, there actually are many ways to measure the quality of transportation service. These means of evaluating service and organizational quality are discussed below and included in the worksheet that follows. Training as a measure of quality is discussed separately in the next module. On-Time Performance On-time performance is one of the most important measures of the success of a transportation program because it directly measures the most basic task of the transportation provider: having people and vehicles in the right place at the right time. On-time performance is a function of vehicle maintenance, scheduling, operating conditions, driver performance, and knowledge of the service area. Most problems encountered in operations will affect on-time performance. On-time III -34

PAGE 73

Part Ill: Evaluation Modules (Quality) performance, in tum, affects other aspects and measures of quality. For instance, poor on-time perfonnance may be caused by vehicle breakdowns, resulting in late trips, reduced service quality, increased complaints, an increase in accidents, and sometimes even an increase in costs. On-time performance should therefore, be monitored very closely. Measuring on-time performance is not without potential problems. A consistent pick -up time policy must be established in writing and communicated to drivers and passengers. The data entered by reservationists and schedulers and recorded by drivers and record keepers must be accurate. A consistent definition of "on-time" must be used. If co mparisons are made among organizations, then definitions need to be similar. Pick-up Procedures The definition of an on-time trip must be based on reservation and pick-up procedures. Policies regarding the pic k -up window, driver arrival time dwell (wait) time, actual boarding time, and drop-off time must be established in writing and communicated to drivers and passengers. Pick-up Window To facilitate on-time hoardings and the monitoring of on-time trips, passengers and drivers must know the scheduled pick-up time and the pick-up window. The use of a pick -up window recognizes that the pick-up time may not be exact. The pick-up window is a pre arranged interval, based on the sc heduled pick-up time during which the passenger should be ready and expected to be picked up. A p ickup window may, for example, be set as the period I 0 minutes before and after the estimated pick-up time or as a 30-minute period beginning at the estimated pick-up time or a 20-minute period begiruring 5 minutes before the scheduled pick-up time. Normal pick-up window policies may be different for scheduled versus will-call trips. A longer pick-up window gives the transportation provider more flexibility. However, a pick-up window that is too long may result in passenger complaints about service reliability and decreased service quality. Drive r Arrival Time This is the time that the ddver arrives at the correct pick-up point for the passenger. If arrival time has been recorded then any delays caused by passengers who were not ready when the driver arrived should not be counted as a la te dde aga inst the driver. Ill 35

PAGE 74

Part Ill: Evaluation Modules (Quality) Dwell (Wait) Time This is the period during which a driver is expected to wait for a passenger to appear. Dwell time recognizes that scheduling prohibits drivers from waiting indefinitely for a passenger to appear, but that a minimum wait time is necessary to give passenger s time to appear. The driver might wait no more than 5 minutes, for example. If the driver arrives earlier than the scheduled pick-up window, the wait time should not begin until the pick-up window begins as passengers cannot be required to be ready before they were told to expect the vehicle. If the driver arrives late for the pick-up (e.g., after the pick-up window) the dwell time usually has to be handled on a case-by-case basis, depending on the schedu l e and how the problem trip is being resolved. Actual Boarding Time This is the time the passenger actually boards the vehicle Significant care should be taken to record this time accurately. Drop Off Time This is the time the passenger actually arrives at her/his dest i nation. For a trip to be on time, a passenger must be dropped off at or prior to her/his scheduled arrival (or appointment time) Keep in mind that even an on-time trip may not correspond to a passenger's preferred trave l time if the system has capacity constraints. Trip Timeliness Categories All trips should first be classified as one of the categories listed be l ow. completed trip passenger no-show passenger cance l ed trip carrier missed trip Completed Trip A completed trip is one in which the passenger was delivered to his/her destination. Passenger Canceled Trip A trip canceled by a passenger after the trip has been scheduled should be recorded. Some systems distinguish between ear l y and late cancels to encourage passengers to cancel trips as soon as they know they will not need service. Ill 36

PAGE 75

Part Ill: Evaluation Modules (Quality) Passenger No Show Only those trips for which the driver waited at the correct location for the prescribed amoWlt of dwell time within the pic.k-up window and the passenger failed to appear are considered passenger no shows. Carrier Missed Trip A missed trip occurs when a scheduled trip is not completed because the carrier failed to attempt the trip (for whatever reason) or when the carrier did not wait the prescribed amoWlt of time within the pick-up window at the correct location. Trips that are completed should then be recorded as one of the categories listed below. on-time trip late trip early trip will-call trip OnTime Trip To be considered on-time, the driver must arrive at the correct boarding location during the scheduled pick-up window or the passenger must actually board during the pick-up window, AND the passenger must be dropped off at the correct destination at or prior to the scheduled appointment/arrival time. Early Trip Pick-ups made prior to the pick-up window are early trips and should be coWlted separately from on-time trips because it is not known whether the early trip was a problem for the passenger. Early trips do not include trips where the driver arrived before the pick-up window, waited, and picked up the passenger during the pick-up window. Late Trip A trip should be considered late in either of two cases: When the driver arrives for the pick-up after the end of the pick-up window, or If the passenger is dropped off later than the scheduled appointment/arrival time. Will-Call Trip Will-call trips usually are requested for a return trip. Although the transportation provider usually knows to expec t a request for a will-call trip, it does not know the exact time a Ill 37

PAGE 76

Part III: Eva/uarion Modules (Quality) passenger will call to request his/her trip. Therefore, the trip cannot be scheduled in advance. Although passengers cannot wait indefinitely, the pick-up time may be harder to predict. Therefore, will-call trips often have different criteria for on-time performance, such as an extended pick-up window and relaxed on-time performance criteria. Whatever parameters are selected, the system should record will-calls and track the elapsed time between when the request was received and when the trip was provided. Trip Length The travel time of a trip only affects the on time performance of that trip if it causes that trip to be late. Long trip lengths also can cause other trips to be late. The length of trip is itself, however, a measure of quality. As a quality measure, trip length is measured in time, normally minutes Evaluation of trips lengths should depend only upon: The established policy, and The normal length of time of the same trip made by a safe driver in a single-occupant vehicle during the same time taking a direct route. Because few systems have access to detailed traffic information, some basic knowledge of average travel times and some "hunch judgment is necessary when developing schedules and routing vehicles. Policies regarding maximum trip lengths can vary widely depending on the size and density of the service area. Trip length can be measured by subtracting passenger boarding times from drop-off t i mes recorded by the driver. Although trip length should be monitored closely by a transportation system, it will not be used in this model to evaluate the coordinator or providers. Other Measures of Service Quality Other important measures of quality involve public interaction with the CTC system, including system reliability, call hold time public information and communication, complaints and compliments, grievances, safety, vehicles, and preventive maintenance. System Reliability By defmition, reliability i s a measure of quality. One measure of reliability is the passenger's ability to place reservations which, as a matter of system capacity, is evaluated under Availability: Ill -38

PAGE 77

Part lll: Evaluation Modules (Quality) Another measure of reliability is the extent to which the system nms on-time, or actually provides the trip, measured as on-time performance (discussed above). Other measures that are related to the reliability of vehicles are discussed below. Call Hold Time The time it takes a patron to place a reservation or make an in quiry also is a measure of quality. lf at all possible, systems should monitor h ow long calls for reservations or trip resolution are placed on hold Both the average hold time and maximum hold time should be measured for certain periods. If the telephone equipment is not capable of reponing hold queue information, spot checks should be made by staff Public Information and Communications For the consumer, information and communications is one of the most important aspects of quality. The usefulness of a paratransit service is limited by its capacity to communicate effectively with riders. I nformation needs to be readily available to the public and be available in formats that are accessible to the expected users. Simple brochures may be used to describe the service provided, who is eligible, days/hours of operation, and reservations procedures, including telephone number(s). Users need to be able to contact the coordirtators or providers with relative ease. The availability of information and ease of communication should be measures as aspects of quality. Complaints and Compliments A system for recording and countin g complai nts should be in place. All individuals in a position to accept complaints should be aware of the complaint policy. Instances where a communication that initially appeared to be a complaint but were genuinely resolved with a simple clarification should n o t be recorded as a complaint. Othenvise all complaints should be recorded as such. Complaints should be reviewed regularly to spot patterns and to take corrective action. Many systems group complaints by subject (such as timeliness, vehicles, customer service, or driver) and evaluate their numbers monthly. Because complaints vary in severity, complaint s also can be classified based on when and how they were resolved Note that simply handling a complaint does not achieve its resolution. Ill 39

PAGE 78

Part Ill: Evaluation Modules (Quality) A complaint can be considered reso l ved when the problem that inspired the complaint is corrected. The first leve l of complaints are complaints resolved at the time of the complaint. True late ride calls are normally resolved on the phone. These late ride complaints should be recorded as a subset of other complaints The second level are complaints resolved later. A complaint about a vehicle's wheelchair lift that resulted in a corrective repair is an example of a complaint resolved later The third level are active complaints These include those that have not yet been corrected and includes both complaints that will be dealt with and those that cannot be corrected for some time. Compliments also should be recorded and handled either as a commendation for an employee's file or posted as encouragement. It i s unfortunate ly, normal that compliments are not sufficiently common to be a useful quality measure. It is up to the entity whether it wishes to track other types of input from passengers. Grievances Each CTC is required to have a written grievance policy. Grievances are generally considered to be those complaints that have not been or cannot be resolved, and have been formally registered and require a formal review. Grievances include any complaints that, having exhausted all existing administrative remedies, have been formally brought to the attention of a relevant entity outside of the organization the initial complaint was against. Grievances should be counted separately from complaints. Grievances should be few, with a target number of zero. Although grievances are not a good measure of quality, high number o f grievances does represent a deficiency in quality. Safety Safety is one of the most important aspects of quality. A failure in safety is a failure of the program Further, failures in safety are costly. Although there are a variety of ways safety can be measured and predicted, safety is most easily measured by monitoring a ratio of accidents to miles traveled. The Florida Statewide Operations Report lists accidents per I 00 ,00 0 miles data for each CTC for comparat ive purposes As with other perfonnance measures, tracking safety is more meaningful with consistent record keeping. Specific safety measures also include numbers of accidents and incidents. III 40

PAGE 79

Part Ill: Evaluation Modules (Quality) Accidents For the AOR, it is necessary to report only those accidents that result in personal injury and/or property damage. However, any time the vehicle comes into contact with another objec t should be recorded as an accident, regardless of damage. The objective is to prevent all accidents, not only the most serious ones. Even a slight scratch can be expensive if it exposes the vehicle to rusting. This zero-tolerance accident reporting process gives a clearer picture of the true level of safety, as the difference between a s light" and "serio us" accident may be only a fraction of an inch. (Near misses also are near hits.) Drivers should be clearly aware that an accident that results in minor damage is no more acceptable than a costly one. Clearly accidents also include any event that results in physical damage to person or property, however minor. Good mana gement practices suggest that each accident should be reviewed and classified as preventable or unpreventable. Note that most accidents are preventable This classification is not ne cessarily the same as deciding who may be legally responsible, charged, or at fault. The concept of preventable accidents is based in the premise of defensive driving. All vehicle operators should strive to prevent accidents from occurring, not just avoid causing them. Accidents also sho u ld be separated into person only, vehicle only, and those involving both vehicle and person. Incidents Any unusual occurrence should be reported as an incide nt. Incidents are safety-related occurrences that did not result in physical damage to person or property and the vehicle did not touch anything other than the road. An example of an incident that should be reported would be if a passenger falls, states she/he is okay, and refuses further assistance. Such incidents and how they occurred should be made part of the permanent record. A separate file of incidents during non-revenue time (e.g., incidents in the shop or in the parking l ot) may be kept. Because incidents include many occurrences that did not eventually result in a problem, incidents always should be recorded but not counted as a specific measure of quality. Vehicles The quality of the system also is reflected in its vehicles. Safety and comfort, important to any transportation system, are especially important factors when assisting the transportation III-41

PAGE 80

Part Ill: Evaluation Modules (Quality) disadvantaged population. Older persons and individuals with disabilities are particularly susceptible to injury. Many testing devices and checklists exist that can be used to measure the safety and comfort of vehicles. A quick evaluation of vehicles as part of a larger picture, however, can focus on just a few aspects. The average age of a fleet can reflect the reliability of vehicles and should be considered. Note, however, that having newer vehicles is not always preferable to keeping costs down by properly maintaining an older fleet. How often vehicles are cleaned, how many have working air-conditioning, whether interior lights work, how high the steps are, and the effectiveness of the preventive maintenance program also should be examined. Preventive Maintenance An effective maintenance program does not just repair broken vehicles, but actively works to prevent mechanical fai l ures through a preventive maintenance program. Breakdowns and the resulting roadcalls are expensive and detract from the reliability of the service. The reliability of vehicles correspond directly with service reliability. A preventive maintenance program, through such activities as regular inspections repairing small problems regularly scheduled maintenance (e.g., oil changes), and following the manufacturers sugge s ted service intervals and DOT standards, seeks to minimize costs and increase the safety, comfort, and reliability of the fleet and, thereby, the system. The success of this program can be measured in terms of mean vehicle miles between breakdowns (the total number of vehicle miles traveled divided by the total number of in-service roadcalls). A successful preventative maintenance program also can result in fewer trips delayed or missed due to vehicle equipment failures if the entity has the capability of tracking the reasons for late and missed trips. Administrative Management The CTC also must be proactive in a number of overall management areas to ensure that the quality of TD transportation remains high. These areas include : risk management goal-setting and plans for improvement, quality management, and special qual i ties of service. Training, which is directly related to system quality, is described briefly in this module and is covered in considerable detail in the next module, "Training as an Aspect of Quality." Risk Management As a legal protection and as a precaution for the safety of passengers, all transportation providers must carry liability insurance. The limits of this insurance in dollar amounts, can be used as one III -42

PAGE 81

Part Ill: Evaluation Modules (Quality) measure of quality when comparing transportation providers. Transportation providers must have liability insurance that covers commercial use. Many personal auto insurance policies do not. Entities that are unsure should check with iheir insurer or read their policy Vehicular liability insurance often has separate limits per person and per occurrence. For Florida TD transportation. the limits of liability coverage must be at least $100,000 per person and $200,000 per incident. Insurance in excess of $1 million per incident must be approved by the TD Commission if its cost is included in service rates.7 CTCs and their transportation providers must be in compliance with the safety requirements of Section 341.061 F.S, which requires adherence to minimum safety standards for systems operating with state funds. A System Safety Program Plan (SSPP) must be developed in accordance \vith Florida Rule 14-90. Federal and state regulations have been developed for drug and alcohol testing, an i mportant aspect of risk management. Federal regulations dealing with blood borne pathogens (e.g., AIDS and Hepatitis B) also have been adopted, and standard operating procedures for handling blood, urine and other bodily fluids should be de v eloped to protect drivers and passengers from infectious diseases. Goal Setting aod Plans for Improvement Evaluations and monitoring are important tools for directing the focus of improvement efforts, yet each CTC needs to strive regularly to improve the quality, availability, and cost-effectiveness of the system. Annul goal-setting exercises should be used to ensure that everyone involved in the delivery of service understands what is expected of the system. Long-term goals also should be developed to give the system an overall. st r ategy for the future. Goals should be supported by measurable objectives. These objective should cover service delivery issues, as well as intemal management issues. These goals and objectives should be incorporated into the C TC' s overall plans for improvement. At any given time plans should be underway for improving the coordinated transportation service in some way, in response to on going monitoring of system performance. 7 Rule Cha pter 4 1-2. 006 F .A.C. Ill 43

PAGE 82

Parr III: Evaluation Modules (Quality) Quality Management All organizations should have a program to ensure the quality of its products and services. This also is true of publicly funded transportation services Quality improvement programs should be tied to the measurable goals and objectives described in the previous section. Today, many of the transferrable concepts and practices that have proven successful in managing quality are embodied in management philosophies such as total quality management (TQM). The TD Commission does not require the adoption of TQM management philosophies per se, but does require quality of service. Quality management programs are useful tools in achieving quality of service Service plans, annual evaluations, the annual operating report, a management information system, passenger surveys this evaluation model, and employee evaluations are examples of the tools necessary for measuring quality and performance. Many reports tailored to each transportation coordinator can be useful such as those described in Part II. Another good source of information and ideas that can contribute to continuous quality improvement are the drivers, dispatchers, schedulers, telephonists, and others who have daily contact with passengers and sponsoring agencies. Special Qualities or Services Some transportation operators or coordinators may provide special services or offer outstanding quality of service that is not measured among the aspects of quality described in this module. For example, some coordinators may call to verify scheduled pick -up times or may offer to wait for the passenger's return trip. These special qualities (and their costs), also should be taken into account when comparing organizations. Training The quality of the system as a whole is directly related to the performance of those involved. Because people are only able to perform within their abilities, the training they receive impacts the quality of their performances. Training also is one of the best ways to ensure that the policies of the system however well developed are carried out consistently. A quick way to measure training is by examining the number or percentage of employees who have completed training Because t he skills of individuals are so important, training is examined more closely in the next module, which begins on page III 54. Ill 44

PAGE 83

Part Ill: Evaluation Modules (Quality) Completing the Worksheets Worksh eet 10 evaluates many of the aspects of quality described above. Decide which transportation providers will be included in the evaluat i o n In mos t cases, these providerS should be the same as those listed on Worksheet 4 in Part II. One column also should be used for noting the CTC's performance during the previous period (e.g., last year). This will help to draw conclusions about the performance of the CTC compared to the previous year Write in abbreviated names for each operator in the heading at the top of both pages of Worksheet lOa. If needed, make one copy of Worksh eet lOb for each organization to be evaluated and write in the names at the top of each page Make copies of Worksheet lOa if more columns are needed for more operators. Worksheet l O a should be the basis for the evaluation and comparison of quality measures. Worksh eet l O b simply provides a suggested format for calculating many of the measures to be included on Worksheet lOa With regard to on-time performance in particular, some of the suggested data may not be obtainable currently If another method is used to calculate a certain statistic, note the method somewhere on the appropriate worksheet. Read through Wo r ksheets lOa and lOb first to see what information will need to be collected or calculated. Statistica l and descriptive information will be needed from each organization included in the evaluation. Tell each entity what information will be needed, then begin to complete Worksheet lOa one column (one operator) at a time Work with each entity to gather information and make sure the evaluator understands the definitions of the i nformation reported by each transportation operator. Measuring quality requires both subjective information, and quantitative information collected according to strict d e finitions. These worksheets rely on both. Worksheet lOb assists in the calculations of many of the quantitative measures, so it should be used for each operator only as needed Worksheet lOb is, however, based on the classifications of quality information suggested in the discussion above. To the extent possible, the numbers used in Wo r ksheet lOa should be based on a classification similar to that suggested As a worksheet, however, the form is flexible enough to include local policies regarding service standards and record keeping. More importantly try to standardize the information collected on or from each operator. If that information is inconsistent note the inconsistencies on the worksheet; indicate on Worksheet lOc t hat standardization of reporting on quality needs to be pursued. Some questions rely on the judgment of the evaluator, so the evaluator must be sure to answer these questions consistently III -45

PAGE 84

Part Ill: Evaluation Modules (Quality) Be sure to record information based on the evaluation/reporting period established m Worksheet I, titled "Defining the Evaluation," completed in Part II. Interpreting the Worksheets The discussion in this module addresses each aspect of quality examined: on-time performance, other measures of service quality, and administrative management. Refer to this discussion for information on the importance of each aspect of quality. After completing Worksheet lOa and reading the discussion, complete Worksheet lOc. After completing Worksheet lOa, the reviewer should have a good understanding of the available data on quality. Indicate on Worksheet JOe if standardized reporting of information reflecting quality is sufficient or needs to be pursued as a quality improvement task. Looking at each major aspect of quality (on-time performance, complaints, safety, etc.), compare the CTC and each operator Indicate on Worksheet lOc the areas in which each seems to need the most improvement. Indicate the provider that excels in each area. Training is examined more closely in a separate module. Compare the coordinated system to its own performance in the previous period. For Question I, indicate whether the CTC has i mproved Based on both Worksheet lOa and the first few conclusion questions in Worksheet lOc, determine whether or not each operator may be counted as an asset to the coordinated system in each aspect of quality. These answers generalize the more specific worksheet questions to create a summary and will be combined with conclusions from other modu les in Part IV of this evaluation workbook if a full evaluation is being conducted. With regard to quality, it should be assumed that all transportation coordinators and providers have the ability to improve quality in all areas. Indicate any specific minimum standards for quality which the local coordi nated system as a whole might strive to reach (optional). III -46

PAGE 85

WORKSHEET lOa EVALUA T ION AND C O MPARISON O F > -- ASPECTS OF QUALITY (Page I of4) Answer all questions, researching answers and using Worl? (number of minutes) 5 In how many p l aces Is information about TO tra n sportat ion available? (number) Comments: Reporting Period: _______ Prepare(s I nitials: ______ Date : ____ lll 47

PAGE 86

WORKSHEET lOa E VALUA T ION AND COMPARISON OF ASPEC T S OF QUALITY (Page 2 of4) CTC ---Question CTC Prev. IfiHiiiiiiij rpj 6. Number of fonnal grievances filed (against) (# or YIN) 7. Does a written policy exist defin ing how complaints are counted? 1 "'" B. 1 per 1000 Trips (From Worksheet 10b) 'm:. < 9 1 per 100.000 miles Worksheet 10b) 10. Does a written exist acc i dents? (YIN) 11. Is each accident reviewed to determine how it could have been I',. and Comfort 3 12. Mean (average) m i les between r oadcalls (From Worksheet 1 Ob line 4c) 13 age of paratransit fleet. 14. Is maintenance performe d accor di ng to DOT standards or manufacture(s recommended i nterva l s? 15 How many days between vehic l e clean i ngs? a. Interior? b. Exter i or? 16 .. Are whee l chair l i fts cycled regularly (e.g daily)? (YIN) !17. Number of vehicles without 18. Number of non -bus vehicles w i thout a seat belt for every seat --Reporting Period: ______ P r epare(s I n i tials: ______ Date: ____ Ill 48

PAGE 87

19. WORKSHEET lOa EVAL U A TIO N AND C OMPARI S O N OF ASPECTS O,F QUAL ITY (Page 3 of4) CTC --Question CTC P re v ... riod without funct i oning in t erior (dome) light --20. Vehicles with step hig her tha n 10" or wit h out a uour boa r ding stoo l 2 1. Number of veh i c l es w it hout fi rst aid kit fire I and reflective warning t riangles Comme nts: ;. .. . . .. . --22. Limit of l iab i lity coverage : Is s 1$ Is a. per perso n (AOR) b. per occurrence ''"'"" I s Is $ Is 1123 I s t h i s l i ab i l ity coverage at l east p erson and at l east $ 2 00 000 per occurrence? 2 4. I s t h i s l i ability cove r age app l icable to commerc ial t ransportat i on? Refer to discussion [ 25. Does th e entity meet14-90, F A.C a nd have a written Sys t em Plan? 1 urv1 126. Does the entity meet current requ i rements for drug and alcoho l jf ( T'IV, _ .,.;;, .'' . -:>;;: _, a n d 1 27. D o es the have written and 28. Are t h ere specific plans i n p l a c e for furt h er improv i n g the quality of transportation i n the n ext year ? In t h e l ong ter m ? 29. Do these p lans i n c l ude an schedule? Comments -$ 1$ $ $ .. Report i ng Period: _______ P repa rer's Initia l s: ______ Da t e: ____ III -49

PAGE 88

WORKSHEET lOa EVALUATION AND COMPARISON OF ASPECTS !)F QUALITY (Page 4 of4) CTC --Question c TC PI9V. """"" --. .. . 30. Is there a quality improvement program? (YIN) a. If so, is it forma l ? (YIN) b. Is it i nfomnal? {YIN) 31. Are top managers committe d to leading quality and (YIN) 32 Is every employee in cluded in the 33. Does the nu21iru improvement program focus on the !YITI'' 34. Is eac h employee part of one or mo re c r ossfunctio n al teams? 1Y_IN) 35. Ale all employees trained in quality i n ways that can enh ance perfomnance i n l h eir area of 36. Is i measured and mo n itored r, (YIN) .(1 an d Services <', ; 37. Does the organization offer any oth er specia l q ua l it ies or services i n the provision of transportat io n service? (YIN, with key words) .. vvuu Reporting Per iod: ______ I nitials: ______ D ate : ____ Ill -so

PAGE 89

WORKSHEET lOb CALCULATION OF QUALITY (Optional, copy as needed for each entity) Read d i scussion beginning on page Ill -34 and instructions beginning on page Ill 44. Complete this worksheet only if needed to complete Worl!sheet 10a. N ame ot Coordi nator or Ope rator: ----------------------b No Will Calls + No. Trips = c. = d. No. Late = e. Percent of Trips OnTime o/o 1 :;;-;;: t \ .,. ..... """' '-< > ... ., w c ., Custome r D river To ta l : I 'evafuation.repOtting p&riod: t :i'J,. Timeliness Veh i c l es Serv ice a. Total C omplai nts b Total Trips Provided c. Compla i nts/1000 trips (fine a I line bx 1000) . ; '' '} i ; fM'; "' 3 . Safety Accidents . . . . . : .... ., .. ,.;,; ... .. . ;:,-... a. Accidents (reported in AORISOR) b. To t al vehicle re euae mi l es (reported in AORISOR) c. Acc i dents/100,000 miles (fine a I fine b x 100,000) = 4 a nili '} .,. >, .. ''/':;: . .. : < ., a T otal vehic l e mi l es (3b from Accidents, above) b. Breakdow n s & Roadcalls c. Mean (averag e ) m i l es between r oadca ll s (line a I fine b) = Report ing Per i od : _____ I n itia ls: ______ Date:. ____ Ill-51

PAGE 90

WORKSHEET lOc CONCLUSIONS REGARDING ASPECTS OF QUALITY (Page I of2) Read discussion beginning on page Ill 34 and instructions beginning on page Ill 44 1. Comp let e the following questions, rating each organization with a consistent scale (+/, 1 or AF) and i ndicat ing Whether the organization is doing well i n that aspect or has a potential problem and whether the CTC is doing better or worse than before Each l i ne may have one question to answer regarding the CTC and one regarding each operator. --Evaluation Area CTC On-Time Performance: Is the CTC improving? Is the operator an asset in th is area (benefits TD service, meets or exceeds minimum requi r ements)? Public Information and Communications: Is the CTC i mprov i ng? Is the operator an asset in this area? Complaints and Grievances: Is the CTC improving? Is the operator an asset in this area? Safety: Is the CTC improving? Is the operator an asset in this area? Vehicle Maintenance and Comfort: Is the CTC i m proving? Is the operator an asset in this area? Risk Manag emen t : I s the CTC i mproving? Is t he operato r an asset in t his area? Goal Setting and Plans for Improvement: Does t he o rga nization have specific goals and plans for improvement? Quality Managem ent: Is the CTC i mprov i ng? I s the operator an asset In this area? Special Qualities and Services: Are there special service features that enhance the overall service? Reporting Period:; _______ Init ials: ____ __ Date.:_-----lll 52

PAGE 91

WORKSHEET lOc CONCLUSIONS REGARDING ASPECTS ,OF QUALITY (Page 2 of2) 2. Is quality i nformation suffaciently standardized and reported in the coordinated system? 3. In dicate the major quality areas i n which each operator appears weak: 0 Yes 0 No LocaiCTC:. ________________________________________________ ___ Operator ___ : ____________________________________________________ __ Operator ___ : ____________________________________________________ __ Operator ___ : __________________ Operator ___ : ____________________________________________________ __ Operator ___ : ______________ f.fh edl:illo>'lef,.,.., 4. Which operato r seems to excel in each area? On-Time Performance?.---------------------I nsurance? _______________ __ Complaints and Grievances? ________________ Total Quality Ma na gement? ______ __ Safety? __________________________ ___ Planning? ________________ __ Vehicle Ma i ntenance? ___________________ __ 5. Notes (if any) other recommendations, and areas CTC should target for improvement. Repcrting Period: ___________ ___ Preparer's Initials: _________ Date: _______ __ 111 -53

PAGE 92

Part Ill: Evaluation Modules (Training) Worksheet 11 Training as a Measure of Quality Every service industry depends upon the quality of the people involved in providing the service. The performance of individuals is not only a function of rules, assignments, and motivation, but also of their skills and abilities. Quality personnel often can ensure successful system performance. Because performance is a function of skills, one factor affecting performance that an organization can control is the experience and training of its personnel. Training, therefore, is a good predictor of quality. A Note About Volunteers Training required of volunteers and employees sometimes varies however, in most cases the same training should be required of both. In the text that follows, "staff," "personnel ," and "employees" are meant to include volunteers as well as paid staff. The Value of Train ing How much training to provide or require is generally based on costs and goals. Previous experience tends to increase starting pay rates. Exceptional training provided after hiring can, likewise, increase pay rates required for retention because new skills increase the marketability of the individual. The skills of individuals, however, are among the greatest determining factors affecting the quality of an organization. Well-trained staff and low turnover rates can result in specific improvements in service reduce the amount of supervision necessary, and in some cases reduce the number of employees necessary to provide the same level of service. How much training to provide depends primarily upon local goals. Training will be examined here as a measure of quality. Training Standards Training programs vary among coordinators and transportation providers. The types, course lengths, curriculum, and total amount of training can all differ. One way a coordinator can improve quality system-wide is by establishing minimum training criteria for itself and all transportation providers under contract. If the coordinator has minimum training criteria an evaluation can discove r whether all operators are, in fact, meeting those minimums and which operators exceed requirements and by how much Ill 54

PAGE 93

Part Ill: Evaluation Modules (Training) Means of Training Training can be provided in a number of ways. Training can be conducted on-site by the organization's own training staff, locally by visiting trainers, at another organization with which the organization has a relationship, or off-site at a training agency Different means of training can be used for different types of training Sharing training resources can be a means of reducing costs, improving quality, and improving coordination. Major Categories of Training Several major categories of training exist, based on when training is provided. New Employee New emp l oyee training must be completed as a condition of employment, prior to starting a new emp loyee's regular duties. In most cases, new hires are paid for their training time. Previous experience may reduce the amount of on-th e -job training every employee must go through. Prior transportation experience a lso can provide an organization with unexpected skills and knowledge No matter how innovative an organization may be or how much training it may provide, it can always benefit from new ideas. An organization may or may not allow experience to substitute for new employee training. On-the-Job On-the-job training teaches employees the necessary information for carrying out his/her responsibilities after they have already begun performing that role. This requires direct, often one-on-one, supervision However, on-the-job training is not appropriate for safety-related tasks or those tasks where mistakes would be critical. Most jobs do involve some limited, informal, on-the-job train i ng Refresher All employees should receive periodic refresher training for the purpose of improving or fme tuning skills they a l ready have It is assumed that an individual does have sufficient knowledge lll 55

PAGE 94

Part lll: Evaluation Modules (Training) to carry out her/his responsibilities adequately. Refresher training is ideal for experienced staff that are resuming a critical function after having not performed it for a while. Retraining L ike refresher training, retraining teaches skills to personnel in areas they have already been trained in, but assumes that the individual does not have the skills to adequately perform his/her responsibilities. Retraining is often required for a driver that has had a preventable accident or who, in a performance evaluation, is deemed to have substandard skills in one or more areas. Skills Enhancement Professional development benefits all employees, not just managers and supervisors. This includes any type of training that improves upon the existing training or skills of personnel. Managers might attend conferences, planners might attend planning workshops, drivers or mechanics may obtain additional certifications drivers may take advanced passenger assistance classes, secretaries may take a word processing course to learn more advanced functions, etc. Professional development is not generally required for employees, rather left as optional training, but has the capacity to improve the performance of skilled employees. It also tends to contribute to job satisfaction. The following paragraphs describe types of training applicable to those involved in transportation servtces. Not all systems offer all types o f training, although all can benefit from all types of training. Types of Training Special Licens ing In Florida drivers operating vehicles-for-hire that are designed to carry I 5 or fewer passengers, such as an automobile or van, must obtain a class D (chauffeur's) license. Most TD transportation systems operate in this mode. However, a conunercial driver's license (CDL) is required of drivers operating a commercia l motor vehicle which includes vehicles designed to Ill 56

PAGE 95

Part Ill: Evaluation Modules (Training) carry 16 or more passengers (including the driver) or a vehicle with a gross vehicle weight rating of more than 26,000 pounds. A CO L has more rigorous testing requirements than a chauffeur's license. Vehicle Operation Vehicle operation training focuses specifically on how a particular vehicle operates, including its controls, special equipment, start up procedures, and information about the vehicle. More specific examples include training drivers in the vehic l e s dimensions, weight, seating capacity, pre trip inspections, normal tire pressure, and wheelchair lift operations. Defensive Driving Defensive driving training focuses specifically on driving to avoid incidents and accidents relying only on the driver's own skills Skill and knowledge areas inc l ude: rules of the road, the various causes of accidents, driving courtesy, judgment, precautions, effects of road and weather conditions, assessing your own sk ill s, following and stopping distance, blind spots, evasive maneuvers, positioning for turns, intersection right-of-way, passing, parking, communicating intentions to other motorists, effects of drugs and alcohol on driving, and hazards of vehicle types E mergency and Accident Procedures Emergency and accident procedures training aims to minimize the effects of an incident and ease accident investigat i on. This training focuses on, not how to avoid incidents and accidents, but how to handle them when they occur. Because a single emergency can involve staff in many positions, it is usually applicable for all personnel involved in operations and any employees that use "company" vehicles Topics normally include emergency equipment, emergency notification and communications response, evacuation, protecting the scene assisting passengers in an emergency witnesses, cooperating with police and other emergency response personnel, documentation, the roles of dispatchers and managers reponing, the concept of a preventable accident and accident review and debriefing Ill -57

PAGE 96

Part Ill: Evaluation Modules (Training) Passenger Relations Passenger relations training focuses on how drivers relate to passengers, handle complaints, avoid conflicts, settle disputes, and enforce rules. Passenger relations training often discusses courtesy policies, dealing with irate customers without arguing or becoming angry, when to call the dispatcher, and establishes the boundaries for providing extra passenger assistance. Passenger Assistance Techniques and Sensitivity Passenger assistance techniques and sensitiv i ty courses teach specific methods for helping passengers who require additional assistance. Typical topics address the effects of djfferent types of impairments and disabilities as they to transportation, including mobility, vision, hearing and cognitive impairments. Passenger assistance training also teaches techniques for helping individuals with specific disabilities, and how to use specialized equipment. Example training areas include wheelchair safety and maneuvering, leadi ng the blind, and speaking to passengers with hearing impairments. Sensitiv i ty courses try to give staff an understanding of what transportation may be like for passengers with disabilities. Stress Management Stress management training often is combined with defensive driving and passenger relations training Operating a vehicle according to a schedule can cause stress in drivers as can the physical demands of providing additional assistance. Because stress can affect the mental and physical health of an employee, reduce performance, and is a safety risk in driving, stress management training attempts to minimize these effects Stress management training with a different perspective is also applicable for any employee Substance Abuse Awareness Substance abuse awareness training communicates laws and regulations, company policy, and drug-testing procedures. It also teaches abou t the effects on health, and specifically effects on driv in g and job performance, of various classes of drugs includ in g types of illegal drugs, prescription and over-the-counter medicines, and alcohol. This training often introduces a company's employee assistance program, if it has one. Ill 58

PAGE 97

Part III: Evaluation Modules (Training) Map Reading Map reading training, usually brief, ensures that drivers are able to use a map index to fmd specific locations on a map, identify the shortest route between points, understand all symbols and standards, identify connecting intersections, work with compass directions, fmd alternate routes, and correctly estimate travel time. Many transportation operators find that map reading skills alone can greatly improve productivity, reduce service delays, and decrease response time. Radio Usage Radio usage training ensures that all staff who use the portable, base station, and vehicle mounted two-way radios are proficient in using such as squelch, monitoring, and channel selection. This training also ensures that staff are familiar with the company's radio communications protocol (how radio conversations initiated and concluded), rules (what may be discussed on the radio, length of transmissions), and codes (10-codes and jargon for frequently used phrases). Establishing protocol rules, and codes ensures that radio communications are clear and concise. Scheduling/Dispatching Scheduling/dispatching training seeks to increase the efficiency and effective ness of the process of trip reservations, scheduling, and dispatching. It generally addresses call-intake; handling immediate deferred, and subscription service requests; the role of reservations and dispatchers; demand queues; scheduling considerations; trip tickets; monitoring driver locations and pick-ups ; wait times; insertions; selecting vehicles; record keeping; and other functions. ASE Certifications The National Institute for Automotive Service Excellence (ASE) offers a number of tests that certify the skills of automotive technicians in a variety of maintenance areas. These measure basic techni cal knowledge correction or repair knowledge and skill, and testing and d iagnostic knowledge and skill. The Institute offers these tests as a voluntary certification program, but the certifications are a widely recognized means ofprovinglhe skills of mechanics. ASE testing and certification also is available for medium and heavy-duty vehicles. Because mechanics can be tested in multiple skills, mechanics can have a variety of certifications. III 59

PAGE 98

Pan l/1: Evaluation Modules (Training) Completing the Worksheets Worksheets lla and llb on the following pages should be used to measure the quality of coordinated community transportation as a function of training. This evaluation process will identify the skills of staff, and the types and amount of training required for the coordinated system as whole, for the community transportation coordinator, and for the individual transportation providers. This same information can also be compared to a peer ere. On Worksheet lla, indicate whether the eTe has a training standard that applies to the coordinated system as a whole. If so, the worksheet column labeled "Sys. Std.'' (for "system standard") should be completed as each question applies to the training standard. This will allow the reviewers to compare each operator to the standard. Identify the organizations to be included. For consistency, the organizations being evaluated should be the same as those identified on Worksheet 4 The first column refers to the standard for the coorctinated system. The organizations included on this worksheet should all be transportation providers, either included in the coordinated system, or being considered for inclusion. If the ere is a transportation provider, the ere should be examined separately by listing the ere transportation as a separate operator. Indicate each of the operators being examined across the top of the table at the beginning of each page. Additional copies of Worksheet lla may be made for additional transportation providers It also would be useful to include information on a peer ere as an "operator" for the sake of comparison. If information is available or training has changed, be sure to include information on the system standard for the previous reporting/evaluation period in one column, so improvements can be noted. Answer each question as it pertains to each operator. It probably will be easiest to address the questions for one organization at a time Questions about specific types of training ask for a "yes" or "no" response, or the total number hours of training in that area. The worksheet may be completed with "yes" or "no" an s wers in a quick evaluation; however, the evaluation will be much more useful if specific amounts of training, measured in hours, are reported instead. Training often is conducted as one large process; if so, ask those in charge of training to estimate the amount of time, in hours, devoted to each subject area, without any overlap in total hours. Questions about mechanics can be applied to tlie organization's own mechanics or those used for maintenance services. 111-60

PAGE 99

Parr III: Evaluation Modules (!'raining) Where the box provided is not large enough, feel free to enter a nwnber and circle it, providing a corresponding footnote at the end of the worksheet. Interpreting the Worksheets First, look at each organization individually. The response to each question will identify areas in which training might be improved. A "no" or "0 hours" response does not necessarily indicate a deficiency, but may indicate an area of training that the coordinator might want to pursue. If all the questions in a subsection of the table are "no" or "0," the area should be targeted for improvement. Compare each operator to any minimwn standards established for the coordinated system. Using Worksheet llb, list separately those operators that do not meet the minimum standard and those that exceed the training standards. Compare each operator, including the coordinator, to its peers. How do the operators measure up against each other? Against the CTC? How does the coordinated system compare to a peer CTC? Should standards be raised? If any of the transportation providers evalUated are being considered for incorporation into the coordinated syste_m under a new contract, identify whether the provider's proposed training program meets the existing training standards. If not, the provider's program should be improved before being included. Using the table provided, indicate which organizations are assets in each area. These examinations also should identify areas that the coordinator should pursue in order to improve quality through training. Briefly describe potential improvements. In Part IV of the evaluation workbook, information will be drawn from this worksheet to compare quality to other factors. III -61

PAGE 100

WORKSHEET lla EVALUATION AND COMPARISON OF TRAINING AS A MEASURE OF QUALITY (Page 1 of2) Read discussion beginning on page Ill -54 and instructions beginning on page Ill 60. Does the Community Transportat ion Coord i nator have m i nimum training standards that apply to the coordinated system as a whole? 0 Yes 0 No (If yes, list the training standald in the first column (Sys. Std.) below.) CTCor Question Sys. -Std. 0 : ' ,;.%. Dr1_vers ---" \<;-.;-y AI; 1. Which special license are drivers requ i red to have? 2. How many hours ot trai n ing does each driver r eceive in h i s/her first year of serv i ce? (total hours) 3. Is driver train i ng provided in the following areas? (total hours in first year for each driver or YIN) a. Defensive driving? b. Emerge ncy and accident procedures? c. Passenger relations? d. Passenger assistance techniques? e Stress management? f. Substance abuse awareness? g Map readi ng? h Radio usage? i. Vehicle operation? j. Other driver train i ng? 4. Approximate ly how much of first-year driver % % % % training is complete d before t h e driver beg i ns t r ansporti ng passengers? 5. Frequency of drive r re fre she r t raining? (YIN, post accident only, annually, quarterly, etc) "' 0 . ; ; ; : ... . . : : q < 0 0 . 6 a. Ale mecha n ics ASE cert i fied? b. How many different types of ASE certifications does the ma i nte nance staff hold? c Other ma i ntenance certifications? ---0 ;. .)'$% % ' r A,.' . Reporti ng Period : ___ __ __ Preparer's Initia ls: ______ Date: ____ III -62

PAGE 101

WORKSHEET lla EVAL UATION AN D COMPARISON OF TRAINI N G AS A MEASURE OF QUALITY (Page 2 of2) Q ue stio n Sys. = -Std. .. ._.:_:. :':,. ..... :, .... ,--... : _-: .. __ ,_ ,,. -. 7 Are reception ists and reservationist s trained in public r elat i ons In additio n t o call I 8. Have schedu l ers a n d dispa t che r s attended a or I 9 Which adm i ni strat ive posit i ons are eli gible f or profess i ona l dev el opment (semi n ars, confe r ences, often? a b. O the r c Supervisors d Pla nners e O ther admin i st rat i ve positio n s 7., . '"''H . 10. What proport ion of emp l oyees had p r io r e x p erience i n a transportationrela ted field? 11. Average y e a r s of ex p erie n ce : a Driv e rs? b Schedu l ers and c. Managers? 12 Turnover rat e : (no. employees r e placed I no. a D r ivers? % % % % % % b Sch edu l ers res ervat ioni sts? % % % % % % c. % % % % % % Rep orti ng P eri od : _______ Pre p are ( s I ni t ials : ______ Da t e : ____ lii -63

PAGE 102

WORKSHEET llb CONCLUSIONS REGARDING TRAINING AS A MEASURE OF QUALITY (Page 1 of2) Re ad discussion beginning on page Ill -54 and instructions beginning on page Ill -60. 1 List any transportation operator whose training meets or exceeds all of the training standards of the local CTC. 2. L i st any transportation operator whose training does not meet all of the tra i ning standards of the local CTC 3. Does the training program of transportation providers bei ng considered for inclusion in the coordinated system meet the minimum standards of the CTC? 0 Yes 0 No 0 No new operators being considered 4. What specific improvements in training are needed for the system? Reporti n g Period: _____ _ Preparer's Initials: ___ ___ Date : __ __ 111-64

PAGE 103

WORKSHEET llh CONCLUSIONS REGARDING TRAINING AS A MEASURE OF QUAL ITY (Page 2 of2) 5. Complete the following questions rating each organization with a cons i stent scale(+/-, 1 10, or A-F) and i ndicat i ng whether the organization i s doing well in that aspect or has a potential prob l em and whether the CTC is doi ng better or worse than before Two questions require a quantitative response. Comp l ete one column for the CTC and each opera1or. -----Eva luation A r ea CTC Does the CTC h ave m i nimum train i ng standards? Does the operator meet the training standards? How many hours of first-year d river train i ng are provided in total? How many subject areas listed on the worksheet are drivers trained in? Drivers: Is t he CTC improving d r iver training for the coordinated system? Which organizations are an asset i n terms of driver t raining (meets or i s above the standard or significantly better t h an the others)? Maintenance staff: Is the CTC i mprov i ng ma i ntena n ce skills for t h e coord i nated sys tem ? Which organ i zat i o n s are an asset in terms of ma i ntena n ce s k ills and training? Other staff: Is the CTC improving staff training for th e coordina t ed system? Which organizations a r e an asset in term s of staff training? All staff: Is t h e experience of the staff improving? Are train i ng standards satisfactory? Reporting Period: _______ Initials: ______ Date: ____ Ill 65

PAGE 104

Part 111: Evaluation Modules (Availability) Worksheetl2 Availability of Service Because of the great demand for TD transportation and the challenging cost of providing service, maximizing the availability of service is one of the most fundamental efforts undertaken by specialized transportation The "Florida FiveYear Transportation Disadvantaged Plan" establishes as a goal for l ocal community transportation coordinators: Ensure the availability of service to the transportation disadvantaged. This goal is supported by three specific objectives for community transportation coordinators: (I) provide services to meet the demand for sponsored trips; (2) provide services to meet the demand for non-sponsored trips; and (3) improve passenger awareness of TO transportation services. Several important measures have been suggested for measuring availability:8 number of program trips provided (and percent of demand met for program trips); number of general trips provided (and percent of demand met for general trips); vetUcle miles per TO capita; number of new TO passengers ; spending on public information; vetUcles per I 00,000 persons; and percentage of trip denials. These and several closely related measures help to quantify the total amount of unmet demand, potential service availability, and distribution of information. Availability also can be examined from the perspective of users. To consumers ;wailability is a function of their ability to arrange for service when needed. These and other useful ways of examining the availability of service are discussed below; several of these and other measures are examined in the worksheets that accompany this module. 8 The first five measures, recommended in the Florida Year Transponation Disadvantaged Plan, are revised here 10 count program and general trips rather than sponsored and non-sponsored trips. The last two measures were suggested in the March 10, 1993, TO Commission internal memorandum DRAFT CTC Service P l an III -66

PAGE 105

Part III: Evaluation Modules (Availability) Measures of Availability The availability of transportation must be examined in relation to the amount of service required which is rarely easy to determine. Evaluating availability may require information that is not always on hand, but is nonetheless necessary to effectively plan transportation services. Make use of other planning and research efforts as needed to evaluate availability. Several influences and measures that reflect avai labilit y are discussed below. TD Population and Demand In working to ensure service availability evety transportation coordinator should have an estimate of the total transportation disadvantaged population and the estimated total demand for trips for the service area (e.g. the county). These figures form an important basis from which to evaluate the service actually provided. In Florida's Coordinated Transportation System, the population eligible for program-sponsored trips is larger than the popula tion eligible for trips funded by the TD Trust Fund Separate population and demand estimates are needed for each of these categories. The Florida FiveYear Transportation Disadvantaged Plan: Technical Memoranda No. 3 and No. 4 discusses TD population categories and demand forecasts in great detail, and provides preliminary estimates. Methodology Guidelines for Forecasting TD Transportation Demand at the County LeveV provides a more detailed methodology that should be employed at the local level. TD Categorie s I and II TD Category I refers to the total popuhition of persons who are eligible for trips through the coordinated transportation program (i.e. persons who have a disability are elderly, children at-risk, and/or are low income). Category II is a subset of Category I and includes those persons who because of their disability, income status, or age, are unable to transport themselves or to purchase transportation. Persons who are included in Category II are eligible for trips funded through the TO Trust Fund. CTCs' five-year TD plans normaUy include estimates of Category I and II populations. 9 Complete citation in Appendix A. III 67

PAGE 106

Part lll: Evaluation Modules (Availability) Service Provided Each trip provided within the coordinated system should be identified and counted as "program" or "general."' Program v. General Trips Trips provided in connection with a particular program should be counted separately from general trips. Because demand also is estimated separately for program and general trips, it is possible for the number of trips actually provided to be compared to estimated demand for each category. Since virtually all program trips are sponsored, all demand for "program" trips should be able to be met. A primary objective for all CTCs is meeting as much demand as possible, although the supply of general trips is dependant on funding from the TD Trust Fund established for non-sponsored trips and other sources Trip Denials It is likely that any coordinated system with less capacity than demand will have to deny some trips. These trip denials should be tracked in order to evaluate the portion of requested trips actually provided. Note that the number of trips denied is normally less than the actual unmet demand. High numbers of trips usually represent a lot more unmet demand, since users may give up requesting trips as capacity becomes constrained. The trips denied often represent more than one trip. Reach of the Coordinated Program Not only bow much service is provided but how many eligible people there are and bow much service per person is provided should be examined. Registrants and Users Providing as many trips as possible is only one aspect of availability. TO transportation also should be available to as many eligible individuals as need service. In coordinated systems that require a registration or certification process, or keep unique records for each person using coordinated transportation services, the total number of persons that have made use of the service can be compared to the eligible TO population. The current number of users also can be compared to the number of users ai a fixed point in the past, such as at the end of the Ill -68

PAGE 107

Part Ill: Evaluation Modules (Availability) last fiscaVreporting year, to determine the number of new TO passengers that the CTC has reached. Trips per TD Capita As a measure of how much service is being provided to the total TO population, it is helpful to examine the numbe r of trips provided per eligible individual, that is trips per TO capita. This number may be surprisingly lo w because demand estimates attempt to identify all demand (unmet and latent). Trips per User It is also useful for the number of trips actually provided to be compared to the number of TD eligible persons who have used coordinated transportation or registered to use it. Each of those using the system may or may not be provided many trips Note that if a high percentage of the eligible population has been registered or has taken at least one coordinated trip, the average number of trips per user will be even lower. Public Info rmation Transportation service is available only to those who know about it and know how to access service. Improving passenger awareness of TD transportation service is an objective in support of availability for each CTC. Efforts in this area may be reflected in the budget expended on public information and in whether or not the coordinated system has coordinated public information. The primary objective of public information efforts is to make sure that'necessary information about the service is readily available to those that need it. It does not need to include marketing efforts designed to attract more riders. Access From the point of view of the use r availability is a function of how easily service is accessed. Access is reflected in at least two measure s : Some systems r equire proper certification to begin using services. The time it takes to register and begin using TD transportation services reflects its availability. Similarly, the lead time for a demand-response trip reflects availability for the user. Thus, the days and hours during which service is available and during which reservations may be placed also should be examined. III 69

PAGE 108

Part III: Evaluation Modules (Availability) Capacity The difference between supply and demand for trips is a function of capacity. The total capacity of a system may be examined in several ways. Requested Pick-up Times This is the time a caller initially asks to be picked up for a trip. In many systems scheduling or capacity constraints may require some negotiation to agree upon a scheduled pick-up time. The difference between the requested pick-up time and scheduled pick-up time reflects those capacity limitations and, therefore, is a good measure of availability if the reservations process can include a means of recording requested pick-up times. Vehicles and Drivers The total capacity of any transportation system, assuming sufficient drivers are available, also is a function of how many vehicles there are, especially in relation to the eligible TD population. Restrictions Restrictions put in place to manage demand also reflect availability. These restrictions include waiting lists caps on the number C)f trips provided to eligible users, and limitations based on trip purpose. Trip priorities are a way to manage demand. Excess capacity, if any, should be noted as should any unused revenue. Completing the Worksheets Worksheets 12a and 12b, beginning on page III 73 will assist the reviewer in evaluating success of the community transportation coordinator in ensuring availability of transportation service to the TD population. Worksheet 12a can be completed by any designated reviewer with assistance from the community transportation coordinator. First, establish the reporting period to be examined. In most cases this time period should correspond to the reporting period examined in the other modules The time period that the annual operating report covers will be most convenient. In fut ure applications of the evaluation model previous data also should be included for comparison. III -70

PAGE 109

Part Ill: Evaluation Modules (Availability) Some of the figures needed for this form are the same as those needed for the annual operating reports required by the TD Conunission. Answer each question on the worksheet, drawing from compiled operating records and the annual operating report as necessary. In some cases, the information will not be available. These instances may suggest possible improvements in accountability and record keeping, or necessary research and planning. The preceding discussion provides additional information which should be read thoroughly. Key instructions for each section of the worksheet are provided below. Demand Refer to local TD research and planning efforts for estimates ofTD populations and demand for the most recent period. If no estimates are available, apply the Methodology Guidelines for Forecasting TD Transportation Demand at the County Level or a similar method to estimate populations and demand. Estimates are also available in the Florida Five-Year Transportation Disadvantaged Plan. Program and general trip demand is examined here because these nwnbers relate to the TO Category I and Category II populations. In most cases, these are the same as sponsored and non-sponsored trips, respectively. If all program trips must be sponsored by the program requesting them, then the demand for program trips may be simply the number of trips programs request and sponsor. If, however, the coordinator contributes to tlie cost of trips for specific programs, then the demand for program trips is the number of trips requested by programs whether they all could be provided or not. Service Provided Refer to the preceding discussion of program and general trips and demand. The numbers recorded here are simply total measures of service, which can be compared to previous years for the same system only to get a general picture of levels of service. These numbers show the nwnber of trips provided, registered users, and miles and hours vehicles were in use providing service. Revenue miles and hours include only "in-service" miles and time. These nwnbers also are used to calculate ratios that measure availability, below. Supply v. Demand U se the information entered under "Demand" and "Service Provided" on Worksheet 12a to calcu lat e the percent of demand met as indicated. The results may be quite low; however, an objective may be to increase availability while managing demand. Ill -71

PAGE 110

Part Ill: Evaluation Modules (Availability) Outreach These measures of availability are calculated, as indicated on the Worksheet 124, from the numbers provided above based primarily on the number of registrants and the TO Category II population. These ratios should nonnally be used to compare from one year to the next, as they do not actually reflect a level of service for each person. Public Information and Access to Service These measures give a general picture of the degree to which the system makes itself available to its intended population. Year-to-year comparisons will be important only if significant changes occur. Capacity This section provides both ratios for making year-to-year comparisons and measures that give a general picture of the capacity of the coordinated system. The ratios are based on the TD Category II population. Estimated excess capacity may need to be calculated separately. It should reflect the total number of additional trips that could be provided now (e.g. within a montb) if additional demand was sponsored Unused revenue normally indicates only that demand and supply are out of step, rarely tbat supply is too great or demand is too low. Note that high measures of availability may not support high measures of efficiency. Hours and Days of Service ; Ability to Place Reservations These questions shou l d be self-explanatory. Th e answers present a general picture of availability from the users perspective and provide a means of measuring improvemenL Interpreting the Worksheets The instructions for completing these worksheets also provide insight into interpreting them. They indicate, for example which figures give a general picture and which should be compared year-to-year. Understanding th e numbers also helps the evaluator to interpret tbe worksheets. Worksheet 12b may be u se d to draw conclusions based on Worksheet 12a, and should be completed by a member or committee of the local coordinating board. Data from a similar previous reporting period can be used for comparison in order to draw conclusions about improvements Current availability measures can help the both the CTC and LCB establish specific standards or task s for the following year. III-72

PAGE 111

WORKSHEET 12a EVALUATION OF AVAILABILITY (Page I of2) Read discussion beginning on page Ill 66 and instructions beginning on page Ill 70. Answer each question in regard to the total coordinated service. Measure 2. TO Category II population 3. Estimated demand for 3 +line 7. General 8. Tota l nu m ber of trips provided (line 6 +line 9. Reven u e miles 10. hours 11. Registrants (numbe r of i nd iv idual users) Date of last 12. Program dema n d met % 13. Ge n eral demand met (line 7 I line % % 14. Perce n t of estimated demand met (l i ne 8 I line % % 15. Trip denials 16. Percent of met line 8 I (line 8 + line 15) % % 17. Ratio of 18 General 1 9 Revenue miles per user (line 9 I line 20. Revenue miles II 9 I line 2) Reporting Period: ______ P r eparer's ______ Date : ____ Ill 73

PAGE 112

WORKSHEET 1 2a EVALUATION OF AYAlLABlLlTY (Page 2 of2) Measure E'laluatlon "'-vlouo Period Period fi;b!!c .... .. n--i,. .?ii<>:d. /,ii:rf, :.:; .!1i:;:' f . 21a Pub lic i nfonnation expenses (i n cl u ding inkind revenue/expenses) $ $ 21b. List public Information efforts (e g., b roChures, newspaper ads) 22. Normal registration dela y (number of days if any) J .r ' c apacity : d' __ .':' ' l-'< 23. Fleet used in coordinated TO transportation (number of vehicles) 24. Vehicles per 100.000 TO II capita (line 241/ine 2 x 100 000) 25. Drivers per 100 000 TO II capita (Coordinated FTE's I lin& 2 x 100,000) 26. Are trip priorities or l i mi ta tio n s used? (YIN) 27. Size of wait l ist (if any) 28. Has the coord i nated system met all demand for sponsored trips? (YIN) 29. Estimated excess capacity (see instructions) Hl:n irsran"'.i1)ay.1ot . .. { I : M, ; _,. i"tJI' b "f:;-I :1.1 '-SJi' 'Tuj -;; W ... 30. Hours per day duri n g which :" coordinated service is available F > ,_ -!(''. 31' Ho u rs per week coordinated service is availabl e (sum of Mon.-sun. ) : ' ., I Ablilty to Place Res'er..ations' .xu Th. F ... Sa 'r' ':(" 32. Number of h ours per day during ''.# ... <';> ' .. .-.:;{ whiCh reservations may be made ,, 33 Number of h ours per week during whiCh reservations can be made (su m of Mon. Sun.) 34. What is the minimum requ ired not i ce for reservations, per n ormal policy (number of hours or days)? a Ambulatory b. Non-ambu la tory c. St retc her 35. How far in advance c an r eserva t ion s be p l aced? (number of days) R epo rting Pe riod: ______ Preparer s Initials: ______ Date: ____ Ill -74

PAGE 113

WORKSHEET 12b CONCLUSIONS REGARDING AVAILABILITY j : Read discussion beginning on page Ill 66 and instructions beginning on page Ill 70. Answer each question in regard to the total coordinated service 1 For each measure of availab i lity above, compare the figures for the evaluation period to that of the previous period If figures are not available for the previous period, skip to Question 5. 2. Which aspects of ava i lab i lity have Improved? If any descr i be 3. Has availability dec re ased in any aspectsince the previous period? 4. If availability has decreased to what extent was this decline within the control of the community transportat io n coordinator? 5. We r e any aspec ts of avai l ab i lity previously targeted f o r improvement? 0 Yes 0 No I f yes, has availability improved i n those aspects? 6. Are standards for availability estab l ished? 0 Yes 0 No I f so, are they met ? 7 Which aspects should be targeted for improvement? Reporting Period: ______ Preparer's Initials : ______ Date: ____ lii -75

PAGE 114

Part Ill: Evaluation Modules (Funding) Worksheets 13 and 14 Funding and Accountability Each community transportation coordinator is charged with a great deal of responsibility in coordinating all TD trips and in administering TD funds properly. The coordinator must use public funding wisely and be accountable for its actions and responsibilities. The "Florida Fiveyear Transp ortation Disadvantaged Plan" establishes some specific goals for the coordinator including: Ensure necessary funding to support the TD program; and Ensure program accountability. Several coordinator objectives are offered in support of these goals, including: (I) increase farebox revenue; (2) increase local funds; (3) submit unifonn, accurate, and timely data and contracts; and ( 4) collect, compile report, and maintain data necessary for evaluation of local coordinator system. These objectives are monitored by closely related measures: percent farebox revenue is of operating expense; percent local funds are of total operating revenue ; and submittal of on-time AOR and MOA. In tenns of funding and accountability, the respons ibilities of the ere are fairly straightforward Accountability of Operators Operators will not be in a position to control funding, but the ir accountability can be assessed and predicted. Before bringing on a transportation provider for inclusion in the coordinated system, the coordinator should provide a list to the operator of the information that would need to be reported regularly. The operator should indica te exactly what measures it is able and willing to report regularly and sign off on it. (([ 76

PAGE 115

Part Ill: Evaluation Modules (Funding) Completing the Worksheets Complete Worksheet 13 to evaluate how well the CTC ensures necessary funding to support the TD program. The worksheet uses information that also is reported in the AOR. Use the same definitions provided in the TD Commission's Instructions for Completing of AOR and the Rural Transportation Accounting Model. Look for Farebox Revenue, Local Revenue, and Number of Sources to increase. Use the periods identified on Worksheet I. Complete Worksheet 14(a, b & c) to evaluate the CTC's accountability. Another way to get extremely detailed information about the accountability of the CTC would be through a financial audit. For purposes of this evaluation, the detailed and costly a udit is not necessary. Worksheet 14, titled "Conclusions Regarding Accountability," should be based on the answers provided on the worksheets and the informed judgment of the reviewer. Interpreting the Worksheets After completing Questions 1-10 on Worksheet 13, compare funding measures between the previous period and evaluation period, and answer questions 11-16 indicating what measures have improved or worsened. Consider how much of this change could be affected by the CTC. Indicate what improvement was expected and what area should be targeted for improvement in the future. After completing Worksheets 14a and/or 14b compare the information in each column and consider the implications of the answers to questions at the bottom of those worksheets. Complete Worksheet 14c, reading each question carefully III -77

PAGE 116

WORKSHEE T 13 E VA L UATION OF FUN D ING Answer each question in regard to the total coordinated saNice. Read discussion beginning on page Ill76 and inslroctions beginning on page Ill -n F undin g Mea s ure Evaluation Previous Perfod Pfllod 1 Total Operat i ng Expenses $ $ 2. Total Revenue $ $ 3. Local Revenue $ $ 4 Farebox Revenue $ $ 5. Local Revenue Percent of Operat i ng Expe n ses (line 3 I line 1) % % 6 Farebox Revenue Pe r cent of Operati n g Expenses (line 4 I line 1) % % 7. Unused Revenue ( i f any) $ $ 8 Number o f Sources of Revenue 9. Does documentation ex ist that ind i cates current tares are based on fu lly allocated costs? (YIN) 1 0 Has funding been spen t according t o any established schedules lor t h e eva l uation period? (YIN) For each measure of funding above compare t h e figures for the current or most recent period to that of t h e prev i ous per i od. I f fi gures for t h e previous period are not ava ilable. skip to Quest ion 14. 11. I n Which areas h as fundi ng improved? 12. Has fu n ding worsened i n any areas since t h e previous per i od? 13 If fu nding has worsened to w hat extent was t h i s w it h i n the control of the commu nity transportation coordinator? 14. Were additiona l fund i ng sources and amounts targeted for i m p rovement? DYes 0 No 15. Have funding leve l s improved since the previous reporting period? 0 Yes 0 No 16 Which a r eas shou l d be targeted for imp r oveme n t? Report i ng Period: ______ Preparer's Ini t i als : ______ Date: ____ IIi 78

PAGE 117

WORKSHEET 14a EVA LUA T ION OF ACCOUN T A B ILITY Answer each que .stion in regard to the total coordinated service Read discussion beginning on page Ill 76 and instructions on page Ill 77. Accountability M easu r e Evaluation Previous ai 6 Period .. 1 Were t h e submitted on t i me? a Annual l I TnV/ b. Memorandum of Agreement I (YIN) c C T C SeNice Plan (with d Grant Applications to TO Trust Fun d e. All oth e r grant app l icat i ons (percentage submitted on t ime): N u mber on t ime I Total Number % % = ;,:,"'"" : 2 Does t h e coo r dinator have a computerize d management information in (YIN) 3 Was the most recent A n nua l Operating Report subm i tted comp l ete and accurate? (YIN) 4 If other eva l uat i on modu l es h ave been used, was all needed informa t ion avai l able? 1 r m 5 What seNice statis ti cs are kep t reg u larly : (./ all that apply) 0 comp l eted tri p s 0 early or un scheduled tri ps 0 vehic l e hours 0 trips by fun di n g so u rce 0 no-s h ows 0 reven u e miles 0 self-paid/co-pa i d trips 0 r egistrants 0 veh i cle mi l es 0 TO Trust Fund trips 0 wa i t l ist size 0 revenue hou r s 0 program t rips 0 call hold-time 0 Heel inventories 0 tri p s by program 0 pho n e calls by type 0 trips by vehicle typ e 0 sponsored trips 0 compla i nts 0 i nciden t s 0 general trips 0 com p la i nt response time 0 accidents 0 trips by trip purpose 0 co m p l aints resolved 0 preventable accidents 0 trip denia l s 0 l ate ride comp laints 0 breakdow n s o r roadcalls 0 l ate t rips 0 compliments 0 statis t ics by transportation 0 missed trips 0 em p loyees t raini n g hours prov ider For each trip: 0 requested pick u p time 0 actua l boarding time 0 actual drop-<>ff odome t er 0 sch ed u led piCkup t i me 0 actua l boa r ding odometer 0 response time for w i ll-call t rips 0 driver arrival time 0 act u a l d r op-off ti me 6. Other records Total 5 : Reporti n g Pe riod : _______ Preparer's Ini t i a l s: ______ Date: ____ III 79

PAGE 118

WORKSHEET 14b ACCOUNTABILITY OF OPERATORS Answer each question in regard to the total coordinated seNice. Read discussion beginning on page Ill 76 and instructions beginning on page Ill 77. ------Accountability Measure 1 Does the operator submit data reports required by the contract? 2 Are required data and reports submitted on time? 3. Are data reliable and accurate the first time? 4. Is the operator able to provide all t h e data that o ther operators provide? 5. Does the operator provide the CTC with accurate Information r egarding ETAs and pick-up t i mes for lat e rides? 6. Summarize any other accountability prob lems with operators. 7 In which areas do particular operators need t o i mprove? Reporting Period: ______ Init ials : ___ ___ Date: ____ lll -80

PAGE 119

WORKSHEET 14c CONCLUSIONS REGARDING ACCOUNTABILITY Review Worksheet 14a. Read discussion beginning on page Ill 76 and instructions beginning on page Ill 7 7. 1. Does the CTC submit t ime ly data and contracts? 2. Does the CTC submit complete and accurate data and contracts? 3. Does the CTC collect, compile, report and maintain data necessary for the evaluation of the local coordinated system? 4. Does the CTC collect, compile, report. and maintain data necessary for planning? For each measure of accountability in Worksheet 14b, compare the answers for the current or most recent pe riod to that of the previous period. If answers for the previous period are not available, skip to Q u estion 8. 5. In which areas has accountability improved? 6 Were any areas of accountability previously targeted for imp rovement? If yes, has accountability improved In those areas? 7. Has accountability slipped in any areas since the previous period? 8. Does lack of information cause any problems in planning, reporting, or evaluation? areas should be targeted for improvement! 0 Yes 0 No o Yes 0 No 0 Yes 0 No If so which 9 Based on review of Worksheet 14a a ny other evaluation models completed, and discussion with the CTC and/or reviewer. what additional service statistics should be kept? Reporting Period: ______ Preparers Initia ls: ______ Date: ____ III 81

PAGE 120

Part Ill: Evaluation Modules (Peer Comparison) Worksheet IS Comparison of CTC to Peers Each community transportation coordinator belongs to several peer groups, based on different criteria. For Florida CTCs, the most important distinguishing criteria are operating environment, organization type, network type, and system size. These important characteristics of the CTC are identified using Worksheet 2 in the preparation for the evaluation. Organizations are compared to their peer groups to see how they are doing in relation to other organizations like them. Within Florida, a CTC can compare itself to other CTCs of the same operating environment, organization type, network type and system size. Other peer groups could be consid ered; however statewide performance measures are compiled for these peer groups This module of the evaluation is conducted last because it looks not at the CTC itself, but at the CTC as it relates to others. Also, the reporting period used here may be different from that used for the rest of the model. Finally, this peer comparison will draw on nine performance measures that reflect a variety of CTC goals. As such it is a quick overview of performance. Completing the Worksheets Worksheet l5(a & h) assists in comparing the CTC to each of these peer groups. These comparisons use median (midpoint) statistics, which are available in the annual SOR. Because the comparison is based on information compiled from many CTCs, the data used for the CTC in this worksheet mus t be for the same reponing period as the most recent SOR. Worksheet 15a is used to record performance measures for the CTC and its several peer groups. Questions I through 4 determine which peer groups should be used for the comparisons. The type of CTC used here should be the same as identified on Worksheet 2. Column A is used to record performance measures for the CTC. The performance measures are computed from information submitted by CTCs and can be found in the SOR's Appendix A, "Detailed System Characteristics," organized by CTC. In columns B D record the median performance measures for each of the CTCs' peer groups. These performance statis tics are shown in tables in the SOR. Use the following tables: Ill 82

PAGE 121

Part Ill: Evaluation Modules (Peer Comparison) Performance Measures by System Size; Performance Measures by Operating Envirorunent and Organization Type; and Performanc e Measures by Operating Envirorunent and Network Type. In the FY 1992193 SOR, these are Tables 14, 16, and 17, and are found just before Appendix A. Be sure to use the tables for the correct peer group and correct year These tables also are duplicated for FY 1992193 h1 Appendix D of this workbook. Interpreting the Worksheets Use Worksheet lSb to draw conclus i ons based on Worksheet lSa. Review Worksheet lSa to see what measures the ere appears to do well in compared to its peers. Place a .r next to the median peer measures on Worksheet lSa that the CTC equals or surpasses using Worksheet 1Sb as a guide for which measure should be high and which low. Count and record on Worksheet lSb how many peer group s the CrC equals or exceeds performance. Also count the number of times the CTC equals or exceeds each peer group and record these numbers on Worksheet lSb. Consider these totals. This comparison must be considered very carefully before drawing conclusions. A numb e r of publications-some are listed in Appendix A describe the use of performance measures and their mean i ng. The measures collected here examine several different aspects of service. These must be considered together because e x ceptio n al measures in one area could be due to an inefficiency that would show up as a poor score in another area. Furthermore, the performance scores to which the CTC is being compared is the median for its peer group. By definition, approximately 50 percent of all CTCs in the peer group will be below the median and approximately 50 percent will be above Therefore, only half of the CTCs in a peer group will equal or exceed the median for each measure. Also, apparently poor performance in one performance measure may or may not be w i thin the CTC s control. How a CTC compares to a peer group may be different from how it compares to all CTCs. Indicate on Worksheet lSb whic h areas should be targeted for exploration and/or improvement. Comparing performance measures is only one form of evaluating performance and is limited to those aspects included i n the analysis; the reviewer should exercise considerable caution in III 83

PAGE 122

Part Ill: Evaluation Modules (Peer Comparison) interpreting the results. This worksheet should not be used without additional evaluation to clarify findings. In addition, several aspects of quality of service are not measured by these performance measures. These measures do not include cleanliness and comfort, operator courtesy, passenger assistance, on-time perfonnance, quality of reserva tion processing, and level of satisfaction with days and hours of operation. In addition to understanding the limits of these performance measures, the reviewers should use cautio n in interpreting the meaning of the various measures. The perfonnance measures do not necessarily provide information regarding which aspects of performance are within control of the agency and which measures are not. A number of factors affected the performance of coordinated transportation, including management/staff local policy decisions, and environment (see list below) Management/Staff Skills & experience Training Leadership Morale Network Type Service quality Local Policy Decisions Land use Local support Service lev els Fare policy Fixed-route availability Organization Type Environment Density Land use patterns Congestion Geography Demographics Perfonnance measures are a useful and important tool in monitoring and improving coordinator performance. However, it should be recognized that the results of peer comparisons are only one small part of fully understanding the performance of coordinators. The issues identified as a result of the analysis provide only a starting point for examining areas which may be improved or already successful. The preceding evaluation modules evaluate various aspects of service more carefully. This peer comparison necessarily depends on the classifications of CTCs established in the instructions for the annual operating reports and compiled in the SOR. Because each CTC is unique, these classificatio ns are necessarily broad. lil -84

PAGE 123

WORKSHEET 15a COMPARI SON OF CI'C TO PEE RS BY STRUCTURE Reed discussion and instructions beginning on page Ill 82. 1 Record the system slze dass illcalion based on riderstlip ( from 1 7) at t tle top of Column 6 (See WOrkStleet 2. Question 6 for sii!e class ification i nformation.) 2 Record the operating environment (Urban or Rural ) and the organization type (Private for-Profit Private non Profit, or Transit Agency) at the top of Column C 3 Record the operating envi ronme nt (Urban or R u ral) and the network type ( Sole P rovider. Partia l, or Complete Brokerage) at th e top of Column 0 4 Using median Statew ide Operations Report (S OR ) data enter performance statistics for the CTC i n Column A, and for peer CTCs i n Col u mns B 0 Column A by PHIGnxlp.,;;; CoiYmn B Column C Cotumn D o Urban a Ufbn Perfonnance Measure a Rural o Rural Loca l sy ...... Organizatii;Jn Notwotl< ere SlO 1 Vehicle M iles per TO Cap ita 2. Revenue M iles per T O Capita 3. Passenger Trips per Vehicle M ile-Total 4 Passenger Trips per Revenue MileT olal 5 Operating Expense per $ $ $ $ Vehicle M i le 6 Ope ra ting Expense per $ $ $ $ Revenue M il e 7 Operatin g Expense per $ $ $ $ Passe nge r TriPTotal 8 Accidents per 100,00 0 Vehicle M iles 9 Vehicle Miles Between Roadca n s tO. Loca l Revenue Percent of % % % % Operating E x pense Be sure to complete Worksheet 15b prior to interpr eting W orl
PAGE 124

WORKSHEET 15b CONCLUSIONS REGARDING COMPARISON OF CTC TO PEERS Read discussion and instructions beginning on page Ill 82 For each performance measure s h own on Worksheet 15a, i nd icate which of its t h ree peer CTC group medians it equals o r surpasses in performance by plac i ng a ./ i n the cell next to the measure. Indicate the number of./ marks (Q-3) for the measures below: 1. Vehicle Miles per TD Capita {higher is better) 2. Revenue Miles per TO Capita (higher is bettet) 3. Passenger Trips per Vehicle MileTotal {higher is better) 4. Passenger Trips per Revenue Mile Total {higher is better) 5 Operating Expense per Vehicle M i le {lower Is better} 6. Operating Expense per Revenue Mile {lower is better) 7 Operating Expense per Passe n ger TriJ>-Paratra nsit (lower Is better) 8. Accidents per 100,000 Vehicle Miles {lower is better) 9 Vehicle Miles between Roadcalls {higher is better) 1 0. Local Revenue Percent of Operating Expense {higher is better) For each peer group, count t he number of performance measures in which the CTC equals or exceeds the median for its peer group. 11. System Size Peers (0 10) 12. Organizat ional Peers (0.10) 13. Networl< Peers {0.1 0) Consider the above counts, recognizing that where the CTC exceeds a median performance measures means t he CTC is (statistically) perform i n g better t han about hall of t he CTCs in that peer group 14. Which areas should be t argeted as goa l s tor exploration and /or improvement? Read "In terpreting the Worlcsheets on page Ill 83 before d ra wing conclusions. Reporting Period: _______ Prepare(s In it ials: ______ Date : _______ III 86

PAGE 125

PART IV: CONCLUSIONS Worksheets 16-18

PAGE 126

Part IV: Conclusions CONCLUSIONS Overview Part IV of the evaluation workbook helps reviewers form overall conclusions. Many of the conclusions regarding each aspect of service were recorded by the reviewers on the last worksheet of each module. In this section, the reviewers will (I) summarize the most important of those conclusions, and (2) lay out the key findings from each module side by side with conclusions and scores from each of the other modules in order to draw comparisons and consider the total picture of the organizations included in the evaluation. Thi s section also suggests typical final products from the evaluation, which will vary according to the evaluat ion's initial goals Before formulating overall conclusions, tbe evaluation team should review Worksheet I, which established the scope and goals for the evaluation. A clear recollection of the intended purpose and scope of the evaluation will help to form appropriate conclusions. Consider the questions listed below. Who requested/required the evaluatio n ? What was to be accomplished by the evaluation? Were the appropriate people includ e d? Were the intended entities evaluated? Were all intended areas evaluated? Was the appropriate level of effort expended? Did the evaluation respond to the event or situation that triggered it? Was the evaluation completed on time? Some of these questions will be answered explicitly in this section (Worksheet 18). Considering these questions now ensures tha t the evaluation i s sufficiently compl e te to begin forming conclusions. Each review e r also should review the worksheets that he/she completed or assisted with prior to meeting as a team to form conclusions. IV-I

PAGE 127

Part IV: Conclusions Assemble the review team to draw conclusions. Usually, all reviewers who helped complete worksheets should be involved in the drawing of conclusions. Their worksheets and experience in examining each aspect of service will be the basis for the conclusions. The evaluation team will need to decide whether others should be included at this point. At least one person will need to record the conclusions of the team. In many cases, the remarks indicated on these worksheets will be draft conclusions. They may be used to write or present a report on findings, depending on the intended product for the evaluation. The type of conclusions drawn and the format in which they are finally presented will depend on the goals and intended accomplishments ouilined for the evaluation. Once the final worksheets are completed, the conclusions can be formulated into results. Use of the results will vary, depending on the reason for the evaluation. Three examples are described below. Full Evaluation If the CTC has conducted a self-evaluation at the request of the LCB, for example, then the review team and DOPA representative might meet to complete the conclusions worksheets together. The DOPA representative might then use copies of aU worksheets or just the concluding worksheets to draft a report for the LCB evaluation subcommittee. In that case, the report should specifically address the concerns of the LCB. Copies of the worksheets might be attached as an appendix for the official record, but those requesting the evaluation should rely on a narrative report and oral presentation Operators E v aluation If the CTC director has requested an evaluation of all operators and the CTC's own operations to formulate plans for proper allocation of trips following service expansion, then the key points from each area and especially the concluding worksheets should be reviewed with the director so that the strengths and weaknesses of each operator are well understood. The worksheets and their conclusions could be used to help draw up a service plan. Partial Evaluation If one manager at a CTC has used a few modules to evaluate his/her department, the concluding worksheets might be used to develop a memo to the director outlining key findings and recommended changes. IV2

PAGE 128

Part IV: Conclusions Because the goals and intended accomplishments of the evaluation vary, three worksheets are provided in this section. Worksheet 16 should be used in most cases. It provides a general fonnat for swrunarizing the most important findings from each module included in the evaluation. Worksheet 16 lists general fmdings for each module, areas to target for improvement, and areas that need further exploration or additional infonnation. Worksheet 17 is somewhat more detailed. It asks for a general assessment of the organizations in many specific areas in order to compare results across different aspects of service and among organizations. Specific recommendations for improvement will not be drawn out in this worksheet, but the worksheet will identify the strengths and weaknesses of one or more organizations for comparison. Worksheet 18 concludes the evaluation by comparing the results of the evaluation to intended results, indicates where to find relevant documents, and records any recommendations for the next evaluation. IV3

PAGE 129

Part If/: Conclusions Worksheet 16 General Findings Worksheet 16 should be used for most evaluations to summarize general fmdings, to recommend improvements, and to indicate additional infonnation needs. If a very detailed report is being prepared directly from the worksheets in each module, it may be appropriate to skip this worksheet. Completing the Worksheet Each member of the evaluation team who participated in completing worksheets should carefully review those worksheets and be prepared to summarize and explain his/her conclusions for the review team. Any additional individuals that should be involved in the drawing of conclusions should complete these worksheets with the review team. The team should complete Worksheet 16 together, using the worksheets from each module to support their conclusions. One person may be designated to record the groups conclusions. General findings include any of the following: highlights, interesting points, a summary, the highest and lowest numbers, which areas look much the same as in the last evaluation, any facts you want brought forward in final conclusions, results that were or were not as expected, areas or operators that looked good or bad, or answers to conclusion questions on worksheets. Suggestions/recommendations for improvement come from the worksheets and are items that evaluators and the review team think are important areas to target for improvement, specific measures that were problem areas, or ideas resulting from specific questions. Their inclusion does not necessarily mean that each suggestion must be implemented but that they are the team's suggestions. These suggestions may be a basis for discussion along with other results from the evaluation. Under Additional research, investigation. or data required, list any areas that were difficult to measure due to lack of information, or areas that might need further or more detailed evaluation. Worksheet 16 i ncludes one page for each module. Complete only those pages for modules that were included in the evaluation. IV4

PAGE 130

WORKSHEET 16 GENERAL FINDINGS (Page 1 o}8) Read discussion beginning on page IV 1 and, instructions on page IV4. General findings: COMPETITION (WORKSHEET 5) Suggestions/ rec ommendations for improvement: AddiUonal research, Investigation, or data required: Reporting Pe riod: ______ Prepare(s Initia ls: ______ IV5 Date: ____

PAGE 131

WORKSHEET 16 GENERAL FINDINGS (Page 2 o/8) Read discussion beginning on page IV 1 and ins tructions on page IV 4. General findings: COORDINATION (WORKSHEET 6) Suggestions/recommendations for im provement: Additional research Investigation, or data required: Reporting Period: ______ Preparer's Initials: ______ IV6 Date: ____

PAGE 132

WORKSHEET 16 GENERAL FINDINGS (Page] of8) Read discussion beginning on page IV1 and instructions on pag e IV4 General findings: COST EFFECTIVENESS & EFFICIENCY (WORKSHEETS 7-9) Suggestions/recommendations for I mprovement: Addl tlonal research, Inv es tigation, or required: Reporting Period : ______ Prepare(s Initials: ______ IV7 Date: ____

PAGE 133

WORKSHEET 16 GENERAL FINDINGS (Page 4 of8) Read discussion beginning on page IV1 and i nstructions on page IV4 General findings: ASPECTS OF QUALITY (WORKSHEET 10) Suggesllonslrecommendationa for improvement: Additional research, investi ga tion, or daU required: Reporting Period: ______ Initials: ______ IV 8 Date: __ __

PAGE 134

WORKSHEET 1 6 GENERAL F I NDINGS (Page 5 of8) Read discussion beginning on page IV 1 and insJructlons on page IV. 4. Ge neral findings: TRAINING AS A MEASURE OF QUALITY (WORKSHEET 11) SuggesUo nstrecommendations for Improvement. Add ition a l ruearch, Investigation, or data required: Reporting Period : ______ Preparer' s Initials : ______ IV 9 Date : ____

PAGE 135

WORKSHEET 16 GENERAL FINDI NGS (Page 6 of8) Read discussion beginning on page IV1 and instructions on page IV4 General findings: AVAILABIL.ITY (WORKSHEET 12) Suggestions/recommendations lor Improvement: Additional research, Investigation, or data required : Reporting Period: ______ Preparers I nitia ls : ______ IV10 Date : ____

PAGE 136

WORKSHEET 16 GENERAL FINDINGS (Page 7 of8) Read discussion beginning on page IV1 and instruct i ons on page IV4. General findings: FUNDING & ACCOUNTABILITY (WORKSHEETS 13-14) Suggestions/recommendations for improvement: Additional researc h Investigation, or data required : Reporting Period : _____ Preparer's Initia l s : ______ IV -II Date : ____

PAGE 137

WORKSHEET 16 GENERAL FINDINGS (Page 8of8) Read discussion beginning on page IV 1 and instructions on page IV 4. General findings: COMPARISON TO PEERS (WORKSHEETS 15) Suggestions/recommendations for improvement: Additional research, i nvestigation, or data required: Reporting Per io d: ______ Preparer's Initia ls: ______ IV 12 Date: ____

PAGE 138

Worksheetl7 Comparative Conclusions Part IV: Conclusions In this section the reviewer will draw conclusions about the CTC and/or transportation operators being evaluated, based on the worksheets completed previously and on the insight gained by the review team. Worksheetl7 is intended to draw out information from the evaluation worksheets in order to compare various aspects of service. This worksheet solicits general conclusions regarding a number of specific aspects of service. The worksheet also solicits information for multiple organizations so that the CTC and/or a number of operators can be compared. Completing the Worksheet Complete Worksheet 17 based on the responses to other worksheets. Complete only those sections for which modules have been completed. Several questions that rely in part on the judgment of the reviewer ask if the operator is an asset to the coordinated system in a certain area (e.g., on-time performance). Such questions may be interpreted to mean: is the operator above average? Is the operator superior to the other operators? Does the operator meet or exceed minimum requirements? Answer these questions after careful consideration The review team may use any response system that is felt will be useful as long as it is consistent. Responses may be"+/-," "Yes/Ok/No," "A-F," or "1-10," for example. IV 13

PAGE 139

WORKSHEET 17 COMPARATIVE CONCLUSIONS (Page I of5) Complete the following questions, rating each organization evaluated with a ccnsistent scale (+/-, YIN, 1-10, or A -F), indicating whether the organization is doing well in that aspect or has a poten tial problem and whether the ere is doing better or worse than before. Several questions require a quantitative response. Each fine may have one queslion to answer regarding the ere and one regarrllng each operator. The Slions below are organized according to modules Ref&r to the appropriate worlcsheets. Read discussion beginning on page IV-1 and instruclk>ns on pege IV-13. Evaluation Area G iven the number of tran sportatio n providers a v a ilable, is the structure of the CTC appropriate? Has the CTC achieved an appropriate level of ccmpet i t i on for Its organization and com m un ity ? ...... ;; Leveliof Coordination -Workshe&t 6 . In how many ways, according to the is the loca l TO system coord i nated? Has the best balance of coordination for the l ocal community bee n achieved (or are there areas that sho ul d be further pursued)? Does (or would) each operator contribute to a h i ghly coordi nat ed sys tem? In how many measu res has the CTC improved (out o f 8)? Wh ich operators compare favorably i n rates? In which expense measure(s) d oes the operator compare fa vo rably? Report i ng Period : ______ CTC . . """'"' : . : .. . ' .'.' j Prepa r e r' s I n it ials: ___ ___ IV 14 .. ,, Date: ____

PAGE 140

WORKSHE .ET 17 COMPARATIVE CONCLUSIONS (Page 2 of 5) Evaluation Area On-time performance: Is the CTC improving? Is the operator an asset in this area (above average, better than the others, or meets or exceeds minimum requ ire ments)? Public i nformatio n and communicatio ns: Is the CTC improving? Is the operator an asset in this area? Complaints and grievances: Is the CTC improving? Is the operato r an asset i n this area? Safety: Is t he CTC i mproving? Is the operator a n asset i n t his area? Vehicle safety and comfort: Is the CTC i mprov i ng? Is the operator an asset i n this area? Vehicle maintenance : Is t he CTC Improvi ng in this a r ea? Is the operator an asset i n t h is area? Risk management: Is the CTC i mprov i ng? Is t he operato r a n asset in t his area? Goal setting and p lans for improvement: Does the organization have specific goals and plans for improvement? Reporting Period: _______ CTC """''" Preparer's Initia ls: ______ IV 15 Date:. ____

PAGE 141

W O RKSHE E T 17 COMPARATIVE CONC LU S IONS (Page 3 of5) E valuation Area CTC "'*-' -.. ,,: ; ;'"'"' ,; c1H. Qua l ity manage m ent: Is t h e C T C i mproving? Is th e oper ator an asset i n this area? Sp ecial qua l ities and service s : Does the CTC have special a s pects of q u ality? Does the operator have special "'*-.,._ : ', ;. > u ;r,:i .... Does the CTC h ave m in imum t rain i ng standards? Doe s the o p erator meet the stan d ards? How many h ours of first-yea r driver I are In total? How many subject areas l isted o n the work sheet are drivers trained in ? Drivers: Is the CTC i mp r ov i ng d r ive r train i ng? I s t h e operato r an asse t i n thi s area? Ma i ntenance staff: Is t h e CTC i m p rov i ng mai n tena nce sk i lls? Is the ope rato r an asse t i n thi s area? Ot h er staff: I s the CTC i m p rov i ng staff train i ng? Is the opera tor an asset i n t h i s a r ea? All staff: Is the exp e r i e n ce of th e CTC staff improving? Is the operator a n asset i n t his area? Report i ng Period: ______ Preparer's I n i t i als: ______ Date : ____ IV 16

PAGE 142

WORKSHEET 17 COMPARATIVE CONCLUSIONS (Page 4 of 5) : Evaluation Area Is the CTC improving in meet i ng demand? Outreach: Is the CTC reaching new users? Public information : Is the CTC information? Capacity: Is the capacity of the coordinated system sufficient or Hours and days of serv i ce : Is service available during the days and hours nee
PAGE 143

WORKSHEET 17 COMPARATIVE CONCLUSIONS (Page 5 of 5) Evaluation Area Does the CTC compare favo r ably lo its size Does the CTC compare favorably to its Does the CTC compare favorably to its networi< peers? Is the CTC meeting its assig n ed goals and ob j ectives a n d contributing successf u lly to i m proved T O in the a r ea? Is t he operator (wou l d the operator be) a s i gnificant asse t t o coo r d i nated i n the servi ce area? CTC Reporting Period : _______ Prepare r' s I n itials : ______ IV 1 8 Date : ____

PAGE 144

Worksheet 18 Summary of Evaluation Part TV: Conclusions After all intended worksheets are completed, and after they have been used to draw conclusions via Worksheets 16 and 17, Worksheet 18 should be used to confirm the completion of the evaluation. Worksheet 18 also indicates where the evaluation worksheets and supporting documents will be filed, identifie s the next step t o be taken, and records any suggestions for the next evaluation. Finally, the lead reviewer should sign-off on the completed evaluation. Completing the Worksheet Review Worksheets 1-5, as described at the beginning of this workbook, to review the purpose of the evaluation. Complete Questions 1-9 on Worksheet18 with this purpose in mind. Be sure to indicate where worksheets and supporting documentation will be filed. Indicate the next step to be taken (e.g., "prepare final report for LCB"). The next step may be to hold a meeting, prepare an action agenda incorporate recommendations into the service plan, summarize conclusions in a memo, draft a final report, prepare a presentation, or other action. Record any additional information relevant to completion of the evaluation process. Worksheet 18 is the last worksheet but not the end of the evaluation process. Be sure that the conclusions result in action Good luck! IV 19

PAGE 145

WORKSHEET 18 SUMMARY OF EVALUATION To b6 completed last. Review Worksheets 1-4, and answer the questions below. Refer to discussion on page IV1 and instructions on page IV19 1 Dale evaluation completed: _,_1....,..-1..1 __ mo day yr 2. Was an i nventory of local services conducted? 3 Were the appropriate organizations evaluated? 4. Has the evaluation conducted been appropriate to the goals for the evaluation? 5. Has the review team r eturned to eac h works heet and identified in writing where each organ izatio n might improve or consider for further research? On time? 6. Indicate the location in which the worl
PAGE 146

APPENDICES

PAGE 147

APP E NDIX A RESOURCE LIST

PAGE 148

Appendix A: Resource List Resource List Carter-Goble and Associates, I nc. "Rural Public T ransportation Perfonnance Evaluation Guide." Washington, D.C.: U.S. Department of Transportation, Technology Sharing Program, 1982. Center for Systems and Program Development, Inc. "Guidebook to Best Practices in Specialized and Human Services Transportation Coordination." Washington, D.C.: The Joint DOT/DHHS Coordinating Council on Human Services Transportation Coordination, 1989 Center for Urban Transportation Research. "Florida FiveYear Transportation Disadvantaged Plan: Final Report, Technical Memoranda No I, No.2, No. 3 No. 4, No. 5 ." Tallahassee, Florida: Transport ation Disadvantaged Commission, 1992. Center for Urban Transportation Research. "Methodology Guidelines for Forecasting ID Transportation Demand at the County Level." Tallahassee, Florida: Transportation Disadvantaged Commission, May 1993. Center for Urban Transportation Research. "Statewide Operat i ons Report, Fiscal Years 1 990/91 and 1991/92." Tallahassee, Florida: Transportation Disadvantaged Commission, June 199 3 Center for Urban Transportation Research. "Statew ide Operations Report, Fiscal Years 1992/93." T allahassee, F lori da: Transportation Disadvantaged Commission, January 1994. "Coordination of Community Transportation Services." Washington, D.C .: Joint USDOT/USDHHS Coordinating Council on Human Services T ransportation, n.d. "Florida's Approach to Coordinated Transportation for the Transporta tion Disadvantaged." Tallahassee, Florida: Transportation Disadvantaged Commission, 1988." Hutchinson, Jo Ann "A Case Study: Coordination of Rural Transportation in Florida." Tallahassee, Florida: Transportat i on Disadvantaged Commission Library, n.d Hutchinson, Jo Ann. "Florida's Decade of E xperience in Coordinated Transportation for the Disadvantaged." Tallahassee, Florida: Transportation Disadvantaged Commission Library 1990. Institute for Urban Transportation. "Handbook for Management Performance Audits DOTT88-21. Bloomington, Indiana: Urban Mass Transportation Administration and U.S. Department of Transportation, 1988. Miller James H. "Shared-Ride Per for mance Evaluation Guide." Washington, D.C.: U.S. Department of Transportation. Tech nology Sharing Program, 1989. National Center for Health Statistics. "Current Estimates From the National Health Intervie w Survey, 1989 ." DHHS Publication No. (PHS) 89-1501. Hyattsville, Maryland: Department of Health and Human Services. 1989 AI

PAGE 149

Appendix A: Resource List (Continued) Ohio Department of Transportation Division of Public Transportation. "A Handbook for Coordinating Transportation Service." Washington, D.C.: U .S. Department of Transportation Technology Sharing Program, 1991. "Planning Workbook." Tallahassee, Florida: Transportation Disadvantaged Commission, June 1991. Prater, Gwendolyn Spencer, Alexander McKinley Jr. and Ruth Williams "An Analysis of Four Selected State Transportation Coordinated Efforts in Social Services and Rural Public Transportation." DO T MS-11-0003. Jacks on, Mississippi: Jackson State University Departments of Social Work and Economics, 1989. Regional Public Transportation Authority. "Developing a Performance Management System for Transit Services." Phoenix, Arizona: U.S. Department of Transportation, Technology Sharing Program, 1989. State of Florida Office of the Auditor General. "Performance Audit of the Transportation for the D isadvanta ged Program of the Coordinating Council on the Transportation Disadvantaged and the Department of Transportation." Tallahassee, Florida: State of Florida Office of the Auditor General, 1987. Transportation Accounting Consortium. Rural Transportation Accounting: A Model Uniform Accounting System jar Rural and Specialized Transportation Providers. DOT -1-87-08. Lansing, Michigan: U.S. Department of Transportation, 1986. Transportation Accounting Consortium. "Simplifying Human Service Transportation and Small Transit System Accounting: A Six State Perspective." Lansing Michigan: U.S. Department of Transportation, 1983. U.S. Bureau of the Census. "1990 Census of the United States, Preliminary Results." Washington, D.C.: Bureau of the Census, 1991. A 2

PAGE 150

APPENDIX B LIST OF ABBREVIATIONS

PAGE 151

ABE AOR ASE CDL CTC CTDP CUTR DOPA FAC FCTS FS FTE LCB MIS MOA MPO POS RFP SOR SSPP TD TDC TQM USDHHS USDOT List of Abbreviations Annual Budget Estimate Annual Operating Report Automotive Service Excellence Commercial Driver's License Community Transportation Coordinator Co ordinat ed Transportation Development Plan Center for Urban Transportation Research Designated Official Planning Agency Florida Administrative Code Florida Coordinated Transportation System Florida Statutes Full-time Equivalent Local Coordinating Board Management Information System Memorandum o f Agreement Metropolitan Planning Organization Purchase of Service Contract Request for Proposals Statewide Operations Report System Safety Program Plan Transportation Disadvantaged Transporta t ion Disadvantaged Commission Total Quality Management Appendix B: Abbreviations U.S. Depa rtment of Health and Human Services U S. Department of Transportation B I

PAGE 152

APPENDIX C GLOSSARY OF TERMS

PAGE 153

Appendix C: Glossary of Terms Glossary of Terms Accidents the total number of reportable accidents resulting in property damage and/or personal Injury. Agency an official, officer, commission, authority, council, committee, department, division, bureau, board, section, or any other unit or entity of the state or of a city, municipality, county, or other local governing body or a private nonprofit entity providing transportation services as all or part of its charter. Annual Budget Estimate a budget estimate of funds available for providing transportation services to the transportation disadvantaged that is prepared annually and covers a period of one state fiscal year. Annual Operating Report an annual report prepared by the community transportation coordinator detailing its designated-area operating statistics for the most recent operating year. Chapter 427, FS the Florida statute establishing the Transportation Disadvantaged Commission and prescribing its duties and responsibilities. Community Transportation Coordinator (CTC) a transportation entity recommended by a designated official planning agency to ensure that coordinated transportation services are provided to the transportation disadvantaged population in a designated service area. Formerly known as a coordinated community transportation provider. Complete (or Full) Brokerage type of CTC network in which the CTC operates no transportation service and contracts with other operators for the delivery of all transportation services. Coordinated Trips passenger trips provided by or through a CTC. Demand-Responsive Service a transportation service characteri.zeQ by flexible routing and scheduling that provides door-to-door or point-to point transportation at the user's request. Designated Official Planning Agency (DOPA) the agency or official body designated by the Transportation Disadvantaged Commission to appoint the community coordinating board and recommend the community transportation coordinator for each service area. Metropolitan planning organizations are automatically the official planning agencies in urban areas. Designated Service Area the geograph ical area, consisting of one or more counties, in which the CTC is the designated provider. Economies or Scale cost savings resulting from combined resources (e.g., joint purchasing agreements that result in a lower cost per gallon or quantity discount for fuel). C I

PAGE 154

Appendix C : Glossary of Terms {Continued) Effectiveness Measure a performance measure that indicates the level of consumption per unit of output. Passenger trips per vehicle mile is an example of an effectiveness measure. Efficiency Measure a performance measure that evaluates the level of resources expended to achieve a given level of outpu t. An example of an efficiency measure is operating cost per vehicle mile. Employees the total number of persons employed in an organization. Fixed-Route Service transit service in which the vehicles follow a schedule over a prescribed route. Full Time Equivalent (FTE) -a measure used to determine number of employees based on a 40-hour work week One FTE equals 40 work hours per week. Fully Allocated Costs the tota l cost of providing coordinated transportation, including those services which are purchased through transportation operators or provided through coordination contracts. General Trips passenger trips by individuals to destinations of their choice, not associated with any agency program. Goal broad conditions that define what the organization hopes to achieve. Latent Demand demand that is not active (i.e., the potential demand of persons who are not presently in the market for a good or service). Local Coordinating Board (LCB) entity in each designated service area composed of representatives appointed by the official planning agency. Its purpose is to provide assistance to t11e community transportation coordinator the coordination of transportation services. Memorandum of Agreement (MOA) a binding standard contract between the Transportation Disadvantaged Commission and a CTC. This contract and its provisions serve as a performance and reporting standard to guide the delivery of services by all agencies or entities that provide transportation disadvantaged services. Metropolitan Planning Organization (MPO) the organization responsible for transportation planning and programming in urban areas. Also serves as the official planning agency referred to in Chapter 427, FS. Network Type -describes how a commu n ity transportation coordinator provides service, whether as a complete brokerage partial brokeroge, or sole provide r Non-coordinated Trip a passenger trip provided outside of the coordinated system. c-2

PAGE 155

Appendix C: Glossary of Terms (Continued) Non-sponsored Trip a passenger trip that is not subsidized in part or in whole by any local, state, or federal government funding source, Objective specific, measurable conditions that the organization establishes to achieve its goals. Operating Cost reported total spending on operations, including administration, maintenance, and operations of service vehicles, and excluding capital spending. Operating Cost per Passenger Trip operating costs divided by the total annual ridership, a measure of the efficiency of transporting riders. One of the key indicators of comparative performance of transit properties since it reflects both the efficiency with which service is delivered and the market demand for the service. Operating Cost per Vehicle Mile operating costs divided by the number of vehicle miles, a measure of the cost efficiency of delivered service. Operating Environment-describes whether the community transportation coordinator provides service in an urban or rural service area Operating Revenue all rev e nues and subsidies utilized by the operator in the provision of transportation services. Operating Statistics operating data on various characteristics of operations, including passenger trips vehicle miles, operating costs, revenue, vehicles, employees, accidents, and roadcalls Organization Type -describes the structure of a community transportation coordinator, whether it is a private-for-profit, private non-profit, government, or transit agency. Paratransit specialized service provided by many types of vehicles between specific origins and destinations (including automobiles vans, and buses) that is typically used by transportation disadvantaged persons. Examples of paratransit service include demand responsive service and subscription service. Partial Brokerage type of CTC network in which the CTC runs part of the transportation service and contracts with other providers to operate the remaining service. Passenger Miles number of annual passenger trips multiplied by the system's average trip length (in miles). This number provides a measure of the total number of passenger miles of transportation service consumed. Passenger Trips annual number of passenger hoardings on the vehicles. A trip is counted each time a passenger boards a vehicle. T,hus, if a passenger has to transfer between buses to reach a destination, he or she is counted as making two passenger trips. Passenger Trips per Vehicle Hour a performance measure used to evaluate service effectiveness. c3

PAGE 156

Appendix C: Glossary
PAGE 157

Appendix C: Glossary of Terms (Continued) employment, education, shopping social activities, or other life-sustaining activities, or children who are handicapped or high-risk as defined in s.411.202. TD Category I includes persons with disabilities, senior citizens, low income persons, and "high risk" or "at risk" children. These persons are eligible to receive certain governmental and social service agency subsidies for program-related trips. TD Category II a subset of the TD Category I population. Includes only those persons who are transportation disadvantaged according to the eligibility guidelines in Chapter 427, FS (i.e., they are unable to transport themselves or to purchase transportation). These persons are eligible to receive TD Trust Fund monies for non-sponsored general trips Transportation Disadvantaged Commission an independent organization created in !989 to accomplish the coordination of transportation services provided to the transportation disadvantaged population. Replaced the Coordinating Council for the Transportation Disadvantaged. Transportation Disadvantaged Funds any federal, state, or local funds available for the transportation of the transportation disadvantaged. Includes funds for planning, administration, operations and capital equipment. It does not include funds used for the transportation of children to public schools. Transportation Disadvantaged Population persons from the total population meeting the transportation disadvantaged definition guidelines. Transportation Disadvantaged Trust Fund a fund administered by the Transportation Disadvantaged Commission in which all fees collected for the transportation disadvantaged program shall be deposited The funds deposited will be appropriated by the legislature to the commission to carry out the commission's responsibilities. Transportation Improvement Plan a staged multi-year program of transportation improvements, including an annual element developed by an MPO specifying program activities for the current fiscal year. Transportation Operator a public, private for-profit or private non-profit entity engaged by the community transportation coordinator to provide service to transportation disadvantaged persons. Trend Analysis a common technique used to analyze the performance of an organization over a period of time. Trip Priorities various methods for restricting or rationing trips. U nmet Demand the number of trips desired but not provided because of insufficient service supply. c. 5

PAGE 158

Appendix C: Glossary of Terms (Continued) Urbanized Area -an area designated by the Bureau of the Census that contains a central city or cities and surrounding closely settled urban fringe (suburbs), which together have a population of 50,000 or more. Vehicle Hours total vehicle hours used in providing revenue service including deadhead time. Vehicle Inventory vehicles used by the CTC, transportation operators, and through Coordination Agreements for the provision of transportation disadvantaged services. Vehicle Miles the number of miles traveled by vehicles that provide passenger service. Vehicle Miles Between Road calls -a performance measure used to evaluate reliability of service. Vehicl e Miles per Vehiclea performance m easure used to evaluate resource utilization and rate of vehic le depreciation. Vehicles number of veh i cles owned by the transit agency that are available for use in providing services. C-6

PAGE 159

APPENDIX D CTC PEER GROUP DATA

PAGE 160

T A BL E D 1 (SORTa bl e 14 ) 1 993 Perf ormance Meas ures b y Syste m Size. .. 1993 Perfonnance "''asijr9s .": ;, ,. ,. ... "'"'"'''Y""' .,_ .. --.,-., 8Ne pet RevenutMile $ 1 .4 0 $ 1 .16 $ 1.41 $1.49 11.35 $ 1 .75 12.38 Opera t ing E xp.eMe Pef PaS4enguTrip Total $29 78 $9.97 $7.86 $ 7.16 S7. 08 $8 .... $4. 7 4 Ope r ating Expense per Paratrana.it $29.7 8 $9.97 $ 1 .86 $ 7. 18 $ 7. 48 $ 6 45 1 11.82 STEM SAF ETY ....... 0.37 0.4 9 1 .56 0 .96 1.12 3 04 2.07 68.389 52,074 3 1,71 4 2 3 .109 32.136 9,341 19 56& 2 7 % 16.0% 17.3% 21.6% 23.8% 33. 7% e as e d on numbe r of tripa. In 18gJ, 1=0-8.999; 2z10, 000-49.999:3 001).99 ,99e;4=100.000-19i,999:W00,0007 4 9 .999;6=?5 0 000.1 ,4H.m7 .. 1,500 ,000nd u p

PAGE 161

a N 1993 Performance Measures Medians SERVICE AVAILABILITY Ve hicl e MiiC$ per TO Capita Revenue Miles per TO Capila SERVICE EFFECTIVENESS PusengcrTI'ips per Vehle l e M ile Tola l Passenger Trips per Vehicl e Mile ParOltransit Passenge r Trips per Revenue Mile Total Passenger Trips per RevtnueM ile Paratran.sit COST EFFICIENCY Opera ting Expen.se per Vehicle f\&e OperatinO Exp; .Mt.li.t' '? #1:.1'xt,. v,t '_ ... . . ,. ,.,.h ... ___ ''')>.,,_,.. ""-"'''" PFI! ... G '"" .. ,. .,., .. C$A:G X ,.,;" '<' ...,.p,.,_);/;1$>;. !0+.f4>-l!'1.f':'"')_ 16.25 15.3 1 0 17 0.16 0 19 0.18 $1.34 $1.43 $8.91 $10.12 1.23 33,523 3.5% 16.90 16.19 0.15 0 13 0.15 0 .14 $1.39 $1.45 $9. 51 $10.50 1.20 32.857 2.4'4 6.92 6.75 0.27 0 .2 7 0.29 0.29 $1.67 $2. 1 5 $6.46 1.16 13.2 27 7% 9.37 8.22 0.25 0.25 0 32 0.32 $1.63 $2.06 $6.47 $6.47 3.32 12 ,34 7 34.3% M edians 11.85 10.15 0.26 0 .19 0.27 0.19 $1.51 $1.69 $S.S3 $8.00 1 .75 19.566 39.4%' Averages .. 7.34 38.85 0.10 0.10 0.13 0.13 $1 .02 $1 29 $9.06 $9.06 0.31 52,785 6 .1% 12.93 1 29.81 11.&0 2.29 0.29 0 12 0 19 0 12 0 .32 0.15 0.21 0.15 $1.60 So.96 $1.75 $1.20 U.45 $7.98 $8.17 $7 .98 1.90 I 0 .60 14,5771 45,855 44.8%1 8 8'4 n/a nJa n/a nla n la nla nla n/a nla nla n/a n/a n/a nJa n/a n/a nla n/a n/a nla n/a nla nJa n/a n/a n/a 10.27 9 .62 0.16 0 16 0.18 0 .16 $1.47 $1 .62 $7.86 $7.86 0.71 42.n2 15.3'4 10.66 9.93 0 20 0 .20 0.21 0 21 $1.46 $1.57 $7.46 $7.45 1.14 30,867 15.8'4 n/a nla n/a nla n/a nla nla nla nla n la n/a n/a n/a nla n/a n/a niO n/a nJa nla n/a niO nJa nlo nJa n/a pNP..Private no n-profit PFPPYiwte for-profit Go:Govemmenl (other than p ublic transit operatOI'); PT=Pubk ttanait operalor (I.e., transit operator). nla: Not applicable. :... s 0 .. () (i "'

PAGE 163

.... .... 1993 CTC I I 1 993 Sel'\llee Area Chrac.teri$ti u "'pet EXpcns., :fi!X>; lD. c< ,_,._ {' ,:".M it: Tolal j :i.-IO :TOII.i 'o:: 'l.A& .-.-11 v'M. v";.. ... l ...... ... "'" ,... "''" 8Jk01' s R "'" .. ;. M :et 4 7 .4'3 oA ,_,.. .. ... .. . .. p u PNP 13 .3, 1303 B rtofor(J c PNP SU4 i ' o .ot,. p u PT 2U9 23 .75 0 1 9 0 .21 $0.72 $1. 02 S1.03 t 1 0.9() BIOWd c u PT 13.M : Ca lh.rr s R PNP a u t 6 1 .03 0 1 9 ... ".0.111 ... Clwrlor:t p u G C i i M p G 6.7 3 6.10 0 37 0 1 8 0.19 S l.35 ,_ '""'" '. .. 0.6& ; t '<) ....... .............. ;o.)W ... $2.1 4 ..... s1 1 1 Clay p u 1'1.30 11;li ..... tt. 19 ' -"SVI S . .. Cellief v PNP 9 .(16 5 .88 0 1 2 0.18 st.n sus $$ 52 0.15 24. 631 Colu,HIIII'I Sv.v p G uo .,, p u PT 11.:16 Ul .. 0 .44 0.43 0 .50 Jl. 32 {, .$5;35 u38 $4. 14 o M .. ,., i1,.o61 A t9.'6&8 S2.24 i2.0& OeS, t hlfJii ,O:.. e c tt.94 1 o.92 c R PNP ..... 31 .9& c u PfP 17.S6 .. tscamllia c u PFP 1 4 .53 ""' .,, ... 0 .12 0 .?3 0.(16 ,;. 0 .2S S1.3S $8. 2 7 ,_,.._. -<.-' "--. ' $0.19 $ 13.1 2 $4) 4 $ t i.4S : ;_' '< -SU4 .. 1.6& .-<# $8..3&3 -;;)'f.H< ,,J'.,; ...,,., 0 .17 28_$2$ ; .... $1.38 $5 0 1 .29 lA.857 s PHP 2?.84 ,. .. '. p PHP ..... 43, 72 p u PHP 29-45 < 16.96 p u PHP ... 5 .78 020 003 020 ; p.n 0 ,04 .. 0 .21 $1.29 .. ., ... n$.4 :ft. $1.86 S.C6. 0 7 o .co 236. 1 70 S 1.1 4 4 1.37\;S$. 7 <-. s )(#---"-' :t ) '" -----:-> .-.. ........... ,... .; -k v;.,.)_ _,.-,_41:;-1<1> >1, ...... ;o. .. ;.: .................. ..... S1.43 S1.52 S7. t 0 1. 13 11,330 1 .p ... 45.$6 ,, ;41.03 0 I I u PHP ... p u G .10.81 ,_1 0.13 I I s ... ir.34 .... p u ..... 1U2 17. 2 1 ,,_.,., s R .... ..... 5u? 002 o.21 ,, 0.12 0.14 "l '" 0.10 0.'2& ., 0.15 -,-; > "'' .,. S 1 .50 NR ..... .. A ... ..... t :1.1hir $().95 $0.97 $0.67 "" ;-" '>;.w\1_&.)9 .; ,.. ' " "-< -'""''"" "'" p u PT 5.59 ... o.tL o .2e o:zr p u '''<, _,,, __ -.: ;_< p 42.3$ l7. 9S o.u 0.11 p u . < 1 5 32 < V"- '$-V c u PT 1 1.85 1 0 .15 0.26 c u . ..... '?'" jot!, c < ': ;; "" < ,-;,i#\<1-.,. p u 4 u ... ., M p u p p c -.!/, p ,._ u '" R p .. ft nil: Hot lflplic:IIM (flO .... ftlf h CICU'It)t). Nft: AOR.,.. diM ..... wae 1 ;; () {i B

PAGE 164

CUIR Center for Urban Transportation Research College of Engineering University of South Florida T ampa, Florida