October 1, 2012 APTA Annual Meeting
Presentation Structure 1. Benchmarking and the RTSC at Imperial College London 2. Introduction to the Benchmarking Groups 3. Overview of the Benchmarking Process and Methodology 4. Managing the Data A Balanced Approach Questions and Discussion 2
1. Benchmarking and the RTSC at Imperial College London 3
Benchmarking is the Search for Best Practices That Lead to Superior Performance A systematic process of continuously measuring, comparing and understanding organizations performance and changes in performance Of a diversity of key business processes Against comparable peers To gain information which will help the participating organisations to improve their performance (Adapted from the definition by Lema and Price) 4
Improving Transit Performance Through Benchmarking Identify best practices and understand: Strengths and weaknesses Where improvements are most likely achievable, helping to set challenging but achievable targets What has/hasn t worked elsewhere How to reduce cost, improve service quality & safety How good ideas can be implemented, at both the strategic and day-to-day levels Rarely is there a challenge that another operator hasn t also faced Focus has to be positive, not punitive! 5
The Railway and Transport Strategy Centre (RTSC) at Imperial College Focus on Independent, Comparable Benchmarking World leader in public transit benchmarking Urban transit operations Transportation economics & policy Fall 2012 rankings show Imperial College London as 6 th in the world Often considered to be the MIT of Europe The RTSC has an international team of 15 - a wide variety of experience and expertise Part of the Centre for Transport Studies, within the Department of Civil and Environmental Engineering 6
2. Introduction to the Benchmarking Groups 7
RTSC History and Experience 18 Years of Successful Worldwide Benchmarking Projects 1994 Group of Five metros (subway systems) formed 1996 Community of Metros (CoMET) founded for large metros 1998 Success of CoMET leads to formation of Nova group for medium-sized metros 2004 International Bus Benchmarking Group (IBBG) established for urban bus operators 2010 International Suburban Rail Benchmarking Group (ISBeRG) established for suburban/regional rail operators 2011 American Bus Benchmarking Group (ABBG) established for mid-sized bus operators in North America Significant benefits have driven continued participation: e.g. New York, London have both been CoMET members for 18 years and IBBG members for 8 years 8
65 Public Transit Operators Worldwide Are Part of the Benchmarking Effort Oslo Brussels Copenhagen Toronto London Vancouver Montreal Seattle Cleveland Paris Munich Moscow Beijing Salt Lake City Lisbon Milan Tokyo Madrid San Francisco New York Naples Barcelona Guangzhou Shanghai Austin Delhi Taipei Mexico City Hong Kong Bangkok Singapore Sao Paulo Rio de Janeiro Santiago Brisbane Sydney Buenos Aires Melbourne 9
Network Length (Km) Billion Passenger Journeys Because Public Transit Generally Has Returns to Density and Not Scale, We Can Compare Organizations of Different Sizes 600 Network Size and Annual Passenger Journeys (2011) Nova Metros CoMET Metros 3.0 500 2.5 400 2.0 300 1.5 200 1.0 100 0.5 0 0.0 Network Length (Km) Passenger Journeys 10
3. Overview of the Benchmarking Process 11
Four Key Principles Guiding the Benchmarking Groups Collaboration giving and taking the good and the bad; members help each other improve, and the greatest benefits come from active cooperation and participation Confidentiality completely open information exchange within the groups and complete confidentiality to the outside Members can be open and honest Anonymization protocols/tools for external dissemination where appropriate Speed moderate group size and study scope, with fast online interactions Independence flexibility to focus on areas of most immediate interest to members 12
Total Capacity per Car Elements of the Imperial College Benchmarking Model KPI System to compare performance, identify lines of inquiry Case Studies In-depth research on topics of common interest, to identify best practices Clearinghouse Studies Shorter, faster studies to quickly draw on group knowledge and experience Website with Online Forum Peers consult with each other, providing quick answers Meetings attended by senior management, plus expert workshops and Imperial College visits 300 250 200 150 100 50 0 Railway-Defined Capacity per Car (Seated + Standing, 2010) SP* Os* Tk* Sy Mu LO SF* LI MN* Ch* Metros Group Average 13
Why We Look at Key Performance Indicators Benchmarking is NOT only a comparison of data or a creation of rankings The structured KPI comparisons can be used for: Stimulating productive why questions Identify lines of inquiry, where drill-down is needed via studies Identifying high priority problems, strengths and weaknesses Identifying and monitoring trends and the best practices behind them for potential transfer and implementation Internal motivation setting challenging but achievable targets Supporting dialogue with stakeholders (confidentiality permitting) But the benefits of measurement should outweigh the cost of data collection CONFIDENTIAL 14
American Bus Benchmarking Group 2012/2013 Key Performance Indicator System Growth & Learning G1 Passenger Boardings (5-year % change) G2 Vehicle Miles and Hours (5-year % change) G3 Passengers per Revenue Mile & Hour G4 Staff Training (by staff category) Customer C1 Customer Information (scheduled and real-time) C2 On-Time Departure Performance (0 <> + 5) C3 Passenger Miles per Revenue Capacity Mile C4 Passenger Miles per Revenue Seat Mile C5 Lost Vehicle Miles Internal Processes P1 Peak Fleet Utilization (not used split by cause) P2 Network Efficiency (revenue miles & hours per total miles & hours, non-revenue split by category) P3 Staff Productivity (total vehicle hours & miles per labour hour) P4 Staff Absenteeism Rate (by staff category) P5 Mean Distance/Time Between Road Calls Financial F1 Total Cost per Total Vehicle Mile & Hour F2 Total Operating Cost per Total Vehicle Mile & Hour (F3 service operation, F4 maintenance, F5 administration) F6 Service Operation Cost per Revenue Mile & Hour F7 Total Operating Cost per Boarding & Pax Mile F8 Operating Cost Recovery (fare revenue & commercial revenue per operating cost) F9 Fare Revenue per Boarding & Pax Mile Safety S1 Number of Vehicle Collisions per Vehicle Mile & Hour (preventable & non-preventable) S2 Number of Staff Injuries per Staff Work Hours S3 Staff Lost Time from Accidents per Staff Work Hours S4 Number of Passenger Injuries per Boarding & Pax Mile S5 Number of 3rd Party Injuries per Vehicle Mile & Hour Environmental E1 Diesel Fuel Consumption E2 CNG Fuel Consumption (per total vehicle mile, per pax mile, and per capacity mile) E3 CO2 Emissions per Total Vehicle Mile & Pax Mile 15
KPI Challenges: Reaching Comparability Takes Time and is a Continuous Effort, Building on 18 Years of Experience One-time benchmarking studies are typically not successful, as it takes iterative cycles and ongoing work to achieve comparability Confidentiality permits an open and honest information sharing environment Comprehensive KPI definitions and handbook Understanding of context is key to interpret performance (use of profile reports and regional data) Data availability/quality: sufficient level of detail and subcategories (e.g. staff categories) necessary Drill-down of detailed cost and performance data, with studies going deeper into areas of interest 16
4. Managing the Data A Balanced Approach 17
Normalization of Data Performance data needs to be normalized for scale as far as reasonably possible and desired Passenger boardings range in the IBBG: 80 million (Brussels) to 2.3 billion (London) For each KPI, the most suitable denominator was chosen: Passenger boardings, passenger miles Vehicle miles, vehicle hours (revenue / total) Capacity miles (seat / all) Staff hours (total / categories) Financial data needs to be expressed in comparable units before being normalized. Inflation corrected The International Groups use the World Bank s Purchasing Power Parity Index 18
IBBG Example: Quantification of the Variety in Service Characteristics Between Similar Agencies Type of service characteristic N µ Min Max CV Average passenger trip length - km 13 4.6 2.8 8.0 1.6 0.35 Network efficiency - % of deadheading km 13 10.4 7.3 17.3 3.5 0.34 Weighted average vehicle planning capacity 12 71.2 52.1 94.8 15.7 0.22 Average commercial speed km/h 11 17.3 12.0 23.3 3.3 0.19 Weighted average vehicle weight - tonne 12 12.5 11.2 14.9 1.0 0.08 N = Number of bus organizations in sample µ = Sample average Min = Minimum value Max = Maximum value = Standard deviation CV = Coefficient of variation 19
IBBG Example: Variability of Speed Between Members Km/h 25 20 Commercial Speed - 2010 Operator A produces 11 revenue kms more per revenue hour than C 15 10 5 0 Bus12 Bus4* Bus2 Bus6 Bus11 Bus8 Bus9 Bus7 Bus5 Bus10 Bus1 Bus3 A B C 20
IBBG Example: Effect of relative speed position on performance normalised by vehicle kms and hours (1) (PPP) B C 21
IBBG Example: Effect of relative speed position on performance normalised by vehicle kms and hours (2) (PPP) B C 22
Imperial College Framework for Balanced Normalization Total Tonne Miles Vehicle Weight Total Vehicle Capacity / Miles Network Efficiency Total Vehicle Hours Revenue Capacity Miles Bus Planning Capacity Revenue Vehicle Miles Commercial Speed Revenue Vehicle Hours Vehicle Utilisation Performance Passenger Miles System Utilisation Performance Trip Length Passenger Boardings 23
Conclusions Rarely is There a Challenge That Another Operator Has Not Already Faced 65 transit operators across the world comparing performance and sharing ideas the benchmarking has continued for 18 years due to clear purpose and benefits Commitment to continuous improvement, with senior-level support and adequate staff resources required for success Benchmarking is becoming an essential and highly cost effective tool for transit managers to meet their increasing and complex challenges 24
Thank You! Questions? Alex Barron Senior Research Associate ABBG Project Manager CoMET and Nova Deputy Manager Railway and Transport Strategy Centre Imperial College London Email: alexander.barron@imperial.ac.uk Mark Trompet Senior Research Associate Bus Benchmarking Program Manager IBBG Project Manager Railway and Transport Strategy Centre Imperial College London Email: m.trompet@imperial.ac.uk 25