A Balanced Approach To Normalising Bus Operational Data For Performance Benchmarking Purposes

Transcription

1 A Balanced Approach To Normalising Bus Operational Data For Performance Benchmarking Purposes RTSC Introduction Bus Benchmarking Update Mark Trompet Railway and Transport Strategy Centre, Centre for Transport Studies Project Manager of the International Bus Benchmarking Group Daniel Graham

2 Introduction to The Railway and Transport Strategy Centre Dedicated to serving the transport industry, with high quality, applied research. 80% growth in last 5 years A contemporary University business model: profits reinvested in further academic research, peer review publication, teaching, PhD scholarships Recognised internationally as benchmarking experts 18 years of experience in urban bus, metro and rail benchmarking High value to industry (> $100 millions) achieved Community of Metros CoMET Experts in PT operations and strategy World leading research in transport appraisal, wider economic benefits, efficient transport pricing, regulation, safety and risk management International Bus Benchmarking Group 2

3 RTSC Management Committee Prof. Stephen Glaister (Chair), Dr, Dan Graham, Richard Anderson, Prof. Washington Ochieng, Peter Higgs Chairman Prof. Stephen Glaister Dr. Daniel Graham Research Director Principal Research Fellow Richard Anderson Managing Director Peter Higgs (Accounts) Robin Hirsch Robin Steel Nigel Harris Roger Allport Mark Trompet Senior Research Associate Alexander Barron Senior Research Associate Ben Condry Senior Research Associate (from March 2012) Patricia Melo Post Doc Research Associate (B) Niovi Karathodorou Research Associate (B) Nicholas Findlay Research Associate (3b) Judith Cohen Research Associate (3b) Alex Su Seconded from Taipei Rapid Transit Corporation Ruben Brage-Ardao Research Associate (A) Daniel Piner Research Associate (3a) Richard Parasram Research Analyst (3a) Xiang Liu Research Associate (A) Shane Canavan PhD Student RTSC Organisational Chart March 2012

4 RTSC Staff. Come say hello in room 605, 607, 612, 613 International Bus Benchmarking Group

5 Recent Research (2009 > ) Public Transport Operations and Economics Research in a variety of areas of public transport operations and economics: Anderson, R.J., Findlay, N.S., Graham, D.J., (2011) Improving Fares and Funding Policies to Support Sustainable Metros, 91st Transportation Research Board Annual Meeting, DC, Jan 2012 Melo, P.C., Harris, N.G., Graham, D.J., Anderson, R.J., Barron, A. (2011) Determinants of Delay Incident Performance of Urban Metros, Transportation Research Record: Journal of the Transportation Research Board, No 2216, pp Allport, R.J., Anderson, R.J. (2011) Managing Strategic Risk - The Worldwide Experience of Metros. Proceedings of the Institution of Civil Engineers - Management Procurement and Law, 164 (4), pp Anderson, R.J., Findlay, N.S., Allport, R.J., (2010) Managing Metro Fares and Funding, Metro Report International, September 2010, pp Anderson, R.J. (2009), Maximizing the Potential for Metros to Reduce Energy Consumption and Deliver Low-carbon Transportation in Cities, Presentation to MetroRail Asia, Delhi, Sept 2009 Graham, D.J., Crotte, A., and Anderson, R.J., A Dynamic Panel Analysis of Urban Metro Demand, Transportation Research Part E, Vol 45 (2009), pp International Bus Benchmarking Group

6 Recent Research (2009 > ) Advances in Benchmarking Methodology, Techniques and Indicators Research in improvements to benchmarking techniques and indicators for the public transport industry, but with applications to other fields as well: Today Trompet, M., Graham, D.J., (2011) A Balanced Approach to Normalizing Bus Operational Data for Performance Benchmarking Purposes, Presented at the 91st Transportation Research Board Annual Meeting, Washington, 23rd January 2012 CTS Seminar 2011 Trompet, M., Liu, X., Graham, D.J., (2011) Development of a Key Performance Indicator to Compare Regularity of Service between Urban Bus Operators, Transportation Research Record: Journal of the Transportation Research Board, No 2216,, 2011, pp CTS Seminar 2009 Trompet, M., Anderson, R.J., Graham, D.J., Variability in Comparable Performance of Urban Bus Operations, Transportation Research Record: Journal of the Transportation Research Board, No 2111, 2009, pp International Bus Benchmarking Group

7 International & American Bus Benchmarking Group Update International Bus Benchmarking Group CONFIDENTIAL

8 Twelve International Bus Benchmarking Group members Vancouver Seattle San Francisco Montreal New York Dublin Lisbon London Brussels Paris Milan Istanbul Barcelona Singapore Sydney Potential new members 2012 International Bus Benchmarking Group 8

9 Recent IBBG Research A number of in-depth research projects have been undertaken and are ongoing in the International Bus Benchmarking Group: 2009/2011 Best Practices in Providing Security within Bus Operations Human Resources Organisation and its Impact on Operations Best Practices in Service Control and Route Management Understanding and Improving Bus Driver Productivity 2011/2012 Defining and Measuring Service Quality for Bus Operations Understanding and Improving Bus Vehicle Maintenance Productivity Customer Satisfaction Survey - 4 th year Bus Service Regularity Performance Measurement 4 th year International Bus Benchmarking Group 9

10 potential for improvement Overall Correlation Between Driver Productivity & Constraints (1) Shift flexibility Layover as part of driver shift NET PRODUCTIVITY 2 Layover as % of total vehicle hours Typical working hours per year Absenteeism Days off per rotation cycle GROSS PRODUCTIVITY 1 NET PRODUCTIVITY 1 GROSS PRODUCTIVITY 2 Driving in revenue service Maximum shift length Split shift spread Typical working hours per day Maximum continuous working days Minimum break Start up / shut-down time Maximum continuous driving time Restricted International Bus Benchmarking Group CONFIDENTIAL Low Productivity, High Constraints Less Restricted 10

11 potential for improvement Overall Correlation Between Driver Productivity & Constraints (2) Maximum shift length Driving in revenue service Typical working hours per day High Productivity, Low Constraints Split shift spread GROSS PRODUCTIVITY 2 Shift flexibility GROSS PRODUCTIVITY 1 Maximum continuous driving time Absenteeism NET PRODUCTIVITY 2 NET PRODUCTIVITY 1 Layover as part of driver shift Days off per rotation cycle Typical working hours per year Start up / shut-down time Minimum break Maximum continuous working days Restricted International Bus Benchmarking Group CONFIDENTIAL Less Restricted 11

12 Customer Satisfaction Survey 2011: 10 cities participated, including London cleaned responses Satisfaction Index** Satisfaction Index: Satisfaction scores are normalised by the average satisfaction score How well are you meeting the expectations of your own customers The bus service is usually reliable (Normalised scores) Question Mean International Bus Benchmarking Group CONFIDENTIAL 0.25 Bc* Bs Db Lb Ln Mt Pa Sg* Sy Vc * Bc dataset 318; Sg dataset 215. All others 1,100. ** Satisfaction index = Average member response for this question / Average member response for Qs

13 American Bus Benchmarking Group: 12 Members Across the US in a Wide Range of Urban and Suburban Environments C-Tran (Vancouver) Lane Transit (Eugene) San Joaquin RTD (Stockton) UTA (Salt Lake City) DART (Des Moines) RGRTA (Rochester) RTA (Dayton) CDTA (Albany) RIPTA (Rhode Island) The T (Fort Worth) Capital Metro (Austin) PSTA (St. Petersburg) American Bus Benchmarking Group CONFIDENTIAL 13

14 Example KPI 2: Financial Effectiveness (anonymised and indexed to comply with confidentiality agreement) F2b: Total Operating Cost per Actual Total Vehicle Hour (Index 1 = 2010 Group Average) High variety in performance leads to drill down questioning to understand order of magnitude differences and trends A B C D E F G H I J American Bus Benchmarking Group 14

15 A Balanced Approach To Normalising Bus Operational Data For Performance Benchmarking Purposes Mark Trompet Railway and Transport Strategy Centre, Centre for Transport Studies Project Manager of the International Bus Benchmarking Group Daniel Graham

16 Benchmarking A systematic process of continuously measuring, comparing and understanding organisations performance and change in performance of a diversity of key business processes against comparable peers anywhere else in the world to gain information which will help the participating organisations to improve their performance Adapted from the definition by Lema and Price 16 International Bus Benchmarking Group

17 Research Context The RTSC within the Centre for Transport Studies at Imperial College London has 18 years of public transit benchmarking experience (Metro, Bus, Suburban Rail) Peer organisations are selected based on types of services, operational characteristics and density of service area These similarities enable organisations to compare performance once their operational data are normalised Hypothesis: Some variety in service characteristics does exist: agencies can observe skewed relative performance when only one type of normalisation is used 17 International Bus Benchmarking Group

18 Research Objectives: To quantify the variety in service characteristics between similar agencies and to understand the impact this has on one-dimensional benchmarking To provide benchmarking practitioners and policymakers with a framework that suggests suitable denominators for balanced normalisation To introduce the alternative data denominators: planning capacity kilometres and tonne kilometres and describe how these have increased comparability 18 International Bus Benchmarking Group

19 Seven Years of Experience Used From the International Bus Benchmarking Group (IBBG) Vancouver Los Angeles Montreal New York Dublin Lisbon London Brussels Paris Milan Barcelona Singapore Sydney 19 International Bus Benchmarking Group

20 The Data All IBBG members are large urban bus operators similar in services offered, technological comparability and role within the city New members are chosen by the existing members The data items only include data from normal service operations, filtering out the effect of charter, tourist bus, paratransit and school bus services Based on a set of detailed and clear definitions Up to ten years of data available ( ) 20 International Bus Benchmarking Group

21 Ensuring Data Comparability (1) Lessons from 18 years of CoMET/Nova metro benchmarking One-off benchmarking studies are rarely successful as it can take many years and iterative cycles to achieve comparable indicators Three years of iterative definition development, data collection and analysis before the member operators were sufficiently satisfied with the level of comparability members re-issue information to adhere to the improved definition, ensuring correct time-series analysis 21 International Bus Benchmarking Group

22 Ensuring Data Comparability (2) The strict confidentiality agreement and their willingness to help and learn from each other created an open and honest information sharing environment Organisations that belong to an international group of private sector operators are not invited to the IBBG This to ensure that existing members remain feeling comfortable about openly sharing (confidential) data 22 International Bus Benchmarking Group

23 Ensuring Data Comparability (3) Ownership: The members own the group and decide what topics and KPIs will be studied and measured No data shielding: Member organisations are not directly (financially) rewarded based on the results of the benchmarking Background information and data is collected, which helps to understand differences in performance This enables the IBBG to show less comparable organisations within the same KPI, however with appropriate caveats 23 International Bus Benchmarking Group

24 Normalisation of Data Performance data needs to be normalised for scale as far as reasonably possible and desired Passenger boardings range: 80 million (Brussels) to 2.3 billion (London) For each KPI, the most suitable denominator was chosen: Passenger boardings, passenger kilometres Vehicle kilometres, vehicle hours (revenue / total) Capacity kilometres (seat / all) Staff hours (total / categories) Financial data needs to be expressed in comparable units before being normalised. The IBBG uses the World Bank s Purchasing Power Parity 24 International Bus Benchmarking Group

25 Results: Quantification of the Variety in Service Characteristics Between Similar Agencies Type of service characteristic N µ Min Max CV Average passenger trip length - km Network efficiency - % of deadheading km Weighted average vehicle planning capacity Average commercial speed km/h Weighted average vehicle weight - tonne N = Number of bus organizations in sample µ = Sample average Min = Minimum value Max = Maximum value = Standard deviation CV = Coefficient of variation 25 International Bus Benchmarking Group

26 What impacts commercial speed? External/Environment Traffic Road works Weather (snow) Bus Priority Infrastructure Traffic Signal Priority Bus lanes Service Characteristics Distance between stops Pre-paid boarding Vehicle Characteristics Number of doors All door boarding 26 International Bus Benchmarking Group

27 Example of Variability in Speed Km/h Commercial Speed Operator A produces 11 revenue kms more per revenue hour than C Bus12 Bus4* Bus2 Bus6 Bus11 Bus8 Bus9 Bus7 Bus5 Bus10 Bus1 Bus3 A 27 International Bus Benchmarking Group B C

28 Effect of relative speed position on performance normalised by vehicle kms and hours (1) (PPP) 28 International Bus Benchmarking Group B C

29 Effect of relative speed position on performance normalised by vehicle kms and hours (2) (PPP) 29 International Bus Benchmarking Group B C

30 INDEX (1 = group average) Normalising for Differences in Bus Capacity Passenger Km per Revenue Vehicle Planning Capacity Km and Passenger Km per Revenue Vehicle Km Indexed Planning Capacity / Bus C E G I H A D K B J F L Passenger Km per Revenue Vehicle Km Passenger Km per Revenue Service Planning Capacity Km 30 International Bus Benchmarking Group

31 Framework for Balanced Normalisation Total Tonne Kilometres Vehicle Weight Total Vehicle Capacity / Km Network Efficiency Total Vehicle Hours Revenue Capacity Kilometres Bus Planning Capacity Revenue Vehicle Kilometres Commercial Speed Revenue Vehicle Hours Vehicle Utilisation Performance Passenger Kilometres System Utilisation Performance Trip Length Passenger Boardings 31 International Bus Benchmarking Group

32 Different Vehicle Types in Fleet: Need for Improved Capacity Normalisation Capacity per vehicle 100 Average Weighted Vehicle Capacity B H A L I K J C G D F E Seating Capacity Service Planning Capacity 32 International Bus Benchmarking Group

33 Different Vehicle Weights: Option for Improved Fuel Efficiency Analysis (Reasons Other Than Weight) INDEX (1 = group average) 1.8 Diesel per Total Diesel Vehicle Planning Capacity Km and Diesel per Total Diesel Vehicle Tonne Km Indexed A B J I D G E H C K F Diesel (liter) per Total Diesel Planning Capacity Km Diesel (liter) per Total Diesel Tonne Km 33 International Bus Benchmarking Group

34 Conclusions (1) Amongst comparable peers, significant variety in operating characteristics has been observed This variety has implications for benchmarking, especially for operators that are on extreme sides of the spectrum(s) Choice of denominator can skew relative performance Two or more KPIs need to be produced and analysed using complementary normalisation factors This research offers a framework to find the suitable complementary normalisation factor 34 International Bus Benchmarking Group

35 Conclusions (2) The IBBG has made significant improvements in data normalisation by introducing: Service planning capacity kilometres Tonne kilometres Seat capacity kilometres normalisation: biased due to differences in proportion of seating vs standing Vehicle weight has a significant impact on fuel efficiency 35 International Bus Benchmarking Group

36 Practical Applications: Who Can Use These Conclusions? Senior Management identifying priority areas for improvement Regulators / Authorities that need to understand relative performance of their contracted services Students researching relative performance 36 International Bus Benchmarking Group

37 Message to take home Use two or more denominators to normalise for size Analyse the graphs simultaneously Achieve a more balanced view of relative performance 37 International Bus Benchmarking Group