Smart Card Data in Public Transport

Transcription

1 Department of Technology & Operations Management Smart Card Data in Public Transport Paul Bouman Also based on work of: Evelien van der Hurk, Timo Polman, Leo Kroon, Peter Vervest and Gábor Maróti Complexity in Public Transport:

2 NETHERLANDS RAILWAYS (NS) IN NUMBERS

3 NETHERLANDS RAILWAYS (NS) IN NUMBERS 1, Journeys per weekday Yearly Passenger km s 97.4 % Train Punctuality 1.5 % Cancelled Trains Train Services per Weekday Train Wagons/Drivers

4 NETHERLANDS RAILWAYS (NS) IN NUMBERS 1, Journeys per weekday Yearly Passenger km s 97.4 % Train Punctuality 1.5 % Cancelled Trains Train Services per Weekday Train Wagons/Drivers

5 SMART CARD DATA (AUTOMATED FARE COLLECTION) Dutch OV-chipkaart Always both check-in and check-out Some differences between modalities CardID Location Time Type Harderwijk 13:42 CHECKOUT Amsterdam 13:43 CHECKIN

6 OVERVIEW Introduction Finding Passenger Routes From smart card transactions to routes A Better Way to Measure Service Quality New Service Indicators from the perspective of the passenger Analyzing Demand Insight into passenger demand enables better service design Discussion, Conclusion

7 FROM SMART CARD AND TIMETABLE DATA TO PASSENGER ROUTES

8 PASSENGER ROUTE CHOICE Knowledge on passenger route choice provides Estimate demand for capacity Test assumptions on passenger behavior and route choice Hind-sight analysis of passenger service (delays) Until now: Surveys and panel data to deduce route choice Models for route choice: maximum utility, regret minimization, Now: We have both the Smart Card Data and conductor checks to determine the routes used by a passenger. Why not use them?

9 PROBLEM OVERVIEW ROUTE DEDUCTION FROM AFC Which route (time, space, trains) did a passenger take? Time +Station ci Conductor check Time +Station co Platform i Station A Platform k Station B trains ci time co

10 PROBLEM OVERVIEW ROUTE DEDUCTION FROM AFC Which route (time, space, trains) did a passenger take? Time +Station ci trains Platform i Station A Conductor Platform k Time +Station co check Step 1: How can we find these route options? Station B ci time co

11 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK 9:21 9:25 9:21 9:25 9:21 9:25

12 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK 9:21 9:25 9:21 9:25 9:21 9:25

13 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK Problem Transferring to another train is free. However, most passengers will prefer to stay in the same train if only a small 9:21 amount of time is saved. 9:25 9:21 9:25 9:21 9:25

14 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK

15 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK

16 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK D A D A

17 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK D A D A

18 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK D Problem Whether a transfer is feasible may depend on the platforms of the trains. A D A

19 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK D A 9:13 D A

20 FROM THE TIMETABLE TO AN EVENT ACTIVITY NETWORK D A 9:13 D A

21 TO SUMMARIZE We have a Basic Event Activity Network where transfers are free that contains an arc for each scheduled trip in the timetable We have a Extended Event Activity Network where we can include penalties on the transfers and make sure that some slack time is included when the passenger makes a transfer.

22 COMPUTING SHORTEST PATHS We use this procedure to obtain an Event-Activity Network G = (V, A) Every node in the graph has a time label If we assume every arc in the graph is associated with an action that takes a strictly positive amount of time, we have a Directed Acyclic Graph When constructing the graph, we can use the time indices to obtain a Topological Ordering of the graph V {v 1, v 2,, v n } such that v i, v j A i < j We can use this to compute a Shortest Path Tree in O( A ) time. For repeated computations on 20-50k Origin/Destination pairs, this matters. Using different cost parameters for the different types of arcs, we can generate paths that favor or avoid different types of routes.

23 PROBLEM OVERVIEW ROUTE DEDUCTION FROM AFC Now we have a set of possible routes. Time +Station ci Conductor check Time +Station co Platform i Station A Platform k Station B trains ci time co

24 PROBLEM OVERVIEW ROUTE DEDUCTION FROM AFC Now we have a set of possible routes. Time +Station ci Conductor check Step 2: Time +Station co Which path should we choose, given the check in and check out? Platform i Station A Platform k Station B trains ci time co

25 METHOD Generate routes based either on The basic Event-Activity Network The extended Event-Activity Network From these, we pick one according to a fixed rule: 1) First Departure (FD) 2) Earliest Arrival (EA) 3) Latest Arrival (LA) 4) Least Transfers (LT) 5) Maximum Path Length (MPL) 6) Select Least Transfers Last Arrival (STA) We will validate the methodology using conductor checks: Did we even find the correct route during route generation? Does our rule pick the correct route?

26 DATA Smart card data Origin station, destination station, start time, end time, card id Realized timetable Departure time station, arrival time station, train number Conductor checks Card id, time, train number General: 5 days Over 500,000 journeys, For a significant number of journeys, we have a conductor check Full Dutch railway network of Netherlands Railways trains

27 RESULTS Method Observed Path Generated Basic Network 75,5% Ext. Network 92,3% Rule Basic Network Ext. Network First Departure 65% 86% Earliest Arrival 67% 86% Last Arrival 65% 86% Least Transfers 68% 90% Max. Path Length 70% 92% Selected Least T. 73% 95% NB: These are percentages over the set of journeys for which the correct path was generated.

28 DISCUSSION OF RESULTS When constructing the Event-Activity Network, transfers matter for your succes rate. Rules solely based on either the arrival or departure time are outperformed by those which include the number of transfers. The Selected Least Transfers rule performs best, as it combines the idea of minimizing transfers with the idea that a passenger will likely depart within 10 minutes of check in and check out within 10 minutes of arrival.

29 MEASURING PASSENGER DELAYS

30 MEASURING SERVICE QUALITY Recall: punctuality score of NS is quite high (a little higher than 97%). This refers to train punctuality Did the train arrive within five minutes of the timetable? Passengers mostly care whether they reach their destination in time. Passenger punctuality would be a better indicator of service quality. In case of transfers, small train delays can have a big impact on the passenger delays. In some situations, delays can be lead to additional transfer opportunities How can we measure this? (MSc. Thesis of Timo Polman) Should be controllable (e.g. the cause can be determined), robust (e.g. not depend on shopping behavior at a station) and simple.

31 METHODOLOGY Smart Card Journey Generate planned route Planned Timetable Execute planned route Realised Timetable Calculate Delay

32 METHODOLOGY Smart Card Journey When the planned route is feasible, use that. If not, recalculate from the first point where it is infeasible. Generate planned route Planned Timetable Execute planned route Realised Timetable Calculate Delay

33 SOME DELAY MEASURES Average Delay (Gross Passenger Delay Minutes / Lost Customer Hours) Relative delay: Delay divided by planned journey time

34 SENSITIVITY TO SELECTION RULES Average Delay (Gross Passenger Delay Minutes / Lost Customer Hours) Relative delay: Delay divided by planned journey time Average Delay (min) Relative Delay Missed Connections Early Arr. First Dep. Last Arr. Least Transfers Sel. Least Transfers 2.23 min 2.23 min 2.22 min 2.13 min 1:45 min % 1.09% 1.10% 0.68% 0.39%

35 SENSITIVITY TO SELECTION RULES Check in Time Check out

36 METRICS BASED ON EARLY ARRIVAL Metric Percentage 5 minutes delayed 8.47% 10 minutes delayed 3.86% 15 minutes delayed 2.05% 30 minutes delayed 0.48% 45 minutes delayed 0.12% 60 minutes delayed 0.07% Metric Value Average Delay 1.93 min Average Travel Time min Average Relative Delay 9.44%

37 DISCUSSION The distinction between what did the passenger want to do? (related to stated choice) and what did the passenger do? (related to revealed choice) is important. Using the Earliest Arrival rule we are robust against late check outs due to for example shopping, but sensitive to strategic planning at the beginning of the journey (for example when early advice is given by smartphone)

38 DEMAND ANALYSIS

39 ACTIVITY BASED DEMAND Smart card data Trip-based Tour-based Activity-based models

40 FROM JOURNEYS TO ACTIVITY TIME INTERVALS Order the journey per individual smartcard according to time and date. An activity is detected if the destination and the origin of two consecutive journeys are equal The time and duration of the activity are based on the check out and the check in time of the consecutive journeys CardID Check in Check out Origin Destination /11 8:06 7/11 9:15 Utrecht Delft /11 22:03 7/11 23:15 Delft Utrecht /11 8:08 8/11 9:30 Utrecht Delft /11 15:16 5/11 15:45 Groningen Zwolle /11 9:15 6/11 10:15 Groningen Zwolle

41 FROM JOURNEYS TO ACTIVITY TIME INTERVALS Order the journey per individual smartcard according to time and date. An activity is detected if the destination and the origin of two consecutive journeys are equal The time and duration of the activity are based on the check out and the check in time of the consecutive journeys CardID Check in Check out Origin Destination /11 8:06 7/11 9:15 Utrecht Delft /11 22:03 7/11 23:15 Delft Utrecht /11 8:08 8/11 9:30 Utrecht Delft /11 15:16 5/11 15:45 Groningen Zwolle /11 9:15 6/11 10:15 Groningen Zwolle

42 FROM JOURNEYS TO ACTIVITY TIME INTERVALS Order the journey per individual smartcard according to time and date. An activity is detected if the destination and the origin of two consecutive journeys are equal The time and duration of the activity are based on the check out and the check in time of the consecutive journeys CardID Check in Check out Origin Destination /11 8:06 7/11 9:15 Utrecht Delft /11 22:03 7/11 23:15 Delft Utrecht /11 8:08 8/11 9:30 Utrecht Delft /11 15:16 5/11 15:45 Groningen Zwolle /11 9:15 6/11 10:15 Groningen Zwolle

43 FROM JOURNEYS TO ACTIVITIES AND TIME INTERVALS In order to simplify our data, we project the time intervals onto a modular ring consisting of 24 timeslots, generating a set of intervals per station. The time interval of an activity has a begin time x b and an end time x e on this modular ring The duration of a time interval is the clockwise distance between begin and end on the ring (and taken modulo 24) We define a parametric distance measure d θ for the distance between two time intervals based on three parameters θ = (θ 1, θ 2, θ 3 ) as follows: d θ x, y = θ 1 x d y 2 d if x b = y b or x e = y e θ 2 x b y 2 b if x d = y d θ 3 x b y b + x d y d 2 otherwise Partition our observations into k sets using the k-means algorithm.

44 k-means CLUSTERING WITH k = 3, θ 1 = 1, θ 2 = 1, θ 3 = 2 Data Entry Start End Cluster Centroids Cluster Begin End

45 DATA AND ROBUSTNESS FRACTIONS Trick for speed up: if a certain time interval occurs more often, we can group them as a single datapoint and adapt the distance function accordingly. Why k-means? It is very fast on large datasets (assigning clusters takes O(nk) time, calculating a centroid is a bit more involved but still efficient enough) We repeat the k-means algorithm for the intervals at a station with: Different random seeds Different settings for the parameters Different values for k For each centroid we calculate how often it occurs in the final output of each run of the algorithm. The average of this frequency over all stations is the Robustness Fraction

46 OUTPUT: ROBUSTNESS FRACTIONS NB: This is based on urban public transport data, not Dutch Railways

47 LABELLING PROCEDURE Looking at the original chains of activities per individual card in the smart card data, we can easily construct chains of time intervals. We label these time intervals according to a labelling procedure inspired by the observed robustness fractions. We will then analyze the frequencies of pairs and triplets of consecutive labels. Duration Time of Day

48 CONSECUTIVE PAIRS OCCURRING FREQUENCIES

49 CONSECUTIVE TRIPLETS OCCURRING FREQUENCIES

50 CONCLUSIONS AND FUTURE WORK The most prominent time patterns observed are associated with home-work travel patterns, but we are also able to detect some less obvious patterns. For now, our method is still quite crude and only serves exploratory analysis. Interesting opportunities for future research: Can we automatically construct labelling rules from our clustering output? Can we also investigate spatial usage patterns using similar methods?

51 CONCLUDING REMARKS

52 CONCLUDING REMARKS Although we have a lot of data, in order to generate valuable insights we need to have a thorough understanding of the underlying processes.

53 CONCLUDING REMARKS Although we have a lot of data, in order to generate valuable insights we need to have a thorough understanding of the underlying processes. By itself, the stream of smart card transactions is not enough to gain this understanding. For route choice, we need to validate our rules using an additional data set collected by the conductors. For passenger punctuality and demand prediction, we need to understand the how passengers plan their journeys.

54 CONCLUDING REMARKS Although we have a lot of data, in order to generate valuable insights we need to have a thorough understanding of the underlying processes. By itself, the stream of smart card transactions is not enough to gain this understanding. For route choice, we need to validate our rules using an additional data set collected by the conductors. For passenger punctuality and demand prediction, we need to understand the how passengers plan their journeys. Information Systems + Human Behavior = Lots of Research Opportunities

55 QUESTIONS? Questions? Suggestions? Thanks for your attention!