A Framework for Big Data in Urban Mobility and Movement Patterns Analysis

Novel large consumer datasets (called ‘Big Data’) are increasingly readily available. These datasets are typically created for a particular purpose, and as such are skewed, and further do not have the broad spectrum of attributes required for their wider application. Railway ticket data are an example of consumer data, which often have little or no supplementary information about the passengers who purchase them, or the context in which the ticket was used (like crowding-level in the train). These gaps in consumer data present challenges in using these data for planning, and inference on the drivers of mobility choice. Heckman’s in-depth discussion of ‘sample selection’ bias and ‘omitted variables’ bias (Heckman, 1977), and Rubin’s seminal paper on ‘missing values’ (Rubin, 1976) laid the framework for addressing omitted variables and missing data problems today. On the strength of these, a powerful set of complementary concerted methodologies are developed to harness railways consumer (ticketing) data. A novel spatial microsimulation methodology suitable for skewed interaction data was developed to combine LENNON ticketing, National Rail Travel Survey, and Census interaction data, to yield an attribute-rich micro-population. The micro-population was used as input to a GIS network, logistically constrained by the transit feed specification (GTFS). This identifies the context of passenger mobility. Bayesian models then enable the identification of passenger behaviour, like missing daily trip rates with season tickets, and flows to group stations. Case studies using the micro-level synthetic data reveal a mechanism of rail-heading phenomena in West Yorkshire, and the impact of a new station at Kirkstall Forge. The spatial microsimulation and GIS-GTFS methods are potentially useful to network operators for the management and maintenance on the railways. The representativeness of the micro-level population created has the potential to alter multi-agent transport simulation genres, by precluding the need for the complexities of utility-maximizing traffic assignment.