BUILDING A CITY-SCALE MOBILITY DATA PIPELINE FOR PASSENGER CROWDING, VEHICLE MOVEMENT AND PUBLIC TRANSPORT DEMAND INTELLIGENCE - (TRANSITSENSE)

Abstract

Public transport agencies in large Indian cities operate thousands of buses and metro trains every day, yet most of them still plan services using fixed timetables and occasional manual passenger surveys. The data needed for better decisions already exists in scattered systems: GPS units on vehicles report positions every few seconds, electronic ticketing machines and smart card validators record every boarding, and automatic passenger counters on some vehicles log entries and exits at each stop. This paper presents TransitSense, a city-scale mobility data pipeline that brings these sources together to measure passenger crowding, track vehicle movement and forecast public transport demand in near real time. The pipeline ingests vehicle location feeds, ticketing transactions, passenger counter records and the published GTFS schedule through a message broker, and processes them in both streaming and batch modes. Streaming jobs match each GPS ping to its route and trip, compute delays against the schedule and estimate the current load of every vehicle by combining boardings from ticket data with alightings inferred from typical travel patterns. Batch jobs build a cleaned and partitioned data lake organised in raw, refined and curated layers, which holds stop-level and hour-level histories of ridership, occupancy and running times. On top of the curated layer, TransitSense trains machine learning models for three tasks. A gradient boosting model forecasts boardings at each stop for the next one to three hours using calendar, weather and recent demand features. A crowding classifier assigns every upcoming trip to a comfort level of low, moderate, crowded or overloaded. A travel time model predicts arrival times at downstream stops from live positions and historical segment speeds. Origin and destination matrices are also estimated from smart card tap sequences to show how passengers actually move across the network.