Trailer Video
Description
This is a college project I did alongside my wonderful classmates for the third year of my degree in Data Science and AI at UCM.
The task that was given to us was to predict taxi and FHV demand and tips in specific zones of New York, with the data from the TLC (Taxi and Limousine Commission).
We came up with a user-first mobile application for accessing the data provided by the models, including a trip recommender.
Development repository: https://github.com/danpanto/NyFlow
Technical details
My job as part of the team has ranged from helping with the processing pipeline, using the Polars Python library, to implementing geographic visualization with a webapp built from scratch.
Another task I had was to create the models for predicting the tip of a specific trip, based on vendor, origin, destination and base fare.
For this task, we face a statistical challenge: a large portion of rides result in exactly $0 in tips (zero-inflated data). Standard regression models struggle with this excess of zeros. The Hurdle Model solves this by splitting the problem into two distinct stages (or “hurdles”): it distinguish between the probability of receiving a tip, P(Y > 0), and the expected tip amount if a tip is given, E(Y | Y > 0). By combining both parts using the Law of Total Expectation, we get our final tip prediction for any given trip.
As for the architecture, it was based on LightGBM.
The final RMSE was around $2.57. Note that in a real scenario, likely some of the inputs would not be available (destination and fare), which would lead to worse results and maybe simpler models would fit better in this case.
Co-Authors
- Daniel Pantoja Joaristi
- Jorge Hernández Palop
- Marco Casteleiro Trigueros
- Álvaro Arbona Rodríguez