Residential Internet usage has rapidly increased and morphed, making it crucial to thoroughly understand which services users access and how they are delivered. However, in general, the community lacks access to open residential traces.
This project collects a new, sharable dataset capturing real-world Internet traffic from the off-campus residential buildings of Columbia University. These residential units are not undergraduate dorms, but rather house graduate students, postdocs, faculty, and their families. We are continuing our data collection campaign and plan to scale up.
(We have been collecting data from hundreds of apartments since 2022. Updated since May 2025: we have scaled up to 24/7 collection for 1,500 residential units.)
We present a detailed, broad view of Internet service usage and delivery for this residential network: we develop methodologies to associate traffic to services and investigate the serving infrastructure and services accessed by our users.
Shuyue Yu, Thomas Koch, Ilgar Mammadov, Hangpu Cao, Gil Zussman, and Ethan Katz-Bassett. 2025. Internet Service Usage and Delivery As Seen From a Residential Network. Proc. ACM Meas. Anal. Comput. Syst. 9, 2, Article 41 (June 2025), 30 pages. https://doi.org/10.1145/3727133
Shuyue Yu, Thomas Koch, Ilgar Mammadov, Hangpu Cao, Gil Zussman, and Ethan Katz-Bassett. 2025. Internet Service Usage and Delivery As Seen From a Residential Network. In Abstracts of the 2025 ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS Abstracts ’25), June 9–13, 2025, Stony Brook, NY, USA. ACM, New York, NY, USA, 3 pages. https://doi.org/10.1145/3726854.3727327
Shuyue Yu, Thomas Koch, Ilgar Mammadov, Hangpu Cao, Gil Zussman, and Ethan Katz-Bassett. 2025. Poster: Collection and Sharing of A Residential Dataset. In Proceedings of the 2025 ACM Internet Measurement Conference (IMC ’25), October 28–31, 2025, Madison, WI, USA. ACM, New York, NY, USA, 2 pages. https://doi.org/10.1145/3730567.3768615
Thomas Koch, Shuyue Yu, Sharad Agarwal, Ryan Beckett, and Ethan KatzBassett. 2023. PAINTER: Ingress Traffic Engineering and Routing for Enterprise Cloud Networks. In ACM SIGCOMM 2023 Conference (ACM SIGCOMM ’23), September 10, 2023, New York, NY, USA. ACM, New York, NY, USA, 18 pages. https://doi.org/10.1145/3603269.3604868
Jiangchen Zhu, Tom Koch, Ilgar Mammadov, Shuyue Yu, Kevin Vermeulen, Matt Calder, Italo Cunha, and Ethan Katz-Bassett. 2025. The New (Pareto) Frontier of Cloud Routing: High Availability, Precise Control, or Con- figuration Stability — Choose Two. Proc. ACM Netw. 3, CoNEXT4, Article 28 (December 2025), 25 pages. https://doi.org/10.1145/3768975
Kahlil Dozier, Loqman Salamatian, and Dan Rubenstein. 2024. Analysis of False Negative Rates for Recycling Bloom Filters (Yes, They Happen!). Proc. ACM Meas. Anal. Comput. Syst. 8, 2, Article 21 (June 2024), 34 pages. https://doi.org/10.1145/3656005