---

Project Summary

Web-based network throughput tests, also known as speed tests, are the most common (often the only) way that typical end users can measure their own broadband performance. As such, they have become important not just for end users, but also for regulators and researchers seeking to validate claims about broadband performance. Speed test services report basic metrics, including latency and average download/upload throughput of a bulk data transfer between users and one or more speed test servers. But the lack of standard approaches to test server deployment, implementation, configuration and even metrics measured might result in misalignment with users’ measurement goals. Uneven server deployment and suboptimal server selection can distort measurement results. And while all speed test platforms use the same brute force approach to bandwidth estimation, they choose their own test parameters and other proprietary implementation details, making it impossible to effectively compare results across platforms.

Tasks

The RABBITS toolkit will be an open-source and deployable Internet measurement data collection tool that facilitates scientific and reproducible Internet broadband performance measurements using speed test infrastructure. We structure the RABBITS toolkit into two modules: RABBITS-Mapper and RABBITS-Perf.

RABBITS-Mapper (Task 1) is a novel tool that gathers macroscopic data about the architecture of speed test infrastructure and collects information on geographical and network locations of speed test servers, which is essential for studying the network paths and latency from VPs to test servers. Also, we will conduct measurements from VPs in CAIDA’s Ark, RIPE Atlas, and Edgenet and running smartphones in packages to understand where test servers are located and how end-users connect to these servers in both wired and cellular networks. The measurement results will be valuable for mapping the accessibility of speed test infrastructures to end-users.

Different speed test implementations can report inconsistent, sometimes conflicting, results to users. These tests provide neither control knobs to adjust measurement parameters nor detailed data for users to analyze. In Task 2 we will design and implement RABBITS-Perf to overcome the challenge of obtaining comparable measurement results across speed test platforms. RABBITSPerf will enhance WebTestKit to provide capabilities to control 1) the number of concurrent measurement flows, 2) the size of HTTP transactions, and 3) server selection. To allow scientific comparisons of measurement results, we will design a unified method to record fine-grained data during the tests, instead of relying on metrics reported by individual tests.

Project Timeline

445 Description
792 Task 1: RABBITS-Mapper: Mapping speed test infrastructure
1.1	Discovering speed test servers with RABBITS-Mapper
1.2	Deploying low-overhead experiments to collect network path and latency to speed test servers
1.3	Preliminary results
493 Task 2: RABBITS-Perf: Unifying cross-platform speed test measurements
2.1	Analyzing HTTP transactions in speed test implementations
2.2	Providing control knobs to speed tests with RABBITS-Perf
2.3	Designing a standardized format for reporting speed test measurements
2.4	Preliminary results

Acknowledgment of awarding agency’s support

This material is based on research sponsored by the National Science Foundation (NSF) grant CNS-2323219. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of NSF.