Archipelago (Ark) Measurement Infrastructure

What is Ark?

Archipelago (Ark) is CAIDA's globally distributed active measurement infrastructure for Internet topology research, serving the network research community since 2007. CAIDA grows the infrastructure by deploying vantage-point hardware — typically Raspberry Pi systems — with as much geographical and topological diversity as possible; in 2024 we began to support software-based vantage points as well. Ark runs canonical background measurements that produce CAIDA's flagship topology datasets, and supports vetted experiments on a security-hardened, coordinated platform.

An Ark node is a single measurement point, also called a vantage point or a monitor; the three terms mean the same thing. Each node is a small computer such as a Raspberry Pi, or software running on a server, hosted by an organization or volunteer.

Ark is tailored for distributed measurements that require communication and coordination. We support vetted measurement experiments, with the goal of lowering the barrier to testing and evaluating novel and interesting measurement techniques. We list current and recent measurement experiments below, and archive a list of historical measurements and experiments.

Researchers can:

• Run vetted, coordinated experiments on a security-hardened, globally distributed platform — see the Getting Started guide for the scamper Python module and shell access.
• Conduct adhoc ping, traceroute, DNS, HTTP, UDP, and alias-resolution measurements through the scamper Python module (shell access granted to vetted academic researchers).
• Access Ark-generated datasets and tools and papers using Ark data through the CAIDA Resource Catalog.

Operators can:

• Host an Ark vantage point (typically a Raspberry Pi or, since 2024, a software-based instance) — start with Why should my network host an Ark vantage point? (also available en Español).
• Submit a hosting request via the Ark monitor request form; CAIDA's team handles deployment follow-up.
• Review the hosting FAQ before committing.

Current Ark measurement experiments

1. 
   Command and control between Spoofer server and Ark vantage points.

   The Spoofer Project: Ark vantage points participating in the Spoofer Project help measure the Internet's susceptibility to spoofed source address IP packets. The vantage points gather data on IP spoofing by receiving potentially spoofed traffic and forward it on to the Spoofer Project's server for analysis. The MANRS Observatory depends on these Spoofer measurements.

2. Internet Topology Discovery: We use geographically distributed Ark vantage points to conduct coordinated, large-scale traceroute-based topology measurements.

   • IPv4 team probing: We probe the full IPv4 routed address space from a team of nearly 400 vantage points, probing over 10 million /24's (that is, the full routed address space subdivided into /24's). We divide the probing work among the vantage points to conduct coordinated, large-scale traceroute-based topology measurements so that we use the vantage points as a team. The Ark IPv4 Routed /24 Topology Dataset.

   • IPv4 prefix probing: We probe the full IPv4 routed address space from a subset of Ark monitors. Each monitor probes the entire set of targets independently, and each monitor aims to complete one pass of the target set every calendar day. The Ark IPv4 Prefix-Probing Traceroute Dataset.

   • IPv6: We probe all announced IPv6 prefixes (/48 or less specific) once every 24 hours from each vantage point that has an IPv6 address. The Ark IPv6 Topology Dataset.

   We distribute the results of these measurements as well as others in various topology related datasets.

   Approximately twice a year, we produce CAIDA's flagship Macroscopic Internet Topology Data Kit (ITDK), which builds a router-level graph from our raw traceroute topology data using alias resolution, infers the autonomous system (AS) operating each router, the locations and vendors of routers, and rules to extract information from DNS records associated with those routers.

3. Anycast Census: The University of Twente's LACeS Anycast census has used the Ark platform every day beginning in March 2024. The vantage points provide delay constraints that allow the LACeS system to infer likely locations of anycast sites. The data is publicly available.

4. Adhoc Measurements: CAIDA provides an easy way for researchers to conduct adhoc measurements on the Ark platform based around the scamper Python module. This platform provides the ability to conduct ping, traceroute, DNS lookups, HTTP, UDP, and alias resolution measurements. Vetted academic researchers will be granted shell access to a CAIDA system that provides the capability to conduct measurements and receive responses in near real-time. We provide documentation that describes how to get started.

For a complete list of CAIDA data, please see the CAIDA Data Overview page.

Getting Started with Ark

After you acknowledge the Ark Memorandum of Understanding (MOU) and submit the request to host an Ark monitor, the CAIDA Ark team will follow up with you. Once your node is running, you can begin using the Ark programming environment to run active Internet measurements. The Getting Started page explains how to access your measurement node and use the Python module built on scamper to issue probes, and also includes an example script to help hosts to launch their first on-demand measurement.

Datasets produced by Ark

The CAIDA Resource Catalog lists collection of Ark tools and datasets generated from the Ark infrastructure, as well as published papers that made use of Ark-related data sets.

Funding Support

Support for the Archipelago Measurement Infrastructure project is provided by the Defense Advanced Research Projects Agency (DARPA) cooperative agreement HR00112020014 Performance Evaluation Network Measurements and Analytics, the Department of Homeland Security (DHS) cooperative agreement FA8750-18-2-0049 Advancing Scientific Study of Internet Security Topological Stability, S&T contract D07PC75579 Supporting Research Development of Security Technologies through Network Security Data Collection, S&T contract HHSP 233201600012C Science of Internet Security: Technology Experimental Research, S&T contract NBCHC070133 Supporting Research Development of Security Technologies through Network Security Data Collection, and S&T cooperative agreement FA8750-12-2-0326 Supporting Research and Development of Security Technologies through Network and Security Data Collection, and the National Science Foundation (NSF) grants CNS-0958547 Internet Laboratory for Empirical Network Science, CNS-1513283 Internet Laboratory for Empirical Network Science: Next Phase, CNS-1901517 Strategies for Large-Scale IPv6 Active Mapping, CNS-1925729 Facilitating Advances in Network Topology Analysis, CNS-2120399 Integrated Library for Advancing Network Data Science, OAC-1724853 Integrated Platform for Applied Network Data Analysis, and OAC-2131987 Designing a Global Measurement Infrastructure to Improve Internet Security. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DARPA, DHS, NSF, or the U.S. Government.

Questions about Ark?

See our Frequently Asked Questions for sites interested in hosting an Ark vantage point to see common questions, but please send questions or comments regarding Ark to ark-info@caida.org if they haven't been addressed by the FAQ.