Logo

Getting Started: Ark Integrated Active Measurement Programming Environment

Getting Started: Ark Integrated Active Measurement Programming Environment

We document the on-demand measurement service of CAIDA's Archipelago (Ark) infrastructure.

Introduction

CAIDA provides an easy way for researchers to conduct on-demand measurements on the Ark platform based around the scamper Python module. This platform provides the ability to conduct ping, traceroute, DNS lookups, HTTP, UDP, and alias resolution measurements. Vetted academic researchers will be granted shell access to a CAIDA system that provides the capability to conduct measurements and receive responses in near real-time. If you are an academic researcher who would like access, please contact

Getting started with the platform

You will be provided shell access to system named ark-mux.caida.org. To log in, the system requires you to use the SSH key that you provided during the onboarding process, and the username that we advised (which will usually be the one you supplied when you provided your SSH key). If your username is foo, we suggest that you create an entry in .ssh/config along the lines of:

Host ark-mux
 HostName ark-mux.caida.org
 User foo

You can then reach the system with ssh ark-mux instead of using ssh foo@ark-mux.caida.org.

The system has various text editors and Python modules installed. Please ask us to install any Python modules and software through the available Debian package, rather than install the software to your home directory.

We support the use of Visual Studio code, which has good support for remote development using SSH and Python. When you have installed VS code on your local system, follow the instructions in the official documentation. If you do not have the Remote-SSH extension installed, use Ctrl+Shift+X to search for the Remote-SSH extension, and install it. Once installed, use Ctrl+Shift+P and select Remote-SSH: Connect to Host... in the drop-down. VS code will provide a list of hosts in your local SSH configuration. If you added an ark-mux entry to your .ssh/config file, then there will be an entry for ark-mux that you can click on.

Getting started with the Python module

The module is extensively documented with examples in the scamper python module documentation on CAIDA’s website. The module supports different methods for interacting with vantage points (VPs) running scamper; we support the multiplexed (mux) interface on Ark, which provides access to all of the VPs that are listed publicly and operational. See Ark monitor locations for the publicly listed VPs.

On CAIDA’s systems, we provide the mux interface at /run/ark/mux.

The following example implements the well-known shortest-ping measurement technique, which conducts delay measurements to an IP address from a distributed set of VPs, and reports the shortest of all the observed delays with the name of the VP that observed the delay. If the script was saved as shortest-ping.py, then you could run this script using python3 shortest-ping.py /run/ark/mux 192.172.226.122

import sys
from datetime import timedelta
from scamper import ScamperCtrl

if len(sys.argv) != 3:
  print("usage: shortest-ping.py $mux $ip")
  sys.exit(-1)

with ScamperCtrl(mux=sys.argv[1]) as ctrl:
  ctrl.add_vps(ctrl.vps())
  for i in ctrl.instances():
    ctrl.do_ping(sys.argv[2], inst=i)

  min_rtt = None
  min_vp = None
  for o in ctrl.responses(timeout=timedelta(seconds=10)):
    if o.min_rtt is not None and (min_rtt is None or min_rtt > o.min_rtt):
      min_rtt = o.min_rtt
      min_vp = o.inst

  if min_rtt is not None:
    print(f"{min_vp.name} {(min_rtt.total_seconds()*1000):.1f} ms")
  else:
    print(f"no responses for {sys.argv[2]}")

Vantage point metadata

Different Ark VPs provide different measurement capabilities according to the preference of the VP’s host. Currently, all VPs support ping and traceroute measurements. Most also support DNS, UDP probes, and HTTP measurements. If a script attempts to use a measurement primitive that is not supported by the VP, the module will raise a Python exception. You can determine the supported measurements for each VP before conducting a measurement by examining the tags reported by the module for each VP. Current tags are:

  • primitive:dns: the VP supports DNS queries
  • primitive:http: the VP supports HTTP queries
  • primitive:udp: the VP supports UDP probes

To get a list of VPs that have the ability to send DNS queries:

with ScamperCtrl(mux='/run/ark/mux') as ctrl:
  vps = [vp for vp in ctrl.vps() if 'primitive:dns' in vp.tags]

We also provide network, hardware, and operating system properties for each VP as tags. Current tags are:

  • network:ipv4: the VP has working IPv4 connectivity.
  • network:ipv6: the VP has working IPv6 connectivity.
  • network:nat4: the VP has IPv4 connectivity via network address translation.
  • network:nat6: the VP has IPv6 connectivity via network address translation.
  • hardware:amd64: the VP is using X86 64-bit userspace.
  • hardware:i386: the VP is using X86 32-bit userspace.
  • hardware:arm64: the VP is using 64-bit ARM userspace.
  • hardware:armhf: the VP is using 32-bit ARM userspace.
  • hardware:physical: the VP is using dedicated hardware.
  • hardware:container: the VP is running as a container.
  • hardware:vm: the VP is running as a virtual machine on shared hardware.
  • hardware:virtual: either hardware:container or hardware:vm.
  • os:debian: the VP is running a Debian or Raspbian operating system.
  • os:debian_bullseye: the VP is running a Bullseye-based Debian operating system.
  • os:debian_bookworm: the VP is running a Bookworm-based Debian operating system.
  • os:debian_trixie: the VP is running a Trixie-based Debian operating system.
  • os:ubuntu: the VP is running an Ubuntu operating system.
  • os:ubuntu_20.04: the VP is running Ubuntu 20.04.
  • os:ubuntu_22.04: the VP is running Ubuntu 22.04.
  • os:ubuntu_24.04: the VP is running Ubuntu 24.04.

Note that the OS tags signal what version of OpenSSL is available to scamper – Bullseye has at least 1.1.1w, Bookworm 3.0.17, and Trixie 3.5.1.

Beyond these tags, each VP will also provide approximate location information via other properties as documented.

Experiments this environment has supported

  • University of Twente’s anycast census. Results updated daily in Anycast Census GitHub.

  • Analysis of nameserver ECS deployments. Results published in CoNEXT 2025.

  • Locating and Enumerating anycast. Results published in ANRW 2025.

Questions about the programming environment?

Please send questions or comments regarding the environment to

Published
Last Modified