Skip to Content
[CAIDA - Cooperative Association for Internet Data Analysis logo]
The Cooperative Association for Internet Data Analysis
www.caida.org > data : active : ipv4_routed_24_topology_dataset.xml
The IPv4 Routed /24 Topology Dataset

|  Data Sources:    Realtime Monitors    Passive    Active    Other    External  |


The IPv4 Routed /24 Topology Dataset


This dataset contains information useful for studying the topology of the Internet. Data is collected by a globally distributed set of Ark monitors. The monitors use team-probing to distribute the work of probing the destinations among the available monitors.

We collect data by sending scamper probes continuously to destination IP addresses. Destinations are selected randomly from each routed IPv4 /24 prefix on the Internet such that a random address in each prefix is probed approximately every 48 hours (one probing cycle). Because team-probing distributes the probing work across all monitors, a single destination /24 will be probed by only one monitor in each probing cycle. The current list of routed IPv4 prefixes was created using RouteViews BGP tables from December 2010. Rather than having a static list of IP addresses to probe, we dynamically pick a new random address in each /24 prefix for every new cycle of probing. The current prefix list includes approximately 9.04 million prefixes (with 6.5 million for data collected before November 2007). Scamper:

Measures Forward IPv4 Paths
scamper records an IPv4 address seen at each hop from a source to a destination by incrementing the "time to live" (TTL) of each IPv4 packet header, and recording replies from each router leading to the destination host.
Measures Round Trip Times (RTT)
scamper collects round trip time measured to each intermediate router as well as to the destination host.

In the current configuration, scamper probes with ICMP packets, using the Paris traceroute technique (ICMP-paris) to improve measurement integrity across load-balanced links. Data prior to November 2007 used an alternate UDP traceroute method. Data collected for each path probed includes:

  • RTTs, including both intermediate hops and the destination
  • IPID, TOS, and TTL, and size fields of response packets
  • IP length, TTL, and TOS fields of the probe packet that reached each hop (extracted from the response packet)
  • The ICMP type and code of responses
Scamper also is able to collect Path MTU information, but current measurements do not include that information. A sample binary warts file is available.

Data has been collected continuously since September 13, 2007, and is made available in hour-duration files for the most recent ten days, as well as a historical archive of 24-hour-duration files for the duration of IPv4 Routed /24 Topology project.

Caveats that apply to this dataset:

  • Unlike CAIDA's previous skitter macroscopic topology measurements, IPv4 Routed /24 Topology Dataset uses a dynamic destination list. Measurements to consistent IPv4 addresses are not available in this dataset.
  • Because team-probing distributes measurements across many monitors, the randomly selected IP addresses in a given routed prefix are not probed by the same set of monitors consistently over time.
  • Scamper outputs traces in a different file format than skitter.

Reading Topology Data

You can analyze this data (available in the warts format) with the sc_analysis_dump tool included in the scamper distribution package. The sc_analysis_dump tool prints out information about each trace in an easy-to-parse textual format (one trace per line). You would typically write a perl script to analyze the output of sc_analysis_dump.

Another tool you may want to consider is the warts-dump tool, which is also included in the scamper distribution. The output of warts-dump is somewhat less easy to parse, but warts-dump prints out practically all information contained in a warts file.

Finally, you can write your analysis scripts in the Ruby language using rb-wartslib, an easy-to-use Ruby binding to the warts I/O library.

Data Use Restrictions

Acceptable Use Policy for the files of the IPv4 Routed /24 Topology Dataset

  1. Macroscopic Topology data will not be distributed beyond authorized users.
  2. I will notify CAIDA of the names and email addresses of any persons (and their respective affiliations) assisting me in research using the macroscopic topology data. This includes graduate students and interns.
  3. At the end of the research, or semi-annually (which ever is more frequent), a summary of the research and any findings/conclusions will be reported to CAIDA. If any research is described on the WWW, a URL will be provided. This information is primarily used in reports to our funding agencies.
  4. In so far as possible, research findings and conclusions using the topology data will be published and/or made publicly available
  5. All users who publish a document (including web pages and papers) using data from a IPv4 Routed /24 Topology Dataset must provide CAIDA with a copy of the publication.
  6. All users who publish a document (including web pages, and papers) using data from this dataset must provide CAIDA with a copy of the publication and must cite:

    The CAIDA IPv4 Routed /24 Topology Dataset - < dates used >, Young Hyun, Bradley Huffaker, Dan Andersen, Emile Aben, Colleen Shannon, Matthew Luckie, and kc claffy http://www.caida.org/data/active/ipv4_routed_24_topology_dataset.xml.

  7. Users are encouraged, but not required, to include the following attribution in the acknowledgments section of their document:

    Support for the IPv4 Routed /24 Topology Dataset is provided by the National Science Foundation, the US Department of Homeland Security, the WIDE Project, Cisco Systems, and CAIDA Members.

  8. All users who create a publicly available presentation using data from this dataset must provide CAIDA with a copy of the presentation and must use the full name of the dataset ("The CAIDA IPv4 Routed /24 Topology Dataset") in the presentation. Users are further encouraged, but not required, to include the URL for the dataset (http://www.caida.org/data/active/ipv4_routed_24_topology_dataset.xml) in their presentation.

IPv4 Routed /24 Topology Dataset Access

Request Access to IPv4 Routed /24 Topology Data

Other Topology Datasets:

References

For more information on topology measurements see:

For more information on the Archipelago (Ark) Measurement Infrastructure, see:

Acknowledgments

Special thanks to Matthew Luckie for development of and assistance with scamper.

The IPv4 Routed /24 Topology Dataset was sponsored by:


National Science Foundation logo Department of Homeland Security logo WIDE Project logo Cisco Systems logo


  Last Modified: Tues Mar-1-2011 14:7:0 PDT
  Page URL: http://www.caida.org/data/active/ipv4_routed_24_topology_dataset.xml