Periscope Looking Glass API

CAIDA's Periscope Looking Glass API is a middleware interface to looking glass and scamper servers, with with options to retrieve measurements status, search by type of measurement, and create new measurements.

General

CAIDA's Periscope API provides a uniform interface to hundreds of Looking Glass and Scamper servers with access to thousands of network probing vantage points ("monitors") that can perform traceroute and bgp queries.

New 2022-06: The Periscope API has been enhanced with new features, but all changes are backward compatible.

  • New API URL: the new API URL is https://api.periscope.caida.org/v2. The old URL https://periscope.caida.org/api/v2 will continue to work for now, but you are encouraged to update your code.
  • Scamper-routeviews: Periscope now implements the "traceroute" command on Scamper servers running on Routeviews collectors.
  • New "type" field: a monitor's type may be "LG" (looking glass) or "SCRV" (scamper-routeviews).
  • New "server" field: server names are now explicit; a (server name, monitor name) tuple uniquely identifies a monitor, whereas (asn, monitor name) did not necessarily. A single server may provide access to monitors on multiple ASNs.

All requests to the Periscope API must be made via HTTP[S] GET and POST requests. All responses to API requests are in JSON format. Responses to successful requests will include an HTTP response code in the range 200-299. Responses to failed requests will include an HTTP response code outside that range, and a body that is a JSON object with the attribute "errors" containing a list of human-readable error messages.

All measurement data collected by Periscope are public and therefore API requests that query the results of measurements (GET requests) do not require authentication, and may be made by HTTP or HTTPS.

Requests to create measurements require a POST request over HTTPS (not HTTP), and authentication with HMAC SHA-256 (Hash Message Authentication Code). Each authorized user is issued a public key and a private key. The private key is used to calculate the HMAC string of the request body. If the request authenticaion fails, the API returns a HTTP 401 error. POST requests must contain the following HTTP headers:

  • Content-type: "application/json; charset=utf-8" (other charset values will also work; just make sure your content matches. Non-ASCII characters occur in Periscope data only in city names, and then only rarely.)
  • X-Public: public key
  • X-Hash: base64-encoded HMAC-SHA-256 signature of request body

The Code samples section provides Python and PHP examples on how to perform the authentication.

To obtain a public and a private key please Request a Periscope account.



Get the available monitors

API Endpoint: https://api.periscope.caida.org/v2/host/list
HTTP Method: GET
Parameters:
Name Type Value Notes
command string command that the monitor must allow REQUIRED
server string server name optional
type string "LG" or "SCRV" optional
asn string monitor's Autonomous System number optional
router string monitor's name optional
city string monitor's city optional
country string monitor's country (ISO-3166-1 alpha 2) optional

A GET request to this endpoint returns a JSON-encoded array of objects describing the available monitors matching the parameters. Note that monitors are sometimes called "routers" or "hosts". The monitor objects have the following attributes:

  • server [string]: The server name
  • type [string]: "LG" (looking glass) or "SCRV" (scamper-routeviews)
  • asn [integer]: The Autonomous System Number of the monitor
  • router [string or integer]: Usually, the name of the monitor, unique among monitors with the same server; but may be the integer 0 if this is the only monitor offered by a looking glass server, and the server does not provide a name for the monitor
  • label [optional string]: A description of the monitor that may be more informative than just the name
  • city [optional string]: The city where the monitor is located, if known
  • country [optional string]: The country (in ISO-3166-1 alpha-2 format) where the monitor is located, if known

When the server provides the city and the country of the monitor we use this information, otherwise we infer the monitor location based on geographical DNS hints in the first hop, or the NetAcuity geolocation database. Therefore the geolocation may not always be accurate.

Example

Request:
https://api.periscope.caida.org/v2/host/list?command=traceroute&country=dk

Response:

  
    [
	{
	    "server": "lg.example.net",
	    "type": "LG",
	    "asn": 64496,
	    "router": "cop01",
	    "label": "Copenhagen router 01",
	    "city": "Copenhagen",
	    "country": "DK"
	},
	{
	    "server": "route-views.example.routeviews.org",
	    "type": "SCRV",
	    "asn": 64511,
	    "router": "192.168.100.42",
	    "label": "Example Net, DK",
	    "city": "Copenhagen",
	    "country": "DK"
	},
	{
	    "server": "lg.foonet.net",
	    "type": "LG",
	    "asn": 65551,
	    "router": 0,
	    "city": "Copenhagen",
	    "country": "DK"
	}
    ]
  



Get the status of a measurement

API Endpoint: https://api.periscope.caida.org/v2/measurement/id
HTTP Method: GET

A GET request to this endpoint returns a JSON-encoded object with the status of the measurement with id id, with the following attributes:

  • id [integer]: The unique identifier of the measurement (same as the id parameter)
  • argument [string]: The IP address used as target in this measurement's queries
  • command [string]: The command used in this measurement's queries.
  • name [string]: The name of the measurement specified when the measurement was created.
  • timestamp [string]: The date and time when the measurement was created.
  • queries [integer]: The number of queries requested as part of this measurement
  • status: object with the following attributes:
    • completed [integer]: number of completed queries
    • failed [integer]: number of failed queries
    • pending [integer]: number of queries still in progress

Example

Request:
https://api.periscope.caida.org/v2/measurement/1230021

Response:

  
    {
	"id":1230021,
	"argument":"198.51.100.7",
	"command":"traceroute",
	"name":"test",
	"timestamp":"2018-12-05 12:33:11",
	"queries":4,
	"status":{"completed":4, "failed":0, "pending":0}
    }
  



API Endpoint: https://api.periscope.caida.org/v2/measurement/find?name=name&command=command&argument=argument
HTTP Method: GET
Parameters:
Name Type Value Notes
name string The user-assigned name of the measurement optional
command string "traceroute" | "bgp" optional
argument string The target of the measurement optional

To search for specific measurements send a GET request to https://api.periscope.caida.org/v2/measurement/find
All parameters are optional but at least one is required.

The response is a JSON array of measurement status objects.

Example

Request:
https://api.periscope.caida.org/v2/measurement/find?name=test

Response:

    
      [
	{
	  "id":1230021,
	  "argument":"198.51.100.7",
	  "command":"traceroute",
	  "name":"test",
	  "timestamp":"2018-12-05 12:33:11",
	  "queries":4,
	  "status":{"completed":4, "failed":0, "pending":0}
	},
	{
	  "id":1230022,
	  "argument":"203.0.113.22",
	  "command":"bgp",
	  "name":"test",
	  "timestamp":"2018-12-05 13:21:47",
	  "queries":5,
	  "status":{"completed":5, "failed":1, "pending":0}
	},
	{
	  "id":1230023,
	  "argument":"192.0.2.250",
	  "command":"traceroute",
	  "name":"test",
	  "timestamp":"2018-12-05 13:37:29",
	  "queries":8,
	  "status":{"completed":8, "failed":0, "pending":0}
	},
      ]
    
  



Get the results of a measurement

API Endpoint: https://api.periscope.caida.org/v2/measurement/id/result?format=format
HTTP Method: GET
Parameters:
Name Type Value Notes
format string "json" | "raw" optional, default "json"

A GET request to this endpoint returns a JSON-encoded object with the status and results of the measurement with id id. The object has all the same attributes as a status object, plus a "queries" attribute:

  • id [integer]: The unique identifier of the measurement (same as the id parameter)
  • argument [string]: The IP address used as target in this measurement's queries
  • command [string]: The command used in this measurement's queries.
  • name [string]: The name of the measurement specified when the measurement was created.
  • timestamp [string]: The date and time when the measurement was submitted to Periscope, in "YYYY-MM-DD HH:MM:SS" format.
  • status: object with the following attributes:
    • completed [integer]: number of completed queries
    • failed [integer]: number of failed queries
    • pending [integer]: number of queries still in progress
  • queries: array of objects with the following attributes:
    • id [integer]: The unique identifier of the query
    • server [string]: The name of the server
    • type [string]: The type of the monitor
    • asn [integer]: The ASN of the monitor
    • router [string or integer]: The name of the monitor, or integer 0 for a single anonymous monitor on a looking glass server
    • city [string]: The city where the monitor is located
    • country [string]: The country (in ISO-3166-1 alpha-2 format) where the monitor is located
    • starttime [string]: The date and time when the query was sent to the LG server, in "YYYY-MM-DD HH:MM:SS" format.
    • endtime [string]: The date and time when the response was received from the LG server, in "YYYY-MM-DD HH:MM:SS" format.
    • status [string]: The status of the query: "completed", "failed", or "pending"
    • result [optional]: If status is "completed", this contains the result of the query, formatted according to the format parameter (see below).
    • raw_result [optional string]: If status is "pending" or "failed", this field may contain a raw result string of unknown quality.
    • error [optional string]: If status is "failed", this will contain an error message.

The format of the result attribute in a completed query depends on the format parameter and the command:

  • format="json", command="traceroute": a JSON-encoded object with the following attributes:
    • dst_addr [string]: destination address
    • hops: array of objects with the following attributes:
      • hop [integer]: traceroute hop number, i.e. the TTL of a probe packet
      • responses: array of objects (usually 3) describing the response to probe packets for this hop. If there was a response, it will contain the following attributes:
        • from [string]: IP address of response
        • rtt [number]: round trip time of response, in milliseconds
        • err [optional string]: code for an ICMP destination unreachable error:
          • N: net unreachable
          • H: host unreachable
          • P: protocol unreachable
          • S: source route failed
          • X: communication administratively prohibited
          • code: other numeric ICMP code
        If there was no response, the object will contain:
        • x: "*"
    • size [optional integer]: packet size in bytes, if known
  • format="json", command="bgp": a JSON-encoded array of objects with the following attributes:
    • dst_net [optional string]: destination network in CIDR notation (almost always present)
    • AS_path [array of strings]: list of ASNs (e.g. "111") or AS-sets (e.g. "{111,222}"). Always ASPLAIN, even if raw output was ASDOT. Confederations are omitted.
    • best [optional boolean]: Whether this path is the best for the given network. Present only if the raw format supports it.
    • communities [optional string]: one or more communities in "xxx:yyy" format with optional parenthesised additional information (output from Bird routers was converted from "(xxx,yyy)" to "xxx:yyy" format)
    • localpref [optional string]: local preference value
  • format="raw": a string containing the original text returned by the server. Because raw format differs between servers, you are advised to use the "json" format instead.

Example: traceroute json

Request:
  https://api.periscope.caida.org/v2/measurement/1230062/result

Response:

    
{
  "argument": "172.16.100.7",
  "command": "traceroute", 
  "id": 1230062, 
  "name": "example1", 
  "queries": [
    {
      "id": 5516730,
      "server": "lg.as64496.net",
      "asn": 64496, 
      "router": "r1-newyork",
      "city": "New York", 
      "country": "US", 
      "starttime": "2018-12-10 22:57:30",
      "endtime": "2018-12-10 22:57:52",
      "status": "completed",
      "result": {
        "dst_addr": "198.51.100.7", 
        "hops": [
          {
            "hop": 1, 
            "responses": [
              { "from": "203.0.113.97", "rtt": 0.592 }, 
              { "from": "203.0.113.97", "rtt": 0.581 }, 
              { "from": "203.0.113.97", "rtt": 0.575 }
            ]
          }, 
          {
            "hop": 2, 
            "responses": [
              { "from": "192.0.2.42", "rtt": 0.214 }, 
              { "from": "192.0.2.42", "rtt": 0.212 }, 
              { "from": "192.0.2.42", "rtt": 0.206 }
            ]
          }, 
          {
            "hop": 3, 
            "responses": [
              { "from": "10.255.255.9", "rtt": 0.202 }, 
              { "from": "10.255.255.9", "rtt": 0.272 }, 
              { "from": "10.255.255.9", "rtt": 0.188 }
            ]
          }, 
          {
            "hop": 4, 
            "responses": [
              { "from": "198.51.100.7", "rtt": 0.34 }, 
              { "from": "198.51.100.7", "rtt": 0.331 }, 
              { "from": "198.51.100.7", "rtt": 0.332 }
            ]
          }
        ], 
        "size": 60, 
      }
    }
  ], 
  "status": { "completed": 1, "failed": 0, "pending": 0 }, 
  "timestamp": "2018-12-10 22:57:07"
}
    
  

Example: traceroute raw

Request:
  https://api.periscope.caida.org/v2/measurement/1230062/result?format=raw

Response:

    
{
  "argument": "172.16.100.7",
  "command": "traceroute", 
  "id": 1230062, 
  "name": "example1", 
  "queries": [
    {
      "id": 5516730,
      "server": "lg.as64496.net",
      "asn": 64496, 
      "router": "r1-newyork",
      "city": "New York", 
      "country": "US", 
      "starttime": "2018-12-10 22:57:30",
      "endtime": "2018-12-10 22:57:52",
      "status": "completed",
      "result": "traceroute to 198.51.100.7 (198.51.100.7), 30 hops max, 60 byte packets\n 1  abcdefg.example.org (203.0.113.97)  0.592 ms  0.581 ms  0.575 ms\n 2  wxyz.example.com (192.0.2.42)  0.214 ms  0.212 ms  0.206 ms\n 3  10.255.255.19 (10.255.255.9)  0.202 ms  0.272 ms  0.188 ms\n 4  target.example.net (198.51.100.7)  0.340 ms  0.331 ms  0.332 ms\n"
    }
  ], 
  "status": { "completed": 1, "failed": 0, "pending": 0 }, 
  "timestamp": "2018-12-10 22:57:07"
}
    
  

Example: bgp json

Request:
  https://api.periscope.caida.org/v2/measurement/1230071/result

Response:

    
{
  "argument": "198.51.100.7",
  "command": "bgp",
  "id": 1230071,
  "name": "test",
  "queries": [
    {
      "id": 5516949,
      "server": "lg.as12741.net",
      "asn": 12741,
      "router": "warsaw2",
      "city": "Warsaw",
      "country": "PL",
      "starttime": "2019-01-22 19:48:27",
      "endtime": "2019-01-22 19:49:49",
      "status": "completed",
      "result": {
        "AS_path": [ "64496", "65536", "64511", "65551" ],
        "best": true,
        "dst_net": "198.51.100.0/24",
        "localpref": "100"
      }
    }
  ],
  "status": {
    "completed": 1, "failed": 0, "pending": 0
  },
  "timestamp": "2019-01-22 19:48:27"
}
    
  

Example: bgp raw

Request:
  https://api.periscope.caida.org/v2/measurement/1230071/result?format=raw

Response:

    
{
  "argument": "198.51.100.7",
  "command": "bgp",
  "id": 1230071,
  "name": "test",
  "queries": [
    {
      "id": 5516949,
      "server": "lg.as12741.net",
      "asn": 12741,
      "router": "warsaw2",
      "city": "Warsaw",
      "country": "PL",
      "starttime": "2019-01-22 19:48:27",
      "endtime": "2019-01-22 19:49:49",
      "status": "completed",
      "result": "BGP routing table entry for 198.51.100.0/24\nPaths: (1 available, best #1, table Default-IP-Routing-Table)\n  Not advertised to any peer\n  64496 65536 64511 65551\n    192.0.2.42 from 203.0.113.78 (203.0.113.78)\n      Origin IGP, metric 0, localpref 100, valid, internal, best\n      Last update: Wed Jan 16 15:47:40 2019\n"
    }
  ],
  "status": {
    "completed": 1, "failed": 0, "pending": 0
  },
  "timestamp": "2019-01-22 19:48:27"
}
    
  



Create a new measurement

API Endpoint: https://api.periscope.caida.org/v2/measurement/
HTTP Method: POST
Parameters:

To create a new measurement, send a POST request to https://api.periscope.caida.org/v2/measurement/ with the payload encoded as JSON dictionary with the following attributes:

  • argument [string]: The IP address to probe (not hostname)
  • command [string]: "traceroute" | "bgp"
  • name [string]: A user-chosen name for the measurement that can be used to find it later. Multiple measurements can have the same name.
  • hosts A JSON-encoded array of objects with the following attributes, identifying the monitors that will be used to execute command:
    • server [optional string]: The looking glass or scamper-routeviews server name (one of server or asn is required)
    • asn [optional integer]: The Autonomous System Number of the monitor (one of server or asn is required)
    • router [string or integer]: The name of the monitor, or the 0-based index of the monitor within the server (all supported servers have at least one monitor with index 0).

If the request is accepted, the response will have an HTTP status of 201 (Created), a "Location" header containing a URL for the measurement, and a payload with a JSON-encoded object with the following attributes:

  • message [string]: "Measurement created"
  • id [integer]: a unique identifier for the newly created measurement

Example

Example request for a new measurement that will attempt to execute a traceroute to 172.16.100.7 from 5 different monitors belonging to 3 different servers.

Request body:
  
  {
    "command":"traceroute",
    "argument":"172.16.100.7",
    "name":"example1",
    "hosts":[
	{"server":"lg.as64496.net", "router":"r1-newyork" },
	{"server":"lg.as64496.net", "router":"r2-paris" },
	{"server":"lg.as64496.net", "router":"r3-london" },
	{"server":"route-views.example.routeviews.org", "router":"192.168.100.42" },
	{"server":"lg.foonet.net", "router":0 },
    ]
  }
  

Response:

  
  {"message":"Measurement created", "id":1230021}
  


Citing Periscope

Please cite the Periscope paper if you use the Periscope Looking Glass API in a publcation:
  V. Giotsas, A. Dhamdhere, k. claffy.
  "Periscope: Unifying Looking Glass Querying".
  In proceedings of the 2016 Passive and Active Measurements Conference (PAM'16)
  31 March - 1 April 2016, Heraklion, Greece

Also, please Report your Publication.

Code Samples

The following Python script creates a measurement like that in the previous example:

  
#!/usr/bin/env python3
# python character encoding: utf-8
import sys
import requests
import json
import hmac, hashlib, base64 # for calculating the HMAC signature

api_url = "https://api.periscope.caida.org/v2"
# Construct the measurement request
measurement = dict()
measurement["command"] = "traceroute"
measurement["argument"] = "172.16.100.7"
measurement["name"] = "example1"
measurement["hosts"] = [
    {"server":"lg.as64496.net", "router":"r1-newyork" },
    {"server":"lg.as64496.net", "router":"r2-paris" },
    {"server":"lg.as64496.net", "router":"r3-london" },
    {"server":"route-views.example.routeviews.org", "router":"192.168.100.42" },
    {"server":"lg.foonet.net", "router":0 },
]

data = bytes(json.dumps(measurement), "utf8")

# load public and private keys from files
with open("periscope_public_key", "rb") as f:
    public_key = f.read().strip()
with open("periscope_private_key", "rb") as f:
    private_key = f.read().strip()

# Calculate the HMAC signature
signature = base64.b64encode(hmac.new(private_key, msg=data,
    digestmod=hashlib.sha256).digest())

# Set the headers
headers = {
    'Content-type': 'application/json; charset=utf-8',
    'X-Public': public_key,
    'X-Hash': signature
}

print("Request headers:  %r" % (headers,))
print("Request data:  %r" % (data,))

# Post the data and headers
response = requests.post(api_url + "/measurement", data=data, headers=headers)

print("HTTP response status: %r" % (response.status_code,))
print("HTTP response headers: %r" % (response.headers,))
print("HTTP response text:")
print(response.text)

if response.status_code == 201:
    decoded_response = response.json()
    print("Measurement ID: %r" % (decoded_response["id"],))
else:
    sys.exit(1)
  

The same example in PHP:

  
<?php
$api_url = "https://api.periscope.caida.org/v2";

// Construct the measurement request
$measurement = [
    "command" => "traceroute",
    "argument" => "172.16.100.7",
    "name" => "example1",
    "hosts" => [
        ["server" => "lg.as64496.net", "router" => "r1-newyork" ],
        ["server" => "lg.as64496.net", "router" => "r2-paris" ],
        ["server" => "lg.as64496.net", "router" => "r3-london" ],
        ["server" => "route-views.example.routeviews.org", "router" => "192.168.100.42" ],
        ["server" => "lg.foonet.net", "router" => 0 ],
    ]
];
$data = json_encode($measurement);

# Calculate the HMAC signature
$public_key = 'my_public_key';
$private_key = 'my_private_key';
$signature = base64_encode(hash_hmac('sha256', $data, $private_key, TRUE));

# Set the headers
$headers = [
    'Content-type:application/json; charset=utf-8',
    'X-Public:' . $public_key,
    'X-Hash:' . $signature
];

print "Request headers:";
print_r($headers);
print "Request data:  " . $data . "\n";

# Post the data and headers
$ch = curl_init($api_url . "/measurement");
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_POSTFIELDS, $data);
$response = curl_exec($ch);
$response_code = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);

print "HTTP response status: " . curl_getinfo($ch, CURLINFO_RESPONSE_CODE) . "\n";
print "HTTP response text:  " . $response . "\n";

if ($response_code == 201) {
    $decoded_response = json_decode($response, 1);
    print "Measurement ID: " . $decoded_response["id"] . "\n";
} else {
    exit(1);
}
  

Related Objects

See https://catalog.caida.org/software/periscope to explore related objects to this document in the CAIDA Resource Catalog.
Published
Last Modified