Internet Outage Detection and Analysis (IODA)

IODA: Internet Outage Detection and Analysis Public and private sector stakeholders around the world seek ways to ensure that the Internet provides the level of reliability and resilience we have long taken for granted from the telephony network. Unfortunately, in spite of the societal and economic impact of Internet connectivity disruptions, we lack near-realtime, scalable and validated methodologies and tools to identify and understand large-scale Internet outages.

Based on experimental work in which we combined measurements at the control plane, active probing and passive traffic analysis, CAIDA developed an operational prototype system that monitors the Internet, in near-realtime, with the goal of identifying macroscopic Internet outages affecting the edge of the network, i.e., significantly impacting an AS or a large fraction of a country.

In August 2021, the IODA service moved from ioda.caida.org to Georgia Tech with PI Alberto Dainotti.


Overview

The IODA system processes and analyzes measurements from:

  • Global Internet routing (BGP): we use data from ~500 monitors participating in the RouteViews and RIPE RIS projects to establish which network blocks are reachable based on the Internet control plane.
  • Internet Background Radiation: we process unsolicited traffic reaching the UCSD Network Telescope monitoring an unutilized /8 address block.
  • Active probing: we continuously probe a large fraction of the (routable) IPv4 address space from several CAIDA Ark nodes distributed worldwide and use a methodology developed by University of Southern California to infer when a /24 block is affected by a network outage.

Our outage inference system combines information from these three data sources, establishes the relevance of an event and generates alerts. The outage events and the corresponding signals obtained through automated analysis are displayed on dashboards and interactive graphs that allow the user to further inspect the data.

A high-level view of the architecture of IODA

Interactive Visual Interface

The IODA system is designed as "Software as a Service", being based on a complex distributed infrastructure and on large and diverse live data streams taken as input. The prototype system is running as an experimental service 24/7 and high-level interactive dashboards are accessible at:

Other examples of IODA visualizations are:

  • Live view of the reachability of BGP prefixes geolocated to North Korea (last 7 days), and linked view of December 2014 outages in North Korea.
  • Animation showing the evolution of the RouteViews and RIPE RIS BGP measurement infrastructure since 2001.
  • An experimental visualization of the impact of the Sandy hurricane (2012) using data from active measurements from Planet Lab nodes carried out by the University of Maryland.

"Transition to Practice" (TTP) - Transitioning research results to practice

Collaboration with Industry

Public Safety

Open-source Software

We released the software components we developed in this project with an open source license. Several of the project's software components offer general applicablity to research and applicative fields of networking.

Presentations

Posters

Publications

Blog Entries

Press coverage

Workshops

  • IMAPS 2018 - Internet Measurement And Political Science (IMAPS) Workshop
  • IMAPS 2017 - Internet Measurement And Political Science (IMAPS) Workshop: Conflict and Contention in the Digital Age
  • IMAPS 2016 - Internet Measurement And Political Science (IMAPS) Workshop: Conflict and Contention in the Digital Age
  • 1st CAIDA BGP Hackathon 2016
  • IMAPS 2014 - Internet Measurement And Political Science (IMAPS) Workshop: Network Outages
  • DUST 2012 - 1st International Workshop on Darkspace and UnSolicited Traffic Analysis

Funding sources

Department of Homeland Security (DHS) National Science Foundation (NSF) COMCAST (COMCAST) National Energy Research Scientific Computing Center (NERSC) Open Technology Fund (OTF) Extreme Science and Engineering Discovery Environment (XSEDE)

Support for the Internet Outage Detection and Analysis project is provided by the Department of Homeland Security (DHS) contract 70RSAT18CB0000015 Multi-source Realtime Detection of Macroscopic Internet Connectivity Disruption, S&T contract HHSP 233201600012C Science of Internet Security: Technology Experimental Research, S&T contract NBCHC070133 Supporting Research Development of Security Technologies through Network Security Data Collection, and S&T cooperative agreement FA8750-12-2-0326 Supporting Research and Development of Security Technologies through Network and Security Data Collection and the National Science Foundation (NSF) grant CNS-1228994 Detection and analysis of large-scale Internet infrastructure outages. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright notation thereon. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of DHS, NSF, or the U.S. Government.

Additional funding to work on visualization interfaces was generously provided by a Comcast research grant.

This research uses resources of the National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility supported by the DoE Office of Science under contract DE-AC02-05CH11231.

This work also makes use of the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by NSF grant ACI-1053575.

This platform is also supported by the Open Technology Fund under contract number 1002-2018-027.

Team

  • Alberto Dainotti, PI (CAIDA)
  • kc claffy, co-PI (CAIDA)
  • Victor Adeyokunnu (CAIDA)
  • Vasco Asturiano (formerly CAIDA)
  • Karyn Benson (formerly CAIDA)
  • Roderick Fanou (CAIDA)
  • Marina Fomenkov (CAIDA)
  • Bradley Huffaker (CAIDA)
  • Ken Keys (CAIDA)
  • Alistair King (CAIDA)
  • Ryan Koga (CAIDA)
  • Alex Ma (CAIDA)
  • Chiara Orsini (formerly CAIDA)
  • Ramakrishna Padmanabhan (CAIDA)
  • Joshua Polterock (CAIDA)
  • Philipp Winter (formerly CAIDA)
  • Mingwei Zhang (CAIDA)

Student Interns

  • Alessadandro Puccetti (University of Pisa)
  • Bernardo Duarte (Ithaca College)
  • Long Tran (UC San Diego)
  • Prakriti Gupta (UC San Diego)
  • Jonathan Yuan (UC San Diego)
  • Hanh On (UC San Diego)
  • Johanna Fleischman (UC San Diego)
  • Adam Velasco (UC San Diego)
  • Erik Muntean (UC San Diego)
  • Ryan Wagner (UC San Diego)
  • Simon Zhang (UC San Diego)

Acknowledgments

Special thanks to Emile Aben for his inspiration and feedback, and to Brian Kantor and Nevil Brownlee for their assistance in this project.


Additional Content

CAIDA Internet Outage Detection and Analysis (IODA) Probe Information

CAIDA developed an operational prototype system that monitors the Internet, in near-realtime, with the goal of identifying macroscopic Internet outages affecting the edge of the network. As part of this goal CAIDA runs a number of periodic and ongoing probes (described by this page).

CAIDA Internet Outage Detection and Analysis (IODA) Probe Information

CAIDA developed an operational prototype system that monitors the Internet, in near-realtime, with the goal of identifying macroscopic Internet outages affecting the edge of the network. As part of this goal CAIDA runs a number of periodic and ongoing probes (described by this page).

Published
Last Modified