Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
Internet Outage Detection and Analysis (IODA)

IODA: Internet Outage Detection and Analysis Public and private sector stakeholders around the world seek ways to ensure that the Internet provides the level of reliability and resilience we have long taken for granted from the telephony network. Unfortunately, in spite of the societal and economic impact of Internet connectivity disruptions, we lack near-realtime, scalable and validated methodologies and tools to identify and understand large-scale Internet outages.

Based on experimental work in which we combined measurements at the control plane, active probing and passive traffic analysis, CAIDA developed an operational prototype system that monitors the Internet, in near-realtime, with the goal of identifying macroscopic Internet outages affecting the edge of the network, i.e., significantly impacting an AS or a large fraction of a country.

Overview

The IODA system processes and analyzes measurements from:

  • Global Internet routing (BGP): we use data from ~500 monitors participating in the RouteViews and RIPE RIS projects to establish which network blocks are reachable based on the Internet control plane.
  • Internet Background Radiation: we process unsolicited traffic reaching the UCSD Network Telescope monitoring an unutilized /8 address block.
  • Active probing: we continuously probe a large fraction of the (routable) IPv4 address space from several CAIDA Ark nodes distributed worldwide and use a methodology developed by University of Southern California to infer when a /24 block is affected by a network outage.

Our outage inference system combines information from these three data sources, establishes the relevance of an event and generates alerts. The outage events and the corresponding signals obtained through automated analysis are displayed on dashboards and interactive graphs that allow the user to further inspect the data.

Interactive Visual Interface

The IODA system is designed as "Software as a Service", being based on a complex distributed infrastructure and on large and diverse live data streams taken as input. The prototype system is currently running as an experimental service 24/7 and high-level interactive dashboards are accessible at ioda.caida.org Full access to the platform is available to collaborating institutions.

Other examples of IODA visualizations are:

"Transition to Practice" (TTP) - Transitioning research results to practice

Collaboration with Industry

Public Safety

Open-source Software

We released the software components we developed in this project with an open source license. Several of the project's software components offer general applicablity to research and applicative fields of networking.

Presentations

Posters

Publications

Blog Entries

Press coverage

Workshops

  • IMAPS 2016 - Internet Measurement And Political Science (IMAPS) Workshop: Conflict and Contention in the Digital Age
  • 1st CAIDA BGP Hackathon 2016
  • IMAPS 2014 - Internet Measurement And Political Science (IMAPS) Workshop: Network Outages
  • DUST 2012 - 1st International Workshop on Darkspace and UnSolicited Traffic Analysis

Funding source

Our support for research on Internet outages comes from NSF award, (NSF CNS-1228994) "Detection and analysis of large-scale Internet infrastructure outages".

This research uses resources of the National Energy Research Scientific Computing Center, a DOE Office of Science User Facility supported by the Office of Science of the U.S. Department of Energy under Contract No. DE-AC02-05CH11231.

This work also uses the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by National Science Foundation grant number ACI-1053575.

This work was supported by Department of Homeland Security Science and Technology Directorate grant FA8750-12-2-0314, cooperative agreement FA8750-12-2-0326, and NSF grant CNS-1228994.

Additional funding to work on visualization interfaces was generously provided by a Comcast research grant.

Team

  • Alberto Dainotti (CAIDA)
  • kc claffy (CAIDA)
  • Vasco Asturiano (CAIDA)
  • Karyn Benson (UC San Diego)
  • Ken Keys (CAIDA)
  • Alistair King (CAIDA)
  • Bradley Huffaker (CAIDA)
  • Chiara Orsini (CAIDA)
  • Ryan Koga (CAIDA)
  • Alex Ma (CAIDA)
  • Marina Fomenkov (CAIDA)

Student Interns

  • Alessadandro Puccetti (University of Pisa)
  • Bernardo Duarte (Ithaca College)
  • Long Tran (UC San Diego)
  • Prakriti Gupta (UC San Diego)
  • Jonathan Yuan (UC San Diego)
  • Hanh On (UC San Diego)
  • Johanna Fleischman (UC San Diego)
  • Adam Velasco (UC San Diego)
  • Erik Muntean (UC San Diego)
  • Ryan Wagner (UC San Diego)
  • Simon Zhang (UC San Diego)

Acknowledgements

Special thanks to Brian Kantor and Nevil Brownlee for their assistance in this project.

  Last Modified: Wed Jan-4-2017 12:25:03 PST
  Page URL: http://www.caida.org/projects/ioda/index.xml