Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
Internet Outage Detection and Analysis (IODA)

IODA: Internet Outage Detection and Analysis Public and private sector stakeholders around the world seek ways to ensure that the Internet provides the level of reliability and resilience we have long taken for granted from the telephony network. Unfortunately, in spite of the societal and economic impact of Internet connectivity disruptions, we lack near-realtime, scalable and validated methodologies and tools to identify and understand large-scale Internet outages.

Based on experimental work in which we combined measurements at the control plane, active probing and passive traffic analysis, CAIDA developed an operational prototype system that monitors the Internet, in near-realtime, with the goal of identifying macroscopic Internet outages affecting the edge of the network, i.e., significantly impacting an AS or a large fraction of a country.

Overview

The IODA system processes and analyzes measurements from:

  • Global Internet routing (BGP): we use data from ~500 monitors participating in the RouteViews and RIPE RIS projects to establish which network blocks are reachable based on the Internet control plane.
  • Internet Background Radiation: we process unsolicited traffic reaching the UCSD Network Telescope monitoring an unutilized /8 address block.
  • Active probing: we continuously probe a large fraction of the (routable) IPv4 address space from several CAIDA Ark nodes distributed worldwide and use a methodology developed by University of Southern California to infer when a /24 block is affected by a network outage.

Our outage inference system combines information from these three data sources, establishes the relevance of an event and generates alerts. The outage events and the corresponding signals obtained through automated analysis are displayed on dashboards and interactive graphs that allow the user to further inspect the data.

A high-level view of the architecture of IODA

Interactive Visual Interface

The IODA system is designed as "Software as a Service", being based on a complex distributed infrastructure and on large and diverse live data streams taken as input. The prototype system is currently running as an experimental service 24/7 and high-level interactive dashboards are accessible at ioda.caida.org.

Other examples of IODA visualizations are:

"Transition to Practice" (TTP) - Transitioning research results to practice

Collaboration with Industry

Public Safety

Open-source Software

We released the software components we developed in this project with an open source license. Several of the project's software components offer general applicablity to research and applicative fields of networking.

Presentations

Posters

Publications

Blog Entries

Press coverage

Workshops

  • IMAPS 2016 - Internet Measurement And Political Science (IMAPS) Workshop: Conflict and Contention in the Digital Age
  • 1st CAIDA BGP Hackathon 2016
  • IMAPS 2014 - Internet Measurement And Political Science (IMAPS) Workshop: Network Outages
  • DUST 2012 - 1st International Workshop on Darkspace and UnSolicited Traffic Analysis

Funding sources

This platform was supported by NSF grant CNS-1228994 Detection and Analysis of Large-scale Internet Infrastructure Outages (IODA), DHS S&T contract HHSP 233201600012C Science of Internet Security: Technology and Experimental Research, and DHS S&T cooperative agreement FA8750-12-2-0326 Supporting Research and Development of Security Technologies through Network and Security Data Collection.

Additional funding to work on visualization interfaces was generously provided by a Comcast research grant.

This research uses resources of the National Energy Research Scientific Computing Center (NERSC), a DOE Office of Science User Facility supported by the DoE Office of Science under contract DE-AC02-05CH11231.

This work also makes use of the Extreme Science and Engineering Discovery Environment (XSEDE), which is supported by NSF grant ACI-1053575.

Team

  • Alberto Dainotti, PI (CAIDA)
  • kc claffy, co-PI (CAIDA)
  • Vasco Asturiano (CAIDA)
  • Karyn Benson (CAIDA)
  • Marina Fomenkov (CAIDA)
  • Bradley Huffaker (CAIDA)
  • Ken Keys (CAIDA)
  • Alistair King (CAIDA)
  • Ryan Koga (CAIDA)
  • Alex Ma (CAIDA)
  • Chiara Orsini (CAIDA)
  • Joshua Polterock (CAIDA)

Student Interns

  • Alessadandro Puccetti (University of Pisa)
  • Bernardo Duarte (Ithaca College)
  • Long Tran (UC San Diego)
  • Prakriti Gupta (UC San Diego)
  • Jonathan Yuan (UC San Diego)
  • Hanh On (UC San Diego)
  • Johanna Fleischman (UC San Diego)
  • Adam Velasco (UC San Diego)
  • Erik Muntean (UC San Diego)
  • Ryan Wagner (UC San Diego)
  • Simon Zhang (UC San Diego)

Acknowledgements

Special thanks to Emile Aben for his inspiration and feedback, and to Brian Kantor and Nevil Brownlee for their assistance in this project.

  Last Modified: Wed Feb-15-2017 10:05:41 PST
  Page URL: http://www.caida.org/projects/ioda/index.xml