Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis > funding : paridine-iodanp
IODA-NP: Multi-source Realtime Detection of Macroscopic Internet Connectivity Disruption
Sponsored by:
Department of Homeland Security (DHS)
The focus of this project is on detection and characterization of large-scale Internet outages significatnly affecting large geographic regions or specific Internet operators.

Funding source: DHS contract 70RSAT18CB0000015. Period of performance: July 25, 2018 - January 31, 2020.

|   Statement of Work     Proposal    IODA Project Page   |

Statement of Work

The proposed effort includes applied research, software development, new data analytics, systems integration, operations and maintenance, and an interactive analysis and reporting service. Tasks and deliverables for the entire project are separated into three years:

Year I: Applied Research and Development

Task 1: Definitions and Metrics for Macroscopic Internet Outage Detection
1.1Define macroscopic Internet outage as the class of Network/Internet scale Disruptive Event (NIDE) targeted for detection
1.2Illustrate potential causes for this type of event and discuss the rationale behind the approach - NIDE Identification Document (NID)
1.3Identify quantifiable metrics and the type of sensors and Internet measurement data to use for detecting and characterizing these events
Task 2: IODA: Macroscopic Internet Outage Detection and Analysis Framework
2.1Develop Internet measurement, data collection, sanitization, aggregation and analysis methodologies to detect the events defined in Task 1 in near-real-time
2.2Identify the technical and operational capabilities that IODA will provide, as well as the metrics and analytic techniques to be used
2.3Document the framework - NIDE Analysis Framework Document (NAFD)
Task 3: Macroscopic Internet Outage Analysis, Validation, and Reporting
3.1Analyze and report Internet outages detected by IODA
3.2Confirm and validate our inferences using available sources and collaborations with other researchers
3.3Evaluate the effectiveness of our approach and identify limitations and challenges
Task 4: IODA Interface and Documentation
4.1Develop Application Programming Interfaces (APIs) to access live stream alerts generated by the system
4.2Develop visualization interfaces/interactive dashboards for targeted inspection of the observed signals
4.3Document the service, APIs, and the visual interfaces
Task 5: Continued Support to Software Development, Prepare for Pilot Activities
5.1Go/No-Go Plan, demonstration, and report
Task 6: Program Management
6.1Participate in PI meetings
6.2Quarterly Program Reviews
6.3Privacy Plan
6.4Final Report

Milestones and Deliverables (Year I)

1Technical and Financial ReportsMonthlyongoing
2Program ReviewsQuarterlyongoing
3Kick-off meeting Privacy PlanAug 25, 2018done
4Complete a first version of the NIDE Identification DocumentReport: NIDE Identification Document (NID)Oct 25, 2018done
5Complete a first version of the NIDE Analysis Framework DocumentReport: NIDE Analysis Framework Document (NAFD)done
6Purchase and deploy a time series database cluster machine and disk traydone
7Deploy the BGP based detection approach Nov 25, 2018done
8Deploy the active probing based detection approachJan 25, 2019done
9Deploy a first version of the HTTP APIdone
10Improve the NIDE Identification DocumentReport: Updated NIDApr 25, 2019done
11Deploy the IBR traffic based detection approachAug 25, 2019done
12Deploy a refined version of the HTTP APIReport: NIDE Framework API documentationOct 25, 2019
13Prepare first NIDE Analysis ReportReport: NIDE Analysis ReportNov 25, 2019
14Prepare Go/No-Go evaluation planDec 25, 2019
15Deploy the detection fusing all three data sourcesSoftware: 1st release of software components (open source)Dec 25, 2019
16Improve the NIDE Analysis Framework DocumentReport: Updated NAFDJan 15, 2020
17Go/No Go demonstration Jan 10, 2020
18Report: Final ReportJan 31, 2020
  Last Modified: Wed Nov-13-2019 21:23:16 PST
  Page URL: