Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
www.caida.org > funding : darpa-penman
Performance Evaluation Network Measurements and Analytics (PENMAN)
Sponsored by:
Defense Advanced Research Projects Agency (DARPA)

The project's goal is to improve substantially the ability of a third party to ascertain the presence of performance bottlenecks along a given path of interest, and identify physical attributes of that bottleneck.

Funding source: DARPA Cooperative Agreement HR00112020014. Period of performance: March 1, 2020 - August 31, 2021.

|   Statement of Work    |

Project Summary

Despite recent innovations in the field of Internet measurement, both the public and private sectors struggle to develop and deploy accurate and usable measurement capabilities for the Internet. This capability gap presents an increasing risk for U.S. Department of Defense (DoD) use of the Internet, especially for complex mission critical applications. This project will provide empirical grounding for research and development of tools and methods that monitor performance of complex distributed applications operating on global public Internet infrastructure, and will stimulate improvements in three supporting measurement and analytics capabilities: (1) Internet topology inference; (2) performance bottleneck inference; and (3) geophysical annotation inference. The project consists of three inter-related tasks that will be pursued in parallel.

The first task focuses on developing new techniques to enable more comprehensive visibility into Internet interconnection topology via powerful on-demand measurement capabilities and trove of information inferred from applying innovative ML algorithms to historical archives of traceroute data.

The second task concentrates on advancing the CAIDA’s prototype Internet congestion measurement and analysis platform. Based on the analysis of the prevalence, duration, and location, of recurring congestion in networks, we will develop the state-of-the-art congestion detection techniques and will integrate them into our platform. This will enable a more reliable detection of flash congestion events, including those from DDoS attacks, outages, and geophysical disasters.

The third task is to synthesize methods for annotation and analysis of the topology and performance data gathered by CAIDA, to support rigorous investigation of causes and implications of anomalies, as well as geographic annotation and visualization of congestion phenomena. This task will provide critical context to inform assessments of Internet congestion and its potential impacts.

The results of these three tasks will be integrated into a set of guidelines for developing distributed platforms for Internet measurement, including novel methods to construct incentive-compatible deployment strategies for crowd-sourced Internet measurement.

This project directly addresses a key short-term objective of DARPA -- an improved understanding of the extent to which DoD’s application performance and robustness needs can best be achieved over the commercial Internet. The results will inform DARPA’s investment strategies for cost-effective improvement of network performance of defense-related enterprises.

Statement of Work

This 18-month project involves academic researchers from UC San Diego’s Center for Applied Data Analysis (CAIDA) and the Computer Science and Artificial Intelligence Laboratory (CSAIL) at the Massachusetts Institute of Technology (MIT). It is structured as three inter-related tasks that are pursued in parallel. It builds on existing CAIDA data, tools, measurement and analysis platforms and on fruitful collaboration history of CAIDA and CSAIL. The deliverables include open source measurement and analytics software, data sets sharable with academic and government-funded researchers, and reports on advances in methods and their applications. The project timeline is subdivided into two phases: Period 1, March 2020 - February 2021, and Period 2, March 2021 - August 2021.


Period 1

MilestonesDeliverables
Task 1: Extend software development and deployment to execute global Internet topology analytics
1.1Extend and deploy topology measurement softwareM1, M2D1, D2
1.2Deduce internal and inter-domain network topologies from large-scale traceroute datasetsM3, M4D3, D4
Task 2: Develop and apply new techniques for inference of congestion in the Internet core
2.1Perform Time-Series Latency Probing to extract Round-Trip Time data over time for targets of interestM5, M6D5, D6
2.2Characterize and cross-correlate episodes of congestionM7, M8D7, D8
Task 3: Practical applications of network topology, performance measurements, and geophysical meta-data
3.1Survey geolocation capability for interconnections and internal network topologyM9, M10, M11D9, D10
3.2Studying the relationship among network topology, congestion, and the quality of distributed applicationsM12, M13D11

Milestones - Period 1

#TaskMilestoneDateStatus
M11.1Extend scamper measurement software platformMay 2020
M21.1Experimentally deploy new scamper module on 5-10 RouteViews collector vantage points (VPs)Nov 2020
M31.2Synthesis of state-of-the-art inter-domain topology analyticsAug 2020
M41.2Documentation of data sets and resulting inferencesFeb 2021
M52.1Make available six month of TSLP probing data from available VPsAug 2020
M62.1Import resulting TSLP data into multidimensional database for use in analyticsFeb 2021
M72.2Develop and document new methods for investigating spatial correlations across massive numbers of linksAug 2020
M82.2Evaluate recently developed nonparametric HMM algorithm for detection of congestionNov 2020
M93.1Select targets for geolocation comparison (subset of topology captured in Task)May 2020
M103.1Compare performance of geolocation data bases for the target listAug 2020
M113.1Develop new data sets of DNS-based hintsFeb 2021
M123.2Attend relevant meetings as requested by DARPA PMas needed
M133.2Study of incentive-compatible deployment strategies for crowdsourced Internet measurementFeb 2021

Deliverables - Period 1

#TaskDeliverableTypeDateStatus
D11.1Scamper releaseSoftwareMay 2020
D21.1Results of deployment on RouteViews VPsReportFeb 2021
D31.2Release of updated topology analytics toolsSoftwareAug 2020
D41.2Release of resulting data sets with network boundaries identifiedDataFeb 2021
D52.1Data sets generated from TSLP probingDataAug 2020
D62.1Multidimentsional data base containing integrated TSLP dataDataFeb 2021
D72.2Implementation of a new spatial correlation algorithmSoftwareNov 2020
D82.2Evaluation of nonparametric HMM algorithmReportFeb 2021
D93.1Identified target infrastructure for geolocationDataJun 2020
D103.1Inferred geolocations of target linksDataFeb 2020
D113.2Analysis of DoD QoS needsPresentationas needed

Period 2

MilestonesDeliverables
Task 1: Improve deductions of internal and interdomain network topologies
1.1Improve deductions of internal and interdomain network topologiesM14D12, D13
Task 2: Operationalize detection of congestion patterns
2.1Operationalize detection of congestion patternsM15, M16D14
Task 3: Practical applications of network topology, performance measurements, and geophysical meta-data
3.1Use improved geolocation capabilities to annotate interconnections and internal network topologyM17, M18D15, D16
3.2Develop recommendations for advancing DoD QoS-sensitive applicationsM19D17

Milestones - Period 2

#TaskMilestoneDateStatus
M141.1Apply VRFinder algorithm on measurements from existing and new CAIDA vantage pointsMay 2021
M152.1Apply algorithms to detect patterns identifying spatial correlationsMay 2021
M162.1Integrate algorithms into data processing pipeline (CAIDA)Aug 2021
M173.1Release inferences of links in the same locationMay 2021
M183.1Publish study of limitations of existing methods and identify potential improvements for future workAug 2021
M193.2Analysis of DoD QoS needs and recommendations for commodity Internet useAug 2021

Deliverables - Period 2

#TaskDeliverableTypeDateStatus
D121.1Results of application of VRFinder algorithmReportJun 2021
D131.1Release of VRFinder work productSoftwareAug 2021
D142.1Data set annotated with evidence of congestion detectedDataAug 2021
D153.1Data set of DNS-hints in hostnames for strategic ISPsDataAug 2021
D163.1Study of limitations of existing methods and report potential improvements for future workReportAug 2021
D173.2Recommendations on use of Internet measurements report to advance DoD QoS-sensitive applicationsReportAug 2021

Acknowledgement of awarding agency's support

This material is based upon work supported by the U.S. Department of Defense (DoD) Defense Advanced Research Projects Agency under Cooperative Agremeent HR00112020014. The views and conclusions contained in this document are those of the authors and should not be interpreted as necessarily representing the official policies, either expressed or implied, of the U.S. Department of Defense.

  Last Modified: Mon Mar-16-2020 11:08:08 PDT
  Page URL: https://www.caida.org/funding/darpa-penman/index.xml