Skip to Content
[CAIDA - Center for Applied Internet Data Analysis logo]
Center for Applied Internet Data Analysis
A Real-time Lens into Dark Address Space of the Internet
Sponsored by:
National Science Foundation (NSF)
We are expanding our existing network telescope instrumentation to capture unique global data elucidating macroscopic events (large-scale attacks, malware spread, censorship, and geophysical disasters such as earthquakes) and make these data available to vetted security researchers.

Funding source: NSF CNS-1059439. Period of performance: July 1, 2011 - June 30, 2014.

All proposed tasks were completed as scheduled.

|   Impact Report    Project Summary    Proposal   |

Project Summary

In the last decade, network telescopes have been used to observe unsolicited Internet traffic ("background radiation") sent to unassigned address space ("darkspace"). Network telescopes are one of the few types of instrumentation that allow global visibility into and historical trend analysis of a wide range of security-related events, including scanning address space for vulnerable targets, random spoofed source denial-of-service attacks, the automated spread of malicious software such as Internet worms or viruses, and miscellaneous misconfigurations. In recent years, traffic destined to darkspace has evolved to include longer-duration, low-intensity events intended to establish and maintain botnets. We propose to expand our telescope instrumentation to enable researchers to exploit this unique global data source to improve our understanding of security-related events such as large-scale attacks and malware spread.

Three pervasive challenges in network traffic research, including the telescope traffic, guide our proposed expansion: collection and storage, efficient curation, and sharing large volumes of data. The volume of data captured by the telescope is expensive to store, limiting the number of researchers who can realistically download data sets. The situation is worse during malicious activity outbreaks when the data volumes increase sharply, yet rapid analysis and response are necessary. Perhaps the most challenging obstacles to sharing any kind of Internet traffic data (even data to unused addresses!) are the privacy and security concerns. Viruses and worms may involve the installation of backdoors that provide unfettered access to infected computers, and telescope data could advertise these especially vulnerable machines.

We propose to deploy and evaluate an innovative shift in network monitoring that explicitly addresses all three challenges: enable near-real-time sharing of traffic data, in a way that maximizes data utility for research and analysis while protecting user privacy. We will improve classification of traffic to use a more modern taxonomy, including classes of DoS attacks, vulnerability scans, and malware spread. A meaningful taxonomy will help to create triggers to detect and notify interested researchers of events that merit more comprehensive measurement and analysis. We will also build infrastructure to allow vetted researchers to run analysis programs approximately one hour after data collection. For safe and ethical data sharing, we will use our recent Privacy-Sensitive Sharing Framework which integrates privacy-enhancing technology with a policy framework using proven and standard privacy principles and obligations of data seekers and data providers. To link research and education, we will create educational datakits out of samples of telescope data containing security event signatures.

Proposed methodology and instrumentation enhancements will increase the utility of network telescope instrumentation, transforming it into a more accessible, practically useful source of security-relevant data. The results of this project will contribute to developing efficient early detection, reaction and mitigation strategies thus enabling more scientific pursuit of cybersecurity research and critical advances in the global fight against pervasive malware.

Management Plan

The schedule of work below shows how we plan to accomplish the proposed tasks in two years of the project.

SubtaskDescriptionProjected TimelineStatus
Task 1: Enhance tools for telescope data analysis and visualization
1.1Refine classification and reportingYear 1 (full year), Year 2 (1st and 2nd quarters)done
1.2Integrate reporting software with ongoing data collectionYear 1 (3rd and 4th quarters), Year 2 (1st and 2nd quarters)done
1.3Write software documentationYear 1 (3rd and 4th quarters), Year 2 (full year)done
1.4Add geographic analysis to real-time report softwareYear 2 (2nd, 3rd, and 4th quarters)done
1.5Improve attribute-based classification after feedback at workshopYear 2 (2nd, 3rd, and 4th quarters)done
Task 2: Enable real-time sharing of telescope data
2.1Purchase a new data server and storage arrayYear 1 (3rd and 4th quarters)done
2.2Create web pages announcing the availability of telescope dataYear 1 (3rd quarter)done
2.3Create user accounts and monitoring system to review requestsYear 1 (4th quarter)done
2.4Invite selected researchers to evaluate our data and approachYear 1 (4th quarter), Year 2 (1st quarter)done
Task 3: Community Development
3.1Write and publish AUP for real-time telescope data accessYear 1 (2nd quarter)done
3.2Organize the workshopYear 1 (4th quarter)done
3.3Publish the workshop report and recommentationsYear 2 (1st and 2nd quarters)done
3.4Refine data-sharing frameworks using the feedback from researchersYear 2 (2nd and 3rd quarters)done
3.5Prepare annotated educational telescope data kitsYear 2 (full year)done
  Last Modified: Wed Mar-27-2019 22:23:18 PDT
  Page URL: