A Real-time Lens into Dark Address Space of the Internet

We are expanding our existing network telescope instrumentation to capture unique global data elucidating macroscopic events (large-scale attacks, malware spread, censorship, and geophysical disasters such as earthquakes) and make these data available to vetted security researchers.

All proposed tasks were completed as scheduled.

Sponsored by:
National Science Foundation (NSF)

Principal Investigator: kc claffy

Funding source:  CNS-1059439 Period of performance: July 1, 2011 - June 30, 2014.


Project Summary

In the last decade, network telescopes have been used to observe unsolicited Internet traffic ("background radiation") sent to unassigned address space ("darkspace"). Network telescopes are one of the few types of instrumentation that allow global visibility into and historical trend analysis of a wide range of security-related events, including scanning address space for vulnerable targets, random spoofed source denial-of-service attacks, the automated spread of malicious software such as Internet worms or viruses, and miscellaneous misconfigurations. In recent years, traffic destined to darkspace has evolved to include longer-duration, low-intensity events intended to establish and maintain botnets. We propose to expand our telescope instrumentation to enable researchers to exploit this unique global data source to improve our understanding of security-related events such as large-scale attacks and malware spread.

Three pervasive challenges in network traffic research, including the telescope traffic, guide our proposed expansion: collection and storage, efficient curation, and sharing large volumes of data. The volume of data captured by the telescope is expensive to store, limiting the number of researchers who can realistically download data sets. The situation is worse during malicious activity outbreaks when the data volumes increase sharply, yet rapid analysis and response are necessary. Perhaps the most challenging obstacles to sharing any kind of Internet traffic data (even data to unused addresses!) are the privacy and security concerns. Viruses and worms may involve the installation of backdoors that provide unfettered access to infected computers, and telescope data could advertise these especially vulnerable machines.

We propose to deploy and evaluate an innovative shift in network monitoring that explicitly addresses all three challenges: enable near-real-time sharing of traffic data, in a way that maximizes data utility for research and analysis while protecting user privacy. We will improve classification of traffic to use a more modern taxonomy, including classes of DoS attacks, vulnerability scans, and malware spread. A meaningful taxonomy will help to create triggers to detect and notify interested researchers of events that merit more comprehensive measurement and analysis. We will also build infrastructure to allow vetted researchers to run analysis programs approximately one hour after data collection. For safe and ethical data sharing, we will use our recent Privacy-Sensitive Sharing Framework which integrates privacy-enhancing technology with a policy framework using proven and standard privacy principles and obligations of data seekers and data providers. To link research and education, we will create educational datakits out of samples of telescope data containing security event signatures.

Proposed methodology and instrumentation enhancements will increase the utility of network telescope instrumentation, transforming it into a more accessible, practically useful source of security-relevant data. The results of this project will contribute to developing efficient early detection, reaction and mitigation strategies thus enabling more scientific pursuit of cybersecurity research and critical advances in the global fight against pervasive malware.

Management Plan

The schedule of work below shows how we plan to accomplish the proposed tasks in two years of the project.

Subtask Description Projected Timeline Status
Task 1: Enhance tools for telescope data analysis and visualization
1.1 Refine classification and reporting Year 1 (full year), Year 2 (1st and 2nd quarters) done
1.2 Integrate reporting software with ongoing data collection Year 1 (3rd and 4th quarters), Year 2 (1st and 2nd quarters) done
1.3 Write software documentation Year 1 (3rd and 4th quarters), Year 2 (full year) done
1.4 Add geographic analysis to real-time report software Year 2 (2nd, 3rd, and 4th quarters) done
1.5 Improve attribute-based classification after feedback at workshop Year 2 (2nd, 3rd, and 4th quarters) done
Task 2: Enable real-time sharing of telescope data
2.1 Purchase a new data server and storage array Year 1 (3rd and 4th quarters) done
2.2 Create web pages announcing the availability of telescope data Year 1 (3rd quarter) done
2.3 Create user accounts and monitoring system to review requests Year 1 (4th quarter) done
2.4 Invite selected researchers to evaluate our data and approach Year 1 (4th quarter), Year 2 (1st quarter) done
Task 3: Community Development
3.1 Write and publish AUP for real-time telescope data access Year 1 (2nd quarter) done
3.2 Organize the workshop Year 1 (4th quarter) done
3.3 Publish the workshop report and recommentations Year 2 (1st and 2nd quarters) done
3.4 Refine data-sharing frameworks using the feedback from researchers Year 2 (2nd and 3rd quarters) done
3.5 Prepare annotated educational telescope data kits Year 2 (full year) done
Published