Skip to Content
[CAIDA - Cooperative Association for Internet Data Analysis logo]
The Cooperative Association for Internet Data Analysis
Internet Laboratory for Empirical Network Science (iLENS)
Sponsored by:
National Science Foundation (NSF)
The iLENS project proposes to upgrade and extend the active measurement infrastructure Archipelago (Ark), to provide academic researchers an unprecedented laboratory in which to quickly design, implement, and easily coordinate the execution of experiments across a widely distributed set of dedicated monitors.

Funding source: NSF CNS-0958547. Period of performance: March 1, 2010 - Feb 28, 2014.

|  Project Summary    Proposal  |

Project Summary

Effective Internet measurement raises daunting issues for the research community and funding agencies. Improved understanding of the structure and dynamics of Internet topology, routing, workload, performance, and vulnerabilities remain a disturbingly elusive priority, in part for lack of largescale distributed network measurement infrastructure available to scientific researchers. The dearth is understandable; measurement of operational Internet infrastructure involves navigating more complex and interconnected dimensions than measurement in most scientific disciplines: logistical, financial, methodological, technical, legal, and ethical. CAIDA has been navigating these challenges with modest success for fifteen years, collecting, coordinating, curating, and sharing data sets for the Internet research and operational community in support of Internet science. With previous NSF (CRI) and other funding, we have been able to design, implement, deploy, and operate a relatively small but secure platform capable of performing various types of Internet infrastructure measurements and assessments. We propose to upgrade and extend -- in geographic scope as well as function -- this active measurement instrument (Ark) to provide academic researchers an unprecedented laboratory in which to quickly design, implement, and easily coordinate the execution of experiments across a widely distributed set of dedicated monitors.

In September 2007 Ark began to support ongoing global Internet topology measurement and mapping, and Ark now gathers the largest set of IP topology data for use by academic researchers. We are using the best available, but still rudimentary, techniques for IP topology mapping, and we also make several processed data sets (AS-links, AS relationships) available as "soft infrastructure" to researchers. We propose to deploy new techniques, as well as supporting software for analysis, annotation, topology generation, and interactive visualization of resulting annotated Internet graphs.

More importantly, we have demonstrated, and now wish to operationalize, the ability for this infrastructure to serve other researchers undertaking macroscopic studies of the Internet. Our first two experiments with external use of the infrastructure resulted in publications in the Internet Measurement Conference in 2008 and 2009.

We look forward to to a broad cross-section of research communities making substantial use of our Internet measurement infrastructure. Our top infrastructure development priorities are: (1) add monitors in geographic and topological areas we lack coverage; (2) improve tools for processing raw topology data, to enable an unprecedented range of Internet mapping research while reducing the burden on individual researchers and students to achieve results; (3) enhance and develop new software modules to support new types of experiments and validation. We propose to conduct annual workshops to collect, synthesize, and plan implementation of feedback on infrastructure operation.

Sustainable funding for large-scale measurement instrumentation past the span of a given funded research project has eluded the Internet research community, which has inhibited the creation of an underlying discipline that formalizes our observations and understanding of this complex networked system. By lowering the cost in time and effort needed to implement a measurement idea, Ark allows researchers to test and evaluate more experimental, sophisticated, and risky ideas, and facilitates integration of measurements and data into course curricula. The data currently provided by our infrastructure has strengthened the intellectual merit of a wide range of network modeling, simulation, analysis, and theoretical research activities. The broader impacts of the proposed work are reflected in the new types of research and data enabled, including historical Internet studies, evaluation of future Internet architectures, and empirical grounding for the emerging discipline of network science.

Management Plan

Throughout the project we will emphasize support for external researchers wishing to run experiments on Ark. We will provide data storage, analysis tools, Internet measurement expertise and advice, and a system for continuous feedback to improve the operation of our experimental infrastructure and to increase user satisfaction. The labor effort includes: 1) maintenance and support of the central server and remote monitors, 2) integration and deployment of new monitors and coordination with remote sites, 3) integration of data and compute servers, and network switch, 4) software development including bulk DNS queries, interactive visualization, and integration of real time routing data, 5) curation, archival and distribution of the data, 6) development of supporting documentation, web pages, surveys, and educational materials, and 7) organization of annual workshops and publication of resulting reports.

CAIDA personnel will be responsible for accomplishing all proposed tasks. The detailed project timeline follows. Note that the submitted budget will support a full-time effort for only one system administrator, and only part-time effort for the other five researchers involved, so we spread some of the proposed tasks, particularly software development, over longer intervals than they would otherwise require.

TaskDescriptionProjected TimelineStatus
1Integrate 8 new Ark monitors into Ark platformYear 1done
2Acquire fast network switch, upgrade and re-configure CAIDA networkYear 1 (1st quarter)done
3Acquire a new data server and put it into production modeYear 1 (3rd quarter)done
4Develop project web pages and post updates on data collection status, list ongoing Ark experiments, and other project-related informationYear 1 (1st quarter)done
5Develop a web-based survey to assess the level of user satisfaction and to provide a communication channel between users and CAIDA personnelYear 1 (2nd quarter)done
6Implement mper measurement engineYear 1 (1st, 2nd, and 3rd quarters)done
7Design interactive visualization to support validation with network operatorsYear 1done
8Conduct a pilot study on integrating Ark topology measurements and real-time routing dataYear 1done
9Organize a workshop on measurement needs for validation of modeling and simulationYear 1 (3rd quarter)done
10Integrate 8 new Ark monitors into Ark platformYear 2done
11Publish the workshop reportYear 2 (1st quarter)done
12Upgrade the code for AS relationship and AS ranking calculations to fully utilize advanced computational capabilities of a new machineYear 2 (1st, 2nd, and 3rd quarters)done
13Develop external API to our bulk DNS lookup serviceYear 2 (1st and 2nd quarters)
14Refine mper measurement engine based on experienceYear 2 (4th quarter)done
15Using the interactive visualization and related web forms, collect feedback from network operators regarding the completeness and veracity of data about their networksYear 2done
16Integrate Ark topology measurements and real-time Internet routing dataYear 2
17Publish our IRB application for active measurement experiments.Year 2 (4th quarter)done
18Organize a workshop to introduce and gather feedback on the first version of the data correlation tools, including on which formatting and post-processed forms of the data are most useful to researchersYear 2 (4th quarter)done
19Integrate 8 new Ark monitors into Ark platformYear 3in progress
20Publish the workshop reportYear 3 (1st quarter)done
21Develop code to automatically annotate topology data with hostnamesYear 3 (1st and 2nd quarters)
22Improve topology mapping techniques based on the feedback received from network operatorsYear 3
23Based on our experience with external researchers using Ark infrastructure, prepare guidelines for Internet measurement data sharingYear 3 (2nd quarter)
24Start serving integrated topology and routing data to the research communityYear 3 (3rd quarter)
25Consolidate, refine, and generalize tool set for active measurement workflowsYear 3 (3rd and 4th quarters)
26Widely circulate and publish project resultsYear 3 (3rd and 4th quarters)
27Organize a workshop to discuss technology transfer possibilities of Ark-based technology, e.g., stand-alone Ark measurement tool kitYear 3 (4th quarter)
28Prepare plans how to maintain funding for the developed large-scale network measurement infrastructure after the end of this projectYear 3

  Last Modified: Fri Nov-8-2013 15:10:07 PST
  Page URL: http://www.caida.org/funding/ilens/index.xml