UCSD/CAIDA is focusing on advancing the capacity to monitor, depict, and predict traffic behavior on current and advanced networks, through developing and deploying tools to better engineer and operate networks and to identify traffic anomalies in real time. CAIDA will concentrate efforts in the development of tools to automate the discovery and visualization of Internet topology and peering relationships, monitor and analyze Internet traffic behavior on high speed links, detect and control resource use (security), and provide for storage and analysis of data collected in aforementioned efforts. CAIDA's cooperative agreement with DARPA for funding these vital activities went into effect on July 16th, 1998.

The base of the project includes development of an OC48 speed passive traffic monitor; development and deployment of an active Internet topology discovery tool; development of security features for existing OC12 passive monitors; analysis of data collected through project efforts, as well as correlation of active measurement data with available wide-area routing data; and storage of all data and analyses with web-based interface for public access.

Project options include development of an OC192 speed passive traffic monitor and a security/utilization fiber-optic light monitor; an increase in the active measurement effort and development of a prototype Internet traffic model; improvment in the availability of denial-of-service tracking software; and expansion of the database storage system for data and analyses resulting from this project.

This report will include descriptions of each of the four major tasks included in this project as well as options, and discussion of planned outreach and reporting activities related to the project.

Top level goals for this project include:

develop oc48mon monitor to scale passive monitoring efforts to the next level of network capacity
build an infrastructure-wide active monitoring and analysis effort focusing on topology discovery as well as routing and performance information
enable passive monitors to be used as security packet filtering hosts with ties to security policy enforcement systems
develop a queriable storage system to house collected data, analyses and tools developed through the efforts of the project

II. General/Administrative

Interaction with the interested parties is vital to the success of any project that aims to provide useful research information to a widespread, rapidly advancing industry. This section details the actions expected to be taken to initiate interaction and discussion with DARPA and a number of relevant Internet-related organizations.

1. Outreach

Since almost every aspect of this project is directly related to the provision of despirately needed information to Internet Service Providers and Internet infrastructure suppliers, outreach and communication with these groups as well as other organizations with similar goals is of paramount importance. Close communication with DARPA/SPAWAR are also key, and will undoubtedly be instrumental in helping to focus the project goals.

Hold Kickoff Meeting with SpaWar Officials and Project Personnel during 1Q
Hold Quarterly Meetings with SpaWar Officials
Participate in periodic DARPA meetings, including NGI Program Meetings and meetings to brief DARPA personnel on progress of the project
Deliver technical presentations to organizations interested in the technical aspects of this NGI research, including the Internet Engineering Task Force (IETF), the Internet Engineering Planning Group (IEPG), the North American Network Operators Group (NANOG), the Cross Industry Working Team (XIWT), the Internet Operators (IOPS). and prepare written publications covering aspects of technical efforts supported through this project
Publish 1-3 papers annually about efforts underway through this project

2. Reporting

Submit Annual Report to DARPA/ITO that includes a Technical Report, Financial Report and Quad Chart
Submit Quarterly Reports to SPAWAR covering progress, status and management -- the first is due October 1998
Submit Quarterly Financial Status Reports - Standard Form 269 -- the first is due October 1998 (UCSD Extramural Funds Dept. submits)
Submit Quarterly Reports of Federal Cash Transactions - Standard Form 272 -- the first is due October 1998 (UCSD Extramural Funds Dept. submits)
Submit Audit Reports, IF an audit is conducted involving this project
Submit Annual Invention Reports (Patent Reports) within 60 days after the end of each fiscal year
Submit a Technical Report that includes research findings at the end of the contract period
Submit Computer Software Product End Items within 30 days of completion of each development phase in a package including documentation, source and executable software

III. TASK 1 - CORAL Monitors

Task Leader: Joel Apisdorf, MCI Worldcom

Coral is a family of low-cost, high performance traffic flow monitors originally developed by MCI's vBNS Team in collaboration with UCSD/SDSC's NLANR group. OC3mon and OC12mon units have been deployed at public peering points and research universities throughout the United States. There is a significant interest in similar passive monitoring systems for OC48 and eventually OC192 speeds. CAIDA, in collaboration with MCI Worldcom, is now addressing this need throught the development of the OC48 monitor (OC48mon) and the development of specifications for an OC192 monitor.

Monitoring a fiber-optic link by collecting information on the flows, or groups of packets between a source and destination application, is a valuable tool for service providers and engineers for tracking traffic trends - for immediate troubleshooting as well as for future capacity planning. Coral monitors can be used to collect information about the amount of traffic (in bytes, packets or flows) traversing a link, as well as important traffic characteristics such as which applications and transport protocols generate the most traffic, what packet sizes are most common, how many packets of various sizes tend to arrive in groups, and data can be aggregated to analyze the characteristics of traffic between individual networks and autonomous systems.

In order to assure that OC48mon development focuses on meeting the needs of Internet service providers, engineers and researchers, CAIDA will deploy prototype monitors on real-world operational backbones. Intended collaboration partners include the NTON and Abilene networks.

Milestones for this task include:

Year 1 (July '98 - June '99):

Quarters 1 - 2:

Finalize subcontract with MCI
Develop specifications and schematics for the OC48 monitor
Prepare/deliver Task briefing for Kickoff Meeting
Review preliminary specs with DARPA, SpaWar and other Next Generation Internet (NGI) collaborators, as appropriate
Set up OC48mon @ caida.org mail list to discuss OC48mon development issues

Quarter 3:

Develop final specifications for cards
Review details of cards and OC48mon specifications with DARPA, SpaWar, other Next Generation Internet (NGI) collaborators, and commercial providers/vendors
Subcontract with hardware firm(s) for engineering and development of the OC48c optics to Utopia-3 card and the Utopia-3 to PCI bus card
Initiate development of OC48mon firmware code

Quarter 4:

Continue development of OC48mon firmware code and card development
Continue discussions of OC48mon development and use with the community

Year 2 (July '99 - June '00):

Quarters 1-2:

Continue development and enhancement of OC48mon firmware code
Begin development of aggregation/analysis code
Begin testing of prototype cards
Continue discussions of OC48mon development and use with the community

Quarter 3:

Begin testing and evaluation of prototype OC48mon
Continue discussions of OC48mon development and use with the community

Quarter 4:

Deploy OC48mon on OC48 research (e.g., NTON or vBNS) or commercial link
Initiate testing of OC48mon
Continue discussions of OC48mon development and use with the community
Develop preliminary specifications and costs for development of the LIGHTmon
Develop preliminary specifications and costs for development of the OC192mon

Deliverables for this task include:

Preliminary OC48mon specifications (for presentation at ISMA and Kickoff Meeting)
Draft OC48mon specifications for consideration by collaborators
Final OC48mon specifications
Development of prototype OC48mon
Summary of the results of testing/evaluation of the OC48mon
Preliminary specifications for the OC192mon
Preliminary specifications for the LIGHTmon

IV. TASK 2 - Tomography Mapping / Modeling

Task Leader: Daniel McRobb, CAIDA

The Internet infrastructure is not static, nor does it have any direct relationship to physical (geographic) localities. Topological hierarchies and routing behavior change frequently. CAIDA is using active measurements in correlation with other available infrastructure and routing information to discover and depict the topology, measure performance characteristics, and monitor connectivity and routing changes in the wide-area infrastructure.

Measurements from several hosts to many thousands of destinations using a "traceroute"-like tool called Skitter will provide the basis for unprecedented analysis of macroscopic Internet behavior. Additionally, Skitter has been designed to execute its pervasive measurement while incurring minimal load on the infrastructure and upon final destination servers. In line with this goal, skitter packets are 52 bytes in length, and probe destination hosts as infrequently as possible, typically about once per hour.

When tuned to a fine granularity (a few selected destinations with frequency of measurements closer to once per minute), Skitter measurements can also facilitate performance testing of Internet hardware. Initial measurements of operational routers, for example, have identified statistically significant problems on certain routers using network route cache technology. Data from routers running more recent (non-caching) software did not reflect these performance problems.

Tomographic depiction of the Internet infrastructure will include correlation among routing (BGP) relationships for major Internet service providers, performance information between specific nodes, and characterization of specific paths, e.g. available bandwidth and throughput across specific links.

Milestones for this task include:

Year 1 (July '98 - June '99):

Quarter 1:

Develop preliminary list of target destinations for monitoring
Deploy Skitter tool on distributed host monitors for collecting round-trip and hop-by-hop data from sites throughout Internet infrastructure
Prepare/deliver Task 2 briefing for Kickoff Meeting
Initiate briefings to Internet community on purpose and initial results of Skitter (e.g., North American Network Operators' Group (NANOG), ISPs, NGI community, etc.) and solicit their inputs on the initiative's goals, technical methodologies, and analysis priorities
Develop/implement a solution (e.g., enhancement of ARTS software) for storage of Tomography-related data, e.g. active and passive measurements and routing data

Quarter 2:

Initiate collection of data from distributed sites throughout the global Internet -- approximately 23,000 hosts
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs
Add additional 1-2 measurement hosts
Begin analysis of data

Quarter 3:

Continue collection of data from distributed sites throughout the global Internet
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs
Add additional 1-2 measurement hosts
Develop prototype tools to analyze and visually depict topology, routing, and performance data

Quarter 4:

Continue collection of data from distributed sites throughout the global Internet
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs; prepare technical paper on initial results from analysis of Skitter/tomographic data
Add additional 1-2 measurement hosts
Enhance prototype tools to analyze and visually depict topology, routing, and performance data

Year 2 (July '99 - June '00):

Quarter 1:

Expand collection of data from distributed sites throughout the Internet -- approximately 30,000 hosts
Develop web-based query forms to permit users to access raw and correlated data
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs
Add additional 1-2 measurement hosts
Enhance the Skitter/Tomography analysis/visualization tools
Place Skitter code on CAIDA FTP site for use by ISP collaborators in monitoring their internal networks

Quarter 2:

Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs
Add additional 1-2 measurement hosts
Enhance the Skitter/Tomography analysis/visualization tools

Quarter 3:

Initiate development of prototype traffic model
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs
Add additional 1-2 measurement hosts
Enhance the Skitter/Tomography analysis/visualization tools

Quarter 4:

Place Skitter code on CAIDA FTP site and solicit collaborative involvement by 3rd parties (researchers and ISPs) in its continued development and enhancement
Test and evaluate prototype traffic model
Continue briefings to Internet community on purpose and initial results of Skitter and solicit their inputs; prepare technical paper on initial results from analysis of Skitter/tomographic data
Enhance the Skitter/Tomography analysis/visualization tools

Deliverables for this task include:

Summary of preliminary findings for presentation at the Project Kickoff meeting
Development preliminary visualization of Skitter data using Otter tool
Deliver 6+ public presentations covering the goals and status of the Skitter tool
Prepare 2 or more technical papers covering the goals and results from the Skitter/Tomography effort
Make Skitter code available to 3rd parties for testing/enhancement/use
Develop prototype traffic model and make code publicly available

V. TASK 3 - Security

Task Leader: Glenn Sager, PICS/SDSC

The evolution of high-performance networks introduces new problems for security management. High performance ATM switches lack general filtering mechanisms, and can even hinder access to higher-level protocol information due to the fragmenting of IP data packets. Additionally, network bandwidth is increasing (much) faster than host speed, resulting in contention for network monitors' CPU and bus resources. Rapid advances above the hardware level, such as new protocol deployment, also introduce unexpected vulnerabilities.

CAIDA is collaborating with the Pacific Institute for Computer Security (PICS) at SDSC to enhance the OC12mon passive traffic monitor to facilitate ubiquitous network monitoring at aggregation points (e.g. DMZ's and up-stream ISP's), by developing dynamic filtering and data collection, security policy compliance monitoring, and security policy enforcement components.

Filtering is required to reduce data, isolate suspicious traffic, minimize contention for the peripheral bus, and permit persistent monitoring of heavily-loaded links. This will be accomplished with two-level filtering: in hardware on the network adaptor FPGA and in the host kernel.

The PICS team will extend the OC12mon capabilities by developing security policy compliance and enforcement modules. The compliance module takes a network security policy and passively audits traffic on the link for compliance. The compliance monitor could be used to signal an enforcement module to actively respond to the non-compliant traffic.

Milestones for this task include:

Year 1 (July '98 - June '99):

Quarter 1:

Finalize subcontract with General Atomics
Initiate efforts on the original SOW [note this SOW focuses on the security features of the OC3mon monitor; due to budgetary cuts, this task was rescoped during the 1st Quarter to focus exclusively on the OC12mon monitor]
Prepare/deliver Task 3 briefing for Kickoff Meeting

Quarter 2:

Initiate porting of kernel packet filtering code to Coral OC12 monitor
Initiate development of prototype firmware (FPGA) packet header filter

Quarter 3:

Initiate performance studies of in-kernel header filtering on OC12mon
Initiate testing of prototype firmware (FPGA) packet header filter

Quarter 4:

Finalize code for in-kernal header filtering and firmware (FPGA) packet header filter
Integrate security compliance monitor software in OC12mon
Initiate performance studies of real-time header capture and security filtering
Develop technical presentation and technical paper covering security features of Coral monitors

Year 2 (July '99 - June '00):

Quarter 1:

Continue performance studies of real-time header capture and security filtering; enhance code as required
Initiate proof-of-concept demonstrations of broadband (e.g. ATM) security enforcement by means such as NNI protocol attacks, switch modification, connection hijacking, or IP RESET forgeries

Quarter 2:

Continue proof-of-concept demonstrations of broadband security enforcement features
Implement and demonstrate security policy enforcement module in OC12mon

Quarter 3:

Test and evaluate OC12card security features in OC3 applications
Develop set of recommendations for additional security applications/implementations of the Coral or related traffic monitoring tools

Quarter 4:

Develop technical presentation and technical paper covering results of these efforts focusing on implementation of the security features of Coral monitors

Deliverables for this task include:

Prototype packet filtering code for OC12mon
Security compliance monitoring/enforcement code for OC12mon
Technical papers/presentations on features and performance of OC12mon based security filtering and enforcement

VI. TASK 4 - Storage / Analysis

Information being developed through the various distributed monitoring devices could amount to terabytes of data by the end of year two. To be useful for real-time engineering analyses of Internet traffic patterns and behavior, these data will be encrypted at the collection points and aggregated at SDSC on a Sun Enterprise 450, stored long term on a Digital RAID array, and archived on SDSC's IBM HPSS storage system. Additionally, another donated server from Sun is expected for use as a web server and secondary process engine for these data and analyses.

Milestones for this task include:

Year 1 (July '98 - June '99):

Quarter 1:

Purchase a disk array for storage of Skitter and related data
Implement file format (database) for storing data

Quarter 2:

Finalize configurations of disk array and front-end collector/processor/storage agent
Transfer collected data to new system
Initiate transfer/storage of data directly from remote monitoring host
Initiate analysis of Skitter and related data (BGP and trace data) by CAIDA researchers

Quarter 3:

Develop reporting format for summarizing data
Develop and implement secure methods of accessing the data and related reports

Quarter 4:

Implement web-based access to Task 2 data/results

Year 2 (July '99 - June '00) :

Quarters 1-2:

Expand collection/storage of data
Enhance analysis code and reporting formats

Quarters 3-4:

Expand collection/storage of data
Enhance analysis code and reporting formats
Post standard analyzed, correlated data to a public CAIDA website

Deliverables for this task include:

Public dissemination of standardized analyses and correlated data

VII. Options

Tomography Mapping/Modeling

Increase collection of Skitter data; increase correlation of Skitter data with passive measurements and routing data; enhance Tomography analysis and visualization tools based on community feedback; test a prototype traffic model and work with vendors to incorporate real traffic data capabilitiesfor future releases of WAN simulation software.

Storage/Analysis

Enhance and expand the database, storage system, analysis code and reporting formats.
Coral Monitors - OC48mon

Deploy second monitor for testing and evaluation; modify OC48mon firmware based on evaluation feedback; and continue statistics acquisition under CAIDA (no cost to DARPA).
Coral Monitors - LIGHTmon

Develop final specifications and review with DARPA and other NGI collaborators as appropriate; subcontract with hardware firm to develop the LIGHTmon card; develop LIGHTmon firmware and aggregation/analysis code; enhance the LIGHTmon host code and deploy on NTON or vBNS or NGI network (as appropriate) for testing and evaluation.
Coral Monitors - OC192mon

Develop final specifications and review with DARPA and other NGI collaborators as appropriate; subcontract with hardware firm to develop the OC192mon card; develop OC192mon firmware and aggregation/analysis code; enhance the OC192mon host code and deploy on NTON or vBNS or NGI network (as appropriate) for testing and evaluation.
Security

Generalize and extend router interface in denial-of-service tracking software (DOStracker) to interoperate with other (non-Cisco) equipment; identify scalability properties of hybrid security enforcement model to higher performance networks; and evaluate requirements for extending OC12mon security modules to OC48mon.