augmentation of a high performance storage system housing up to terabytes of Internet-wide traffic and topology data.
These technical thrusts are essential to the robustness and safety of the dynamic, growing Internet. Coupling these areas into a single project will allow us to significantly expand the data and tools available to network engineers and researchers and to build upon important synergies between the tasks. Collaboration and openness is a critical goal of both CAIDA and the National Laboratory for Applied Network Research (NLANR) both of which are sponsored by the University of California, San Diego / San Diego Supercomputer Center (UCSD/SDSC). Finding/conclusions and raw data (filtered to protect privacy) developed through this effort will be made available to other NGI PIs, researchers, ISPs, and vendors as they are available. Availability of these raw and correlated data will be invaluable in the development and testing of new Internet hardware and software products and planning for next generation protocols and technologies.
2. Technical Approach
a. Coral Monitors
"Coral" is a term used to describe the family of low-cost, high performance traffic flow monitors that are being developed by MCI's vBNS Team in collaboration with UCSD/SDSC's NLANR personnel. Through this MCI/NLANR effort, monitors operating at OC3 and OC12 levels have been developed with support from the National Science Foundation (NSF). OC3mon units are now being deployed at select internetMCI nodes, at vBNS and Internet-2 nodes, and at the NGI's West-coast Internet eXchange (IX), FIX-West. CAIDA is also working with several ISPs to deploy monitors on their backbones (under non-disclosure agreements). Several OC12mon units will be deployed in 1998. Given the high research and development cost associated with these monitors, neither NSF nor MCI have plans currently to develop monitors at greater than OC12 speeds.
However, for optimal configuration and management of high-performance networks, OC48 and beyond, near real-time acquisition and analysis of traffic flow data is needed. The ability to get a single, integrated, view of traffic load and network behavior is key to intelligent control of network resources in order to meet quality of service (QoS) requirements of demanding applications. These control functions must scale up as quickly as the complexity and capacity of the networks scale up. The development effort outlined in this task leverages current expertise and resources related to the development of monitoring and analysis for high-performance networks to produce a more sophisticated capability for next-generation networks, e.g. OC48.
Two options are also described below: (1) the development of a LIGHT Monitor (LIGHTmon) which would provide both security and engineering insights to analysts relative to the presence or absence of light (traffic) on SONET lines and (2) an OC192 version of the Coral flow monitor. We have separated these two activities from the base project since we feel that the technical requirements and costs associated with their development require clarification that can only occur during the development of OC48mon.
Base Period - OC48mon Development:
Similar to the OC3mon and OC12mon versions, an OC48mon device would provide realtime monitoring of packet headers traversing a node at OC48 speeds. Given sufficient RAM and storage capacity, the device would also permit capture of packet trace data.
As it is important to develop monitoring capabilities in parallel with the development of the switches to be monitored, we propose that the first phase of the OC48mon platform be implemented outside the switching devices. Passive optical splitters can be used to gain access to all (possibly 128) DWDM channels of a switch port at once, then they can be split up using WDM demultiplexors. Both these components utilize commercial-off-the-shelf (COTS) technology. Later phases can migrate the monitoring functionality inside the switches themselves and/or use switch-based tapping mechanisms for external equipment, if vendors are willing.
For applications where SONET is used as the outermost layer of modulation, we propose to build a device capable of monitoring packet headers (IP or other) at OC48 (2.5 Gbps) rates, with scalability to OC192 (10 Gbps) and beyond. Some of the ways we believe this can be done include: (1) widening of the on-card data pipelines to allow the same-speed devices to handle more data at the same clock speed; (2) on-card aggregation of packets into flows (in addition to the packet-header selection done by current Coral monitors); and (3) using faster hosts. Round-robin polling of multiple channels and ports could be achieved using commercially off-the-shelf (COTS) electromechanical switching boxes. Examples of uses that this device could be put to include: problem diagnosis, security intrusion detection, and capacity planning.
For the OC48mon, we will enlist the assistance of a subcontractor to build a card that accepts Tektronix's 80 MHz 32-bit Utopia-3 bus (for OC48c fiberoptics inputs) on one side and PCI on the other. The Utopia-3 interface is 32 bits wide and 80+ MHz fast (roughly twice the speed of similar OC12c devices). We intend to widen this bus to 64 or 128 bits using bipolar or Gallium Arsenide logic, to lower the clock rate by a factor of 2 or 4. The Xilinx 40125XV-1 FPGA's will be commercially available in 1998 and will have roughly four times the amount of logic per chip as the 4036EX which MCI/vBNS currently use one of to do ATM AAL5 reassembly on the OC12mon monitors, at roughly twice the internal speed. We estimate at most two of these devices would be required to build ATM reassembly for OC48mon per interface card, with two cards needed to monitor a bi-directional ATM link.
The PCI interface is probably 66 MHz fast and 64 bits wide, although we may have to make do with 32 bits wide and 33 MHz fast. In between, there will be GaAs or bipolar Si logic to widen the Utopia-3 bus while slowing it down, so that FPGA's can deal with the SONET payload data reliably.
These FPGA's will perform ATM AAL5 reassembly or HDLC framing/bytestuffing/scrambling, depending upon how they are downloaded. After selection of the data bytes for the packet's IP and TCP headers, those packet headers will be aggregated into flows. When no packet has been seen on a flow for a defined interval of time, the flow is copied to the host via PCI bus. SRAM on card will hold both ATM state per VPI/VCI (if doing ATM AAL5 reassembly) and incomplete flows and their hashtables.
Overview of possible link multiplexing schemes for these monitors, and how we see this proposal relating to them is described at the end of this section.
LIGHTmon (Option 1):
For light-transparent applications, where the switching equipment makes no demands upon the interpretation of traffic flowing down a DWDM channel (wavelength) while it is allocated to an application, we propose to build a monitoring platform which is capable of detecting the presence/absence of light as a measure of channel utilization. These detectors can operate at least an order of magnitude slower than those used to actually demodulate the data, because the switches cannot alter cross-connects faster than the granularity of a packet/cell, so it is cost-effective to have one for every channel on every port of a switch.
Equipment to passively tap the control channel(s) used to setup and tear down cross-connects within the switch will also be developed as part of this effort. Whether the control is on a separate WDM channel dedicated to the purpose, or time-multiplexed on each individual channel, or on an outboard electronic interface not available to the WDM-connected applications such as a serial, ethernet, or T1 port, it will still be feasible to gain access to and interpret this channel.
Information obtained from the light detectors and the control signaling interpretation will give the monitoring device the ability to measure utilization and allocation of channels. It will provide reports to engineers responsible for network design and policy and make possible real-time, feedback to control algorithms, both local to the switch and global to the wide-area network, in order for device to make informed call admission and routing decisions.
OC192mon (Option 2):
Based on the experience with the development of OC48mon, we propose that DARPA consider continued funding to support the development of an OC192 version of the Coral monitor. Note that much of the experience gained in building the OC48 monitor would be relevant to the software development needed for the higher performance monitor. The hardware (OC192 controller card) however would require a completely new architecture. Given the pace of advancements in switching/routing technology, we feel that a reasonable assessment of the requirements and costs associated with developing this hardware can be made by early 1999.
Multiplexing Schemes for OC48mon/LIGHTmon/OC192mon:
(1) In-band, fine-grain routing, with a fixed modulation scheme - This is the mode of operation of most networks today. LAN examples include 10/100/1000 Mbit ethernet, 100 Mbit FDDI, and 4/16 Mbit token ring. WAN examples include frame relay over T1/T3 and ATM over OC3/OC12 SONET. A header in front of each packet or cell indicates what receivers should do with it, such as "this is for endstation ralph" or "switch george should copy this to his port 2 and send to marcia". Stations may have their transmitters enabled all the time (as with SONET) or only when they have data and think they own the (shared) link (such as ethernet or FDDI), but they only use one modulation scheme.
We propose to monitor this multiplexing scheme using passive devices that understand the underlying modulation and addressing and protocols. The first of these will be OC48mon for IP inside SONET at OC48 (2.5 Gb) rate. We believe the same device can be reconfigured in software to decode IP inside ATM inside SONET, also at the OC48 rate.
One disadvantage of this multiplexing scheme is that the monitoring device (not to mention the switching equipment) must be re-engineered for each increase in speed of the equipment on the link. We intend the design rules and data reduction of OC48mon to be applicable to the design of a future OC192mon. Based upon our experience with OC12mon and industry trends, it is unreasonable to expect computers and their buses to scale at the same rate as network speeds (4X in one year). So we intend to reduce data from full link bandwidth to packet headers, and aggregate packet headers into flow descriptors, all on the interface cards.
Passive optical splitters to tap off 10% of the optical power, would be permanently inserted into all ports of a switching device. An electromechanical switch would have as its input the monitor-jack pairs of all the splitters, and would output to a DWDM demultiplexor. The 128 channels of the DWDM demux would pass through another electromechanical switch whose output is the OC48mon, which is itself only capable of monitoring a single channel. The switch-boxes allow the OC48mon to sample many ports on a round-robin or random basis, and are controlled by whatever entity polls the OC48mon. Multiple OC48mons could be connected to the same network of splitters and switch-boxes to view multiple ports at the same time. Although the switch-boxes have a latency on the order of milliseconds, this is not an issue for flows-based monitoring because we would want to monitor many complete flows while connected, and flows are typically many seconds.
A connection diagram of one half of the OC48MON is shown below:
(2) In-band, coarse-grain routing, with multiple modulation schemes - Headers indicate the destination and handling of the link for a fixed period of time, not a variable size packet. The time interval will probably be chosen to be the time it takes to send one large packet on the slowest link speed that will be supported (say 9500 bytes OC48), or some multiple thereof. The header is sent in a specified, fixed modulation scheme, then the payload is modulated as the user desires, with the caveat that it does not interfere with other DWDM channels on the same physical fiber. The switching devices will ensure that they do not confuse the user's modulation with a header by ignoring the channel until the timeslice has expired.
In essence, the fiber is "dark" while you own the timeslot, so you can employ analog modulation or digital, at any speed you like. This also means that the switching equipment does not have to get faster just because more bits got transferred during the timeslice. Also, as long as the monitoring equipment does not want to look at the user-level data (such as IP if only MAC headers are in the known-modulation header), it doesn't need to get faster as user-modulations do.
Note that this multiplexing scheme is different from ATM cells. ATM cells stay the same size regardless of the link speed, even though the avowed reason for fixed-size cells was to have deterministic time delay for media access, and more data can be packed into that time as links get faster. With ATM cells, packets always get broken into multiple cells, but with this multiplexing scheme, packets get clumped into groups that get larger as the user-modulation scheme gets faster.
SONET would probably not be used for the header modulation scheme, as it requires several SONET frames for clocks, pointers, and other control structures to synchronize on the 2 sides of the link.
Monitoring the control header requires demodulating it, for which we would build interface cards for a host computer. The size of the timeslice chosen for the multiplexing scheme would be critical in determining the cost of these interfaces. The choice of fields available in the MAC header would control how useful a single monitor would be: if source address is omitted, the only way to do a source-destination traffic matrix is to monitor all ports and reconcile the traffic in both directions. This might prove to be prohibitive.
In order to know when, within their timeslice, the user's transmitter is on, we propose building an array of inexpensive light sensors. They only need to respond to variation in intensity as fast as the smallest interval in the timeslice we would like to report upon, so if the timeslice is 32 microseconds (about 10KB at OC48 rate), and we want to report on one-percentage increments of that, each sensor must respond to (and timestamp) 320 nS events, which implies a 3.1 MHz clock rate. Also, since no demodulation is really done by this array, very little supporting electronics need to fit between it and the host interface. Of course, we intend to use as much COTS hardware as possible, to minimize cost and development time.
Connection diagram of one half of the inband multimodulated monitor:
![[img]](http://www.caida.org/funding/ngi/content/images/DIAG_2~1.GIF)
3) Out-of-band, coarse-grain routing, with multiple modulation schemes - This is another transparent time-multiplexing scheme. It is similar to option (2) except that the header is carried on a dedicated DWDM channel on the same fiber as the many user channels. The headers for data on many channels can be aggregated on a single channel because of their low bandwidth relative to the timeslice size. Some optical switches may choose to have a low duty cycle, which further loosens constraints on the timing of the modulation chosen for the control channel.
This scheme is even easier for users, as the fixed modulation scheme for changing cross-connects does not need to be supported by the same transmitter as the custom user modulation scheme. If desired, the control information can even be generated by a third party.
Monitoring this scheme is very similar to (2) above, but easier because the control-channel demodulator must only be built once per fiber, not once per channel, in order to get the same coverage.
Note that extreme cases of out-of-band control include: having a TTL line to raise per channel you want to request, having an RS232 serial port per optical fiber, and having a single T1/ethernet per optical switch for a centralized control machine to setup cross-connects. The latter case is the ultimate in ease for monitoring, since only one low-bandwidth channel must be monitored per switch, and COTS interfaces are readily available.
Connection diagram of one half of the out-of-band multi-modulated monitor:
![[img]](http://www.caida.org/funding/ngi/content/images/DIAG_3~1.GIF)
Or if the control channel is one-per-switch:
![[img]](http://www.caida.org/funding/ngi/content/images/DIAG_3~2.GIF)
The following is a sample report from the light-transparent monitor. Note that although this is in humanly-readable format and accumulated over a long time-period (in this case 5 minutes), the same information would be available to the light switches and other equipment, both locally and globally, on a near-real-time basis. This information will enable that equipment to do load-balancing in the wide area as well as locally.
--------------
time since last polling: 301.9 seconds
port 3 channel 12 of 128 lit 6% of the time, allocated 8% of the time
warning: port 2 channel 53 lit on 21 occasions when not allocated,
for a total duration of 32.5 seconds
port 4 channel 11 allocation details by source device (owner):
3% a.reston.monet.net
7% p.pym.monet.net
by destination device:
port/channel cross-connects (pairings) by utilization (% of their own
allocated time):
7/14 to 2/14: 81%
4/11 to 6/123: 57%
4/23 TO 9/23: 45%
allocation-to-usage delay by port/channel:
4/11: 120 mS
4/13: 16 uS
7/10: 5 uS
7/18: 56 uS
unused-to-deallocation delay by port/channel:
data gaps:
4/11: none
4/13: 1 with 23 uS total duration
7/10: 5 with 3 mS total duration
Additional technical and operational details on Coral monitors are available at http:www.caida.org/tools/measurement/coralreef/ and in the paper enclosed in Section III of this proposal.
b. Internet Tomography Mapping/Modeling
The Internet infrastructure is not static, nor does it have any direct relationship to physical (geographic) localities. Topological hierarchies and routing behavior change frequently. Tools to discover and depict it must be capable of describing the Internet as a dynamic, metamorphic entity [1995 thesis research by V. Paxson (LBL) addresses the dynamic properties of routes]. Tomographic depiction of this infrastructure requires correlation among:
-
BGP relationships for major Internet service providers
-
Performance information between specific nodes
-
Characterization of specific paths, e.g. available bandwidth and throughput across specific links
Our plans for developing such a tool set are described below.
BGP Relationships: NLANR/CAIDA (Dave Meyer, University of Oregon) is currently collecting BGP peering information from a router in Oregon. Nine key ISPs are participating in this project, peering with the University of Oregon router. They include large ISPs such as MCI, Sprint, CerfNet, Verio, BBN, providing a relatively comprehensive picture of their peering behavior and inter-provider relationships underlying the Internet infrastructure. (Several additional ISPs have indicate interest in participating in this effort.)
Through NLANR, Hans-Werner Braun (UCSD) is developing a tool to construct and visualize interconnection maps by Autonomous System (AS) numbers. The tool downloads about 40 MB/day of Meyer's raw BGP routing table data and derives several statistical data sets: histograms of net/mask (prefix) distributions; summaries of ASes according to their number of peers; lists of which peers each AS has; and a list of net/mask pairs (`prefixes') that have multiple originating AS numbers across the observed BGP peers.
To visualize the reduced (though still huge) quantity of resulting data, a separate web-based form generates virtual reality modeling language (VRML) objects in real-time based upon client specifications. Details on this tool are available at http://rwac.ucsd.edu/AS.) Insights provided by such BGP analysis tools are fundamental to our ability to scale the Internet and to debug routing and other problems in real-time.
Path Information / Performance:
NLANR/CAIDA have roughly a dozen machines deployed across the Net (at FIX-West/Palo Alto, MAE-West/Palo Alto, NCAR/Boulder, NCSA/Chicago, PAIX/San Jose, PSC/Pittsburgh, MCI/Reston, UofOregon/Eugene, NCREN/Raleigh, and SDSC/San Diego). We anticipate that we will need to deploy a minimum of 20 additional hosts at various points throughout the U.S. and abroad in order to gather sufficient information on traffic paths and performance to provide meaningful insights regarding autonomous systems and the Internet as a whole. NLANR/CAIDA has close working relationships with many ISPs in the U.S., Asia and Europe, as well as with NGI networks and Internet-2 gigaPoPs who have expressed interest in collaborating with us on measurement endeavors such as this one.
CAIDA and NLANR currently collaborate with numerous public and private networks and research institutions, including established relationships with individuals associated with the following networks: ANS (US); APAN (Korea/Japan); AT&T (US); BBN (US); CERFnet (US); DRA (US); DREN (US-NGI); Ebone (Europe); ESnet (US-NGI); IIJ (Japan); LINX (Europe); MCI (US); NSI (US); NUS (Singapore); PIPEX (Europe); RIPE (Europe); Terena (Europe); Verio (US); Zocalo (US). These networks and key university and corporate institutions will be contacted regarding deployment of servers and participating in this effort. CAIDA is also establishing a peering facility at SDSC for ISPs operating in the San Diego area, to be operational Jan 1998. We expect that participating institutions will provide approximately half (10) of the necessary servers, and have budgeted in this proposal to cover the other half.
CAIDA collaborators are currently working on a robust variation of ping that can be obtain path and reachability data on tens of thousands of hosts (including those on high performance networks). The tool, skitter, uses a Patricia tree structure of polled addresses to achieve fast lookups (O(log(N)) on the input side (where ICMP packets arrive). It is designed not to hang under adverse traffic conditions (unlike several commercial NMS pollers that are often nearly synchronous and can stall for many minutes if a few nodes are unreachable). A skitter design goal is to sustain continuous monitoring of up to 60,000 hosts, with a 60-second maximum poll cycle and minimal number of packets without compromising the integrity of the sought up/down indicators.
Skitter's record-route option is optimized to allow tracking first nine hops to each polled destination, a critical feature for tomography mapping. Beyond nine hops, and in cases where record-route does not work, we will use a method similar to traceroute.
![[img]](http://www.caida.org/funding/ngi/content/images/skitter_path.gif)
Note that several platforms do not currently implement the IP record-route option, and firewalls sometime refuse its use. It is listed only as a 'MAY' rather than 'MUST' in RFC-1122 (host requirements) and RFC-1812 (router requirements), so vendors can still ignore them and remain IP-compliant.
Skitter will therefore result in a spanning tree structure originating at the polling host and extending into the infrastructure at least nine hops deep. Data will then be aggregated into a centralized database for correlation and depiction as a top-down, macroscopic view of the Internet. Juxtaposition of such data sets has been a remarkably unattended area given the promise it offers for insights into the infrastructure as a whole.
Path information provided through skitter will be useful in two ways: for collapsing the spanning tree originating at the skitter host so it can effectively pinpoint problems and for deriving and mapping dynamic changes in Internet topologies. It will also be a useful quality check for backbone engineers to compare the expected with actual results of new configurations.
We will also use skitter to track approximate round trip times (RTT) and variances. (Note we will have to compromise here and our focus is not on analyzing per link or path latency performance per se. In particular with our planned use of record-route to extract topology information, we take a hit on our ability to measure RTT accurately. We can't have it both ways. Routers typically do substantially more work dealing with optioned packets (as record-route ones will be), which will impact RTT values. We might still be able to track valid variance data from record-route packets, but actual legitimate RTT values will require non-optioned packets (and a lot more design constraints as well).
![[img]](http://www.caida.org/funding/ngi/content/images/skitter_RR_out_pkt.gif)
At left is a picture of the ICMP echo request to be sent by skitter with the IP record-route option. The only fields that we will fill from application space:
-
user transmit timeval in the payload
-
destination IP address in the payload
-
record-route will be initialized
-
hdr len will be set when we enable the IP record-route option
-
all of the ICMP header fields (type, code, checksum, identifier and sequence number)
Below is a picture of the ICMP echo request to be sent by skitter when not using the IP record-route option. The fields we will fill from application space include:
-
user transmit timeval in the payload
-
destination IP address in the payload
-
ttl in the header (via setsockopt(fd,IPPROTO_IP,IP_TTL,...))
-
all of the ICMP header fields (type, code, checksum, identifier and sequence number)
![[img]](http://www.caida.org/funding/ngi/content/images/skitter_ttl_out_pkt.gif)
Three-dimensional graphs of RTT variance matrices provide a method to easily indicate regions of the infrastructure experiencing abnormal delay. Plotting deltas from day to day or week to week also helps track changes in traffic patterns. As mentioned earlier, we envision running skitter to up to 60,000 hosts (starting with our internal latitude/longitude database and addition at least one host from each route in the IRR). At 300 pps outbound (~200 kbits/sec), a poll cycle would typically take around 200 seconds. A route discovery algorithm would be run off-line daily.
Skitter would sleep for 10-60 seconds (depending on length of poll cycle), but otherwise run continuously. A lower pps rate, (possibly under 100 due to 100 Hz timer resolution) might enhance accurate RTT measurements. But we go into the activity with a lot of background trying to measure, analyze, and visualize RTT, and are acutely aware of the difficulties. It is difficult to trust RTT measurements when the timestamping occurs in the user space, particularly for a program that introduces non-trivial network load on the polling host if we want a short poll cycle.
The data derived through skitter will be backstopped by a comprehensive database of physical topologies. CAIDA has the initial stages of such a database being developed to support its java-based mapping tools, e.g. MapNet, MapRoute, MapMBONE, MapCache, and MapPerf. Note that this CAIDA database is also a repository for latitudinal and longitudinal data for identifying the physical location of these nodes.
Use of skitter, particularly integrated to the BGP data, will allow engineers to discern who is announcing what to whom over specific paths. Note that these data will not answer why this is happening or if such traffic behavior is optimal, but it can provide real-world inputs to traffic models/simulations designed to answer these questions. These data can also help pinpoint routing instabilities and other anomalies, and track their secondary, downstream effects, e.g. on roundtrip times, availability, packet loss across specific paths. A repository of these data/analyses will significantly enhance our predictive capabilities on the Internet.
Path/Flow Characterization:
In addition to characterizing performance between to end nodes and charting the path between them, we will need to perform in-depth, periodic characterization of the actual path between specific hops. Tools such as TReno [M. Mathis, PSC] and pathchar [still in beta - V. Jacobson, LBL] provide information on bulk throughput and link-by-link characteristics. While too invasive to be run often, these tools, along with mping, ttcp, and related performance tools provide further depth to our understanding of traffic behavior.
At sites where public flow tools are available (e.g., Fix-West) we will also attempt to correlate these data with the other path and performance data being gathered for the site for a more comprehensive description of network traffic patterns and characteristics.
Visual Depiction:
Through the CAIDA, UCSD/SDSC personnel are developing visual tools for describing and diagnosing problems with Internet traffic flows. Java-based tools such as: MapNet(a 2D map of ISP backbones http://www.caida.org/tools/visualization/mapnet ); MapRoute(a tool for visually depicting traceroute information); MapMBONE(a depiction of mbone topologies); MapCache (a visual depiction of the NLANR global cache hierarchy (almost 400 caches) and related traffic flows); and MapPerf (a visualization tool for describing traffic performance data) help provide alternative means of depicting data about the Internet's topologies and performance. Virtual reality modeling language (VRML) tools are also being developed by CAIDA collaborators to depict BGP peering relationships of major ISPs -- see rwac.ucsd.edu/AS.
The proposed tomography tool will build upon the lessons learned in the development of these tools. Currently, we are evaluating 3-D languages (both VRML and Java-3D) as software for developing the Tomography tool. (Note: VRML browsers are currently available on PCs, Macs, and SGI workstations, but not other UNIX boxes. Java3D is in alpha and will be available on PCs and Sun workstations, but not other UNIX boxes. In addition, browsers for Java-3D, the more powerful of the two languages, are not expected to be publicly available until late 1998.)
Tomography-based Modeling:
The primary product of this task will be the performance/path/BGP data and tomography tool used for visualizing the data. However, we also plan to develop a simple traffic model which can be used for conducting 'what-if' scenarios of traffic behavior based on these data. This model is intended as a proof of concept and will be used primarily as a entre to working with commercial vendors who invest millions of dollars annually in developing commercial network simulation products. It is not our intention to usurp the service they provide to the Internet community, but rather to provide them with tools and data to make their products more directly relevant and useful to emerging high performance networks. Organizations with whom we are communicating regarding these topics include: Cisco (NetSys), Mil-3 (Opnet), Makesys, USC (Vint), and LBL (ns).
c. Security
The opportunities for provision of security capabilities, defenses and countermeasures, are significantly more complicated in the high performance network environment than in today's commercial Internet. Common defense mechanisms such as flow filtering on routers are generally not available for high performance network -- or are available at exorbitant prices.
We believe that security for NGI networks requires a multi-tiered, defense-in-depth posture that employs: targeted use of encryption, end-host defense mechanisms (e.g. host firewall capabilities), ubiquitous network monitoring and enforcement, and proactive investigative tools. Our proposed effort leverages key team capabilities in traffic flow measurement (NLANR/MCI), traffic monitoring and enforcement (SDSC), and investigative tools (iMCI). Specifically, we propose to: (a) leverage development of OC3mon, OC12mon (and beyond) to provide ubiquitous, low-cost stations that monitor and enforce network security policy; (b) generalize and expand the capability of the iMCI DOStracker to optimize performance next generation networks.
We also plan to explore options for effectively conducting real-time traffic correlation analysis, based on data from router polling, as a means of tracing persistant denial-of-service attacks cross heterogeneous network nodes outside of administrative control.
Light-weight, low cost firewalls for High Performance Networks:
Security of high performance networks requires fundamentally different strategies than the firewall-centric architectures typically employed in present generation networks -- particularly given the performance constraints of commercially available firewall technology. It also requires cost-effective approaches that can be deployed with relative ease through a network's infrastructure.
SDSC personnel will work with MCI to test the limits of high performance real-time filtering of flows using both the OC3mon and OC12mon Coral monitors. Trade-offs between on-card, firmware filtering using Field Programmable Gate Arrays (FPGAs) and bus-limited on-host analysis will be evaluated as it relates to effective security-related filtering vs. optimum performance. For the case of application-level packet filtering, a subset of the well-known BSD Packet Filter (BPF) language specifying source and destination IP address/network and port is sufficient. This specification may be implemented as a very simple binary logic using bit masks, AND, OR, and XOR. In fact, such a filter is a deterministic case of a Binary Pattern Correlator widely used in pattern matching and frame synchronization applications. We will develop the required front-end modules needed to interface the BPF filter specifications with FPGA configuration design software (e.g. Xilinx Foundation design tools).
In the case of filtering to the transport layer, it is sufficient to capture only the header of the first packet of a connection-oriented flow. That, combined with the additional on-card filtering, must provide enough data reduction to overcome the PCI bus limits (about 1 Gbps). The traffic performance effects of this packet filtering will be quantified, and a qualitative analysis of the security benefits associated with its use shall be prepared and posts to the Coral website.
As a performance risk mitigation strategy, we propose to develop a hybrid approach combining centralized, passive real-time flow analysis and coordinated active responses from the monitor or delegated to downstream nodes. Code would be written to permit Coral monitors to perform two distinct and complimentary functions: (1) network security policy compliance auditing and (2) policy enforcement.
The Compliance Module would permit the host to contain specifications of traffic policy; usually that relating to destination IP addresses and port numbers. SDSC has developed a similar compliance monitor for FDDI environments (100 Mbps) which has been deployed in continuous operation in the SDSC production environment for over one year. Tests will be conducted to see how much of the traffic analysis function could be handled as a separate process (or in the kernel) of the monitor without significantly degrading the monitor's performance.
Traffic matching a specified policy would generate a signal indicating detection of non-compliant traffic. In the simplest cases, a handler would enter a message in an audit log. Other responses include signaling another monitor dedicated to complete packet capture of suspicious flows. These data may then be used by an Intrusion Detection System for high-level analysis and archived as intrusion evidence.
The Policy Enforcement module would be integrated as an active signal handler in the Compliance Module framework. One option to which will be evaluated is that of developing an IP enforcement mechanism that terminates non-compliant TCP traffic by transmitting forged RESET packets to both end-hosts. [This is the approach employed at SDSC.] Other mechanisms, such as attacking the virtual circuit control protocol to either hijack the circuit or initiate tear-down on a switch with which one has administrative control will also be explored.
Denial of Services Detection. End-host Tracing
Source-based tracing is essential for all networks, including the next generation's high performance networks where flaws in early-release hardware/software and new protocols will provide a wealth of opportunities for attacks. Denial-of-service attacks, are expected to increase in frequency, with attackers using forged source addresses to obscure their location. The emergence of nomadic computing will undoubtedly provide additional opportunities for attacker obscurity. In the face of such threats, enhancing networks' abilities to trace the source of the attack beyond their networks' boundaries is essential.
SDSC staff will work with internetMCI personnel to expand the capabilities of MCI's DosTracker software to operate in high performance networks and interface with alternative (non-Cisco) nodes. DosTracker works both proactively and reactively, by constantly monitoring a network and, upon detecting a denial-of-service attack, automatically traces the transmission to its physical source. The application is designed to detect and trace SYN, ICMP flood, bandwidth saturation, and concentrated source attacks. Details on DosTracker are available at http://www.security.mci.net/dostracker. Specific functionalities that we feel are critical to the development of this capability for high performance networks include:
-
modularizing attack signature detection to facilitate rapid implementation of new signatures as new attacks are launched;
-
developing algorithms for identifying anomalies in traffic patterns in near real-time across high performance networks;
-
generalizing router interfaces to enable interoperation with non-Cisco platforms;
-
developing source-detection capabilities and interfaces of Coral monitors to participate at information sources in tracking activities.
Underlying the challenges of effective traces in high performance inter-networks is the issue of secure access and sharing of tracking data among platforms that are under differing networks' control and administration. While a comprehensive design is beyond the scope of this proposed work, we will develop a pre-conceptual design of a system outlining "agent" design, access control, auditing, and encryption strategy to facilitate this broader sharing of tracking data.
d. Data Storage / Analysis
Information being developed through the various distributed monitoring devices could amount to terabytes of data by year 3. To be useful for real-time engineering analyses of Internet traffic patterns and behavior, we are proposing that these data be encrypted at the collection points and aggregated at SDSC on a high performance RAID disk array system.
Under the NLANR effort, SDSC plans to deploy a scalable disk array for use in monitoring NSF High Performance Connection (HPC) institutions and related vBNS Partner Institutions. Under this NGI proposal, we have budgeted for an additional controller and 11 additional 9 GB disks. In addition, we will use SDSC's High Performance Storage System (HPSS) for back up of all data.
Individuals active in SDSC's DARPA-funded Distributed Object Computation Testbed (DOCT) project will take lead roles in the development of the Tomography database and storage system. Information on the DOCT project can be found at http://www.sdsc.edu/DOCT/
3. Definition of Terms
CAIDA - Cooperative Association for Internet Data Analysis
Cell - fixed size, typically 53 bytes
Channel - one of 128 wavelengths in the 1550 nm band
COTS - commercial off-the-shelf
DWDM - dense wave division multiplexing
NLANR - National Laboratory for Applied Network Research
Packet - variable size, typically 64 to 9500 bytes
Port - somewhere to physically connect a fiber to a switch
Reassembly - process of putting many cells together into a packet
SAR - segmentation and reassembly
SDSC - San Diego Supercomputer Center
Segmentation - process of breaking a packet into many cells
Tomography - a technique of integrating information (particularly visual) in a such a manner
to permit depiction of planes of data in isolation or combination with each other; integrated data can
support diagnostic and predictive/diagnostic analyses of intricate routing, peering relationships,
and traffic anomalies in the Internet infrastructure.
C. DELIVERABLES
Key deliverables for this project are described in the sections below.
1. Coral Monitors
A prototype of the OC48mon monitor will be delivered during year 2 of the project. This will include two OC48c-to-PCI bus interface cards (prototypes) with firmware loaded to the cards, software to be run on the host computers (that provides a report when polled of TCP/IP flows it has seen on the attached link). This deliverable will also include software for a central site to aggregate these reports into graphs in response to web page queries.
Similar to the OC3mon and OC12mon monitors, all code developed through this effort will be publicly posted on the CAIDA/NLANR websites. Hardware specifications for building these monitors using COTS products will be detailed on the websites once they are available. Interested individuals in the community will be invited to participated in the Coral Developers mail list (coral-dev@nlanr.net) where design issues and debugging of Coral monitors is discussed.
Performance tests and data acquired through initial testing/deployment of prototype monitors will also be made available via the website. CAIDA/NLANR will also actively work with the community to encourage third parties to augment the efforts being undertaken through this project. Involvement by interested organizations has assisted in getting the OC3mon ported to additional platforms. Other NGI agencies, e.g. NIST, have indicated interest in participating in future Coral monitor development and testing.
With the concurrence of the DARPA Program Manager, a revised Scope of Work/Technical Approach and cost proposal will be provide in month 12 relating to the LIGHTmon (Option 1) and/or in month 15 for the OC192mon (Option 2). We feel that spacing these tasks in this manner will permit more realistic assessments of their technical feasibility and costs. It will also permit time for associated technological advancements.
Option 1 and/or 2:
The deliverables for these option tasks will include a control channel monitor, a 32-channel light sensor array with PCI interface card, along with software to provide reports such as the one listed above, and to generate graphs in response to web page queries. As in the OC48mon, all details will be posted on the web and specifications, test results, and other activities will be discussed publicly on the Coral Developers mailing list.
2. Tomography Mapping / Modeling
The preliminary Internet Tomography graphs and associated AS and routing data collected through this effort will be available on the CAIDA website http://www.caida.org by the 3rd quarter of year 1. The skitter tool and code for other measurement/diagnostic tools used in preparing the tomography graphs will be made publicly available for comment/use as they enter the beta stage of development. By early year 3, we anticipate a comprehensive database (historical and current) will be available for use by Internet engineers and others -- using the tomography grapher or traffic modeling front-end. Prototype versions of these data and tools will be available during year 2.
During year 3, code for a tomography traffic model will be available for comment. Updates will be posted to the website as they are available. We hope to collaborate with one or more network simulation software company (e.g., MakeSys, Cisco, HP, ...) who will incorporate the real-world tomography data into their enhanced WAN simulation products.
3. Security Security related scripts, software and filter firmware associated with the Coral flow monitors and internet MCI's DosTracker will be made available in beta form by the during Year 2. The Hybrid Compliance/Enforcement modules will be available in year 3. Details of the testing and evaluation associated with the Coral monitors will be discussed on the Coral Developers mailing list and posted on the CAIDA and NLANR websites as they are available.
4. Storage / Analysis
Significant quantities of Internet topology, performance and traffic data will be gathered through this project. How these data are stored and how analysis is accomplished will be detailed on the web. Data (filtered to protect privacy) will be available on the web.
General:
It is CAIDA's policy to make Internet data available when ever possible to other researchers for use in their projects. We would encourage them to notify us of their findings/conclusions and any relevant technical papers so that we could point to these products from our websites. In addition, the UCSD/SDSC and MCI and other researchers involved in this project will prepare technical papers on each of these thrusts for journal publication and presentation at major events.
All code and schematics developed through this project will be publicly available. Neither MCI or UCSD claim proprietary rights on any software or firmware developed through this initiative.
D. STATEMENT OF WORK
The statement of work for this project is described in sections 1-4 below.
Task 1. Coral Monitors (MCI-Lead)
OC48mon
Year 1
-
development of specifications and schematics for the OC48 monitor
-
subcontract with hardware firm to develop the OC48 controller card
-
develop OC48 host code
Year 2
-
enhance OC48 host code
-
develop aggregation/analysis code
-
test and evaluate the OC48 monitor (both at MCILab and CAIDALab)
-
deployment, test and evaluate the OC48 monitor on the vBNS and the NTON optical network, as appropriate
LIGHTmon
Year 1
-
with concurrence of the DARPA Program Manager, submit revised Technical Approach/Scope of Work and Cost Proposal for development of the LIGHTmon
Year 2 - if option is exercised
-
development of specifications for the monitor
-
subcontract with hardware firm to develop the LIGHTmon controller card
-
develop LIGHTmon host code
Year 3 - if option is exercised
-
enhance LIGHTmon host code
-
develop aggregation/analysis code
-
test and evaluate the LIGHTmon monitor
-
test and evaluate the LIGHTmon monitor on the NTON optical network, as appropriate
OC192mon
Year 1 - No activity
Year 2
-
with concurrence of the DARPA Program Manager, submit revised Technical Approach/Scope of Work and Cost Proposal for development of the OC192mon
- if option is exercised
-
development of specifications for the monitor
-
subcontract with hardware firm to develop the OC192 controller card
-
develop OC192 host code
Year 3 - if option is exercised
-
enhance OC192mon host code
-
develop aggregation/analysis code
-
test and evaluate the OC192mon monitor
-
test and evaluate the OC192 monitor on the NTON optical network, as appropriate
Task 2. Tomography Mapping / Modeling (UCSD/SDSC lead)
Year 1
-
deploy servers throughout the Internet
-
develop a database for storage/analysis of Tomography-related data
-
develop tools to analyze and visually depict these data
Year 2
-
collection of data from distributed sites throughout the Internet
-
develop web-based query forms to permit users to access raw and correlated data
-
enhance the Tomography analysis/visualization tools
Year 3
-
increase collection of data and number of deployed measurement hosts
-
enhance the Tomography analysis/visualization tools
-
develop a prototype Internet traffic model and work with vendors to incorporate real traffic data capabilities for their future releases of WAN simulation software
Task 3. Security (UCSD/SDSC lead)
Year 1
-
port kernel packet filtering code to Coral monitors; conduct performance studies of in-kernel header filtering
-
integrate security compliance monitor software in OC3mon; conduct performance studies of real-time header capture and security filtering
-
generalize and extend router interface in DOStracker software to interoperate with other (non-Cisco) nodes
-
develop initial studies of traffic correlation analysis for denial-of-service attack tracing
Year 2
-
construct prototype firmware (FPGA) packet header filter
-
perform proof-of-concept demonstrations of broadband (e.g. ATM) security enforcement by means such as NNI protocol attacks, switch modification, connection hijacking, or IP RESET forgeries
-
integrate security compliance monitor software and filtering firmware in OC12mon; conduct performance studies of real-time header capture and security filtering.
Year 3
-
implement and demonstrate security policy enforcement module in OC12mon
-
identify scalability properties of hybrid security enforcement model to higher performance networks
-
develop conceptual design of secure router data sharing framework for network tracing
Task 4. Storage / Analysis (SDSC/GA lead)
Year 1
-
develop a database for these data, building upon the flow data elements described at http://www.vbns.net/ and the topology data collected through the skitter tool and the Tomology initiative
-
begin data storage and development of analysis code
-
develop reporting formats for summarizing data
Year 2
-
collection/storage of data
-
develop analysis code and reporting formats
-
posting of standard analyzed, correlated data to the website
Year 3
-
enhance/expand the database and storage system
-
enhance analysis code and reporting formats
E. MILESTONES
Milestones for this project are described below:
Task 1:
-
completion of schematics (month 6)
-
contract with subcontractor (month 6)
-
completion of prototype (month 14)
-
completion of test/evaluation (month 18)
Task 1 Option 1:
-
submission of detailed cost and technical proposal, at request of DARPA Program Manager (month 12)
-
completion of test/evaluation (month 18)
Task 1 Option 2:
-
month 15 - submission of detailed cost and technical proposal (at request of DARPA Program Manager)
-
completion of test/evaluation (month 18)
Task 2:
-
finalize methodology; initiate deployment of measurement (month 6)
-
preliminary tomography graphs/formats (month 9)
-
prototype traffic model (year 3)
Task 3:
-
security related scripts, software and filter firmware for Coral monitors and internetMCI DosTracker (end Year 2)
-
hybrid compliance/enforcement modules (end Year 3)
Task 4:
-
data formats and initial reporting formats completed (end Year 1)
-
historical/current traffic/topology database (end Year 3)
F. TECHNOLOGY TRANSFER
This proposal is being submitted as a part of the Cooperative Association for Internet Data Analysis (CAIDA). CAIDA, and the National Laboratory for Applied Network Research (NLANR) from which CAIDA originated, is based on principles of collaboration and cooperation. The code for all tools and products developed by CAIDA projects are freely available to the public and are generally downloadable from the CAIDA web pages.
In the case of the Coral monitors, all board schematics, layouts, silkscreens, gerber plots, will be in the public domain, as will related software and FPGA downloads. The subcontractor selected for the PCI card development will provide the same boards to the general public at the same per-unit cost that the CAIDA team paid, without requiring payment of additional engineering fees.
For the Coral monitor development and the Coral security effort, all progress, findings and conclusions will be posted on the Coral-Developers mailing list which has the active participation of researchers, vendors, and ISPs throughout the U.S. and abroad.
Based on our experience on the OC3mon and OC12mon monitors, we anticipate that the costs associated with assembly and deployment of these Coral monitors will be orders of magnitude less than commercial products (when they become available). Vendors and external researchers alike will participate in the development of the monitors which should permit spin-off technologies to develop concurrent with these efforts.
Execution of the Tomography task will be simplified if it includes active participation of several ISPs, exchange points, and major user sites throughout the world. By enlisting the support of the community, we believe that we can not only ensure the early acceptance of these efforts, but also their sustainability once DARPA funding concludes. As important, we will encourage others to initiate spin-off efforts using either the tools or the data being developed through this project. Some of the ISPs that have participated in CAIDA and NLANR related activities which we hope to work with on this task include: ANS, SREN, AT&T, BBN, CERFnet, DRA, DREN, Ebone, ESnet, IIJ, LINX, MCI, NSI, NUS, PIPEX, RIPE, Terena, Verio, and Zocalo. FIX-West, PAIX, and MAE-West are all hosting CAIDA/NLANR equipment currently and may be ask to host additional measurement hosts.
Vendors who are currently active participants in CAIDA/NLANR projects include CISCO, Digital, and Sun. These vendors, as well as numerous start-ups and traditional telecommunications firms have expressed a strong desire for reliable Internet traffic data against which they can design and test their products. By enlisting their involvement early in this process, we can help to ensure that the data are presented in a manner that is relevant to vendor needs, as well as the needs of primary target community -- the Internet backbone engineers.
Technical papers, presentations and journal articles will be prepared by the CAIDA team personnel. We will also encourage others in the research community to publish findings and conclusions using data or products derived through this project.
G. COMPARISON WITH OTHER ONGOING RESEARCH
Coral Monitors: We know of no other Internet traffic monitoring equipment which is currently available or under development, which operates at OC48 or OC192 speeds or meets the needs identified for the LIGHTmon, e.g., use in security-focused monitoring initiatives. If other efforts are underway in this area, it is unlikely that they are developing open architecture options (as proposed in this project), nor are they making all data, schematics, and code publicly available.
Tomography Mapping/Modeling: We are aware of and are coordinating with several related traffic monitoring initiatives, including those of the Common Solutions Group (CSG), Internet-2, Merit's Internet Performance Measurement Analysis (IPMA) effort, the National Internet Measurement Initiative (NIMI), and the Cross Industry Working Team's Internet Performance Working Team (XIWT/IPWT). None of these organizations are attempting to aggregate, analyze or visualize their data at a scale or for the engineering purposes identified in this proposal.
Security: SDSC/PICS staff are actively developing concepts and tools related to host and LAN security and post-intrusion investigations. The proposed work in the broadband environment complements that scope. Research engineers at the Network Storage Group have developed a specialized (and expensive) ATM firewall that filters a subset of the IP-in-ATM protocol. Several researchers (Xerox, Bellcore, Sandia National Labs, TIS, IBM) have actively studied ATM protocol modifications that would facilitate secure setup, authentication, and filtering. This work does little to address identification and termination of malicious traffic, and where it does so, protocol restrictions are unlikely to be practical and adopted (e.g. one-to-one VC-session mappings).
Storage: While there are numerous mass storage options available, none are being utilized for real-time acquisition, analysis, and visualization of traffic data via a publicly-available web front-end.
Related research by UCSD/SDSC's Networking Research Division include:
NLANR (www.nlanr.net) - a collaboration among the NSF-supported supercomputer centers to further and support use of the NSF/MCI very-high-performance Backbone Network Service (vBNS). Through this effort, UCSD/SDSC researchers lead initiatives to develop and deploy tools for traffic flow characterization and make related data publicly available; to visualize Internet traffic flows and topologies; and to deploy/support a global cache hierarchy. Future NLANR Measurement and Operations Analysis Team (MOAT) efforts will focus on deploying Coral monitors at NSF-supported High Performance Connections sites (particularly Internet-2 gigaPoPs), monitoring / analyzing traffic flows through the vBNS, enhancing data analysis/storage capabilities, as well as continuing development of the cache hierarchy. UCSD/SDSC personnel will also assist in extending Coral to support alternative architectures, e.g. DEC Gigaswitch, IP over Sonet, and DS-3. Efforts are also underway within MCI/vBNS to develop 'playback' and TCP behavior analysis capabilities of the monitors.
CAIDA (www.caida.org) - is a new effort at UCSD/SDSC aimed at data collection and analysis in support of a robust and scalable commercial Internet. Participants include government sponsors, ISPs, Internet eXchange (IX) providers, and vendors. The CAIDA Tool Taxonomy and mapping/visualization tools described above are current priorities. Visual depictions of network topology maps, MBONE and cache traffic flows, and 2-D and 3-D illustrations of BGP peering relationships between autonomous systems (ASes) are available at http://www.caida.org/tools/. A CAIDA/SDSC-sponsored exchange point for regional traffic will be established at SDSC in January 1997. [Note SDSC currently hosts several large users, e.g., SDSC, UCSD, and Scripps; peers with NGI networks such as the vBNS, DREN, and ESnet; and hosts several carriers, e.g., TCG (OC3), MCI (OC3 & OC48), PacBell (OC3), TimesWarner (OC3), Worldcom (OC3).] CAIDA is also in the process of establishing the CAIDALab were CORAL monitors are being assembled and where Internet hardware and software will be tested and evaluated (by late 1998, real-time Internet traffic will be used for these purposes).
PICS (pics.sdsc.edu) - The Pacific Institute for Computer Security (PICS), was organized at SDSC in 1995 through funds provided by the Institute for Defense Analyses (IDA). PICS seeks to conduct and publish leading-edge research into real-world computer and network security issues, with an emphasis on real solutions to real security problems. Work especially relevant to the proposed scope includes: IP security compliance and enforcement software for FDDI networks, adaptive IP filtering for security monitors, and kernel-based host firewalls. PICS has contributed to several CERT advisories, and PICS researchers already have developed numerous security tools which are publicly available.
DOCT (www.sdsc.edu/DOCT/) - The Distributed Object Computation Testbed (DOCT) is creating an environment for handling complex documents on geographically distributed data archives and computing platforms. A persistent object representation based on the Legion Computation environment is being used to integrate text search systems, document handling systems, and intelligent agents that support electronic submission of documents. The documents are stored on distributed databases systems that are integrated with archival storage systems. Research activities include development of a distributed scheduling system, and development of the requirements for supporting electronic filing of documents.
vBNS (www.vbns.net) - In 1995, MCI was named as the provider for the National Science Foundation's very-high-performance Backbone Network Service. Under the terms of the cooperative agreement with the NSF, MCI provides Internet Protocol (IP) and connectionless networking protocol (CLNP) services at 622 megabits per second (OC12). Employing SONET and other fiber optic and high-speed switching and transport technologies, the vBNS will migrate to gigabit speeds by the late 1990s. The vBNS was designed jointly for scientific and research communities and originally provided high speed interconnection among NSF's supercomputing centers and connection to NSF-specified Network Access Points. As of November 1997, 71 institutions were connected to the vBNS, including 5 supercomputing centers. As a part of this NSF/MCI collaboration, MCI researchers have developed and deployed the OC3mon and OC12mon monitors. At this stage, MCI and NSF have no agreement to continue this research toward the development of higher speed (OC48 or OC192) monitors or specialized monitors, e.g. LIGHTmon
H. KEY PERSONNEL
kc claffy - (10% commitment) is a visiting research scientist with the University of California, San Diego (UCSD). She received her Ph.D. in Computer Science and Engineering, University of California, San Diego in 1994. Currently she is principal investigator for the Cooperative Association for Internet Data Analysis (CAIDA) effort - NSF support extends from 1997-00; co-PI for the National Laboratory for Applied Network Research (NLANR) - 1995-98, follow-on expected; PI for the Information Provisioning project (1996-98); PI for the Internet Engineering Curriculum Repository (IECR) project 1997-00; PI for the Caching Hierarchy project (1997-00).
Dr. claffy's research is focused on elements of the Internet infrastructure, particularly development tools for acquiring and analyzing data of use in planning and engineering wide area networks and on improving Internet performance through implementation of solutions such as caching. The Coral tools developed by MCI are, in part, an outgrowth of research by k claffy and Hans-Werner Braun (UCSD/SDSC) in the early 1990s. Recent efforts have included several projects to develop tools visually depicting Internet topologies and traffic flows. Illustrative papers are included at the end of this section. Key websites describing research the goals and products of these research efforts include: www.nlanr.net, www.caida.org, and iec.caida.org.
Joel Apisdorf (100% commitment) created a realtime flows-based monitoring device called "OCxMON/Coral" for 155 megabit per second (Mbps) and 622 Mbps full-duplex links with up to 16 millions simultaneous VPI/VCI's, consisting of an optical splitter, a Pentium PC and custom SONET network interface cards (NICs) for PCI bus. He wrote C++ and assembly language device drivers, TCP/IP stack, and statistical analysis software resident on the PC, as well as the ATM AAL5 segmentation and reassembly (SAR) and PPP HDLC engines embedded in the cards. He also wrote scripts for data collection and PERL CGI programs to deliver the data as web pages.
Prior to joining MCI's vBNS team, Apisdorf was a consultant with Cable and Wireless where he created a piece of telecom inband/outband load test equipment, consisting of a SUN workstation running UNIX which communicated via TCP/IP with an intelligent peripheral (switch) for controlling 128 T1/E1 telecom links. Wrote all software resident on the SUN, including C++ base classes for timers, bit/byte conversion/alignment, communications, and user interface. Modified the ROM-based tasks on various cards of the switch. Gathered requirements, planned phased releases, wrote documentation, and trained end users. A key website describing related research is www.vbns.net.
Dr. Andrew Gross (50% commitment) is an Information Security Researcher at the San Diego Supercomputer Center (SDSC). Gross has worked at SDSC for six years, specializing in networking and security issues. Gross currently heads the Pacific Institute for Computer Security (PICS) project at SDSC which focuses on post detection computer security. Gross has extensive experience in the design and implementation of dynamic and static network monitors, traffic analysis, and countermeasures for serial, Ethernet, FDDI, and ATM environments. For the last five years, he has worked with Tsutomu Shimomura in post-mortem analysis of intrusions and has developed countermeasures and software security tools.
Gross holds a Ph.D in Computer Security from the Electrical and Computer Engineering department at the University of California at San Diego (UCSD). Gross also holds an M.S. in Electrical Engineering from UCSD and a B.S.E. in Electrical Engineering from Duke University.
Glenn T. Sager (50% commitment) is a PhD candidate (University of Illinois) and security researcher at SDSC's PICS. His research focus includes monitoring and analysis of network data traffic, intrusion response, and host triage. In collaboration with PICS researcher Andrew Gross, Sager has developed and publicly disseminated several tools to perform security analysis of IP traffic on FDDI and Ethernet networks (see ftp://ftp.sdsc.edu/pub/sdsc/security/PICS). The tools included: (1) real time packet capture tools with autonomously modifiable filters, (2) a network security policy compliance monitor, and (3) a traffic data reduction and aggregation tool. In related work, concepts and software were developed for the reconstruction of remote NFS filesystems from passively collected IP datagrams.
As part of PICS on-going field research, Sager has principal responsibility for network level monitoring and intrusion detection at SDSC. As part of Unix host level security integration, Sager collaborated with PICS researchers in designing and developing a centralized host logging system. The system coordinated the secure collection of log data from a heterogeneous collection of several hundred hosts (>100 MB/day), and performed data reduction and time intrusion analysis in near-real time. Sager also collaborated in the Distributed Object Computational Testbed (http://www.sdsc.edu/DOCT) project where he was responsible for development and deployment of the host-level security architecture and monitors. Sager is expected to continue on DOCT efforts at 50% in 1998.
Rick Wilder (5% - cost-shared by MCI) Rick Wilder is Senior Manager for Internet technology at MCI and manages the engineering team responsible for the vBNS network. He has 17 years experience in the design and implementation of commercial and government communications systems. His primary expertise is in the areas of protocol analysis, network performance measurement, and internet traffic analysis. Demonstrated skills in project management and technical team leadership. Prior to joining MCI, he was a senior engineer for Advanced Network and Services (1992-94) and researcher for Mitre Corp. (1987-92). Key areas of research included simulations, and measurement studies of TCP/IP traffic transmitted over wide-area ATM networks; analysis of router and ATM switch requirements to support high-speed IP traffic in an ATM network; and congestion avoidance and control strategies for network and transport layer protocols in an internetwork environment.
Relevant Publications:
Thompson, Miller, Wilder, "Wide-Area Internet Traffic Patterns and Characteristics", IEEE Network, Nov/Dec 1997.
k claffy and Tracie Monk, "What's Next for Internet Data Analysis? Status and Challenges Facing the Community" IEEE journal, October 1997.
Tracie Monk and k claffy, "Internet Data Acquisition and Analysis: Status and Next Steps," proceedings of INet '97, June 1997.
K. Thompson, J. Apisdorf, R. Wilder, k claffy, "OC3MON: Flexible, Affordable, High-Performance Statistics" proceedings of INet '97, June 1997.
k claffy, "The NGI's Role in Facilitating Internet Data Acquisition and Analysis", white paper presented at NGI Workshop, May 1997. Claffy also chaired Internet Traffic Engineering working group and edited their final report.
k claffy, "but some data is worse than others: measurement of the global Internet", Telegeography, 1996.
Tamara Munzner, Eric Hoffman, k claffy, and Bill Fenner "Visualizing the global topology of the mbone," Infoviz 96. November 1996.
Joel Apisdorf, k claffy, Kevin Thompson, and Rick Wilder, "oc3mon: flexible, affordable, high performance statistics collection", LISA (Usenix conference), October 1996.
k claffy and Tracie Monk, "Cooperation in Internet data acquisition and analysis," Harvard workshop on Coordination and Administration of the Internet, (proceeding pub. June '97), September 1996.
Jamison, Wilder, "vBNS: The Internet Fast Lane for Research and Education", IEEE Communications, Jan. 1996.
K. Claffy, G. Polyzos and H.-W. Braun, "A parametrizable methodology for Internet traffic flow profiling", Mar 1995, IEEE JSAC Special Issue on the Global Internet.
H.-W. Braun and K. Claffy, "Web traffic characterization: an assessment of the impact of caching documents from NCSA's web server", Second International World Wide Web (WWW) Conference '94 proceedings, October 15-17 1994, Chicago, IL
K. Claffy and H.-W. Braun and G. Polyzos, "Tracking Long-term Growth of the NSFNET", March 1994, Communications of the ACM, 1994.
I. FACILITIES/RESOURCES
1. SDSC - Currently, SDSC's production systems are a CRAY C90 vector supercomputer, CRAY T3E parallel supercomputer, an Advanced Visualization Laboratory, and an High Performance Storage System for archival storage . These resources are available for use by academic researchers and students and are available under special cost-sharing arrangements to commercial and government researchers in the U.S. and abroad. Currently, more than 5,100 researchers at more than 240 institutions are using these platforms for their research. (See http://www.sdsc.edu/resources/Resources.html) SDSC's network connections include: an OC12 connection to vBNS and DS3s to Cerfnet. SDSC also peers directly with several NGI networks (ESnet and iDREN) at SDSC and hosts other academic networks at our facilities. SDSC also hosts numerous carriers, including: ICG (OC3); MCI (1 OC48; 1 OC3); PacBell (OC3); TCG (OC3); Times Warner (OC3); and WorldCom (OC3).
In addition to the facilities/resources of SDSC, CAIDA is in the process of developing a laboratory for testing Internet hardware and software and a peering facility for use by ISPs interested in peering in the San Diego region. This IX will also be used for traffic monitoring research purposes. SDSD/PICS will also make available its network security research testbed. The Testbed is a sheltered (and isolatable), heterogeneous network of Unix and Linux hosts. The hosts are interconnected on separate ATM, FDDI, and Fast Ethernet private networks. Traffic data archival capabilities are provided by a dedicated RAID array. The Testbed provides a flexible and compact facility on which the group conducts diverse research and development activities, including: network monitor development; host and network vulnurability assessments; and development of new attack and countermeasure methodologies (wargames).
2. MCI - Development of the OC48mon, LIGHTmon, and OC192mon would take place at MCI's Reston, VA offices, as would initial testing. For more in-depth tests, MCI is offering the use of its Richardson, Texas laboratory. The laboratory (a cluster of four lab in the Dallas metroplex) are connected with Non- Dispersion Shifted Fiber (NDSF), DSF and/or DSF/Long Span fiber to the Dallas Metro ring which is connected to the MCI backbone network. Facilities are also equipped with OC-48, OC-192, Optical amplifier and the associated passive/active devices for a full-fledged network demonstration and characterization. The Lab is also equipped with the state-of-the-art test and computerized measurement systems, including: PMD measurement system, Chromatic Dispersion measurement, OTDR, Optical spectrum analyzer, and Automated BER test system. Lab floor space is dedicated for the characterization of new technologies. Current technologies under evaluation include: Dense Wavelength Division Multiplexing (WDM) with bi-directional amplifiers, Dispersion compensators, and PMD compensators.
questions should be directed to info@CAIDA.org
Cooperative Association for Internet Data Analysis (CAIDA)