Passive ISMA 9901 survey question answers


  1. Biosketch - including name, affiliation, title and description of activities/accomplishments (relating to Internet statistics and metrics analysis)

  2. Description of current activities in the field of passive monitoring

  3. Description of future plans in the field of passive monitoring

  4. Identification of tools currently used, including comments on their strengths and/or weaknesses [if any are available for demo'ing or a short presentation at the ISMA, please describe the tool and space/equipment/network requirements]

  5. Identification of what you see as the most critical requirements, challenges, or new directions for this field

  6. Identification of relevant URLs covering tools or initiatives in this sector


Respondents:

Answers:


Abha Ahuja

  1. abha ahuja
    Merit Network
    Internet Engineer

    i have been working with the RSng and IPMA projects at Merit. Activities include routing policy and routing traffic analysis.

  2. current activities - working on getting traffic analysis for Michnet for both production and research efforts.

  3. future plans - doing more work with OC3mon and related tools.

  4. tools currently used include mrtg, cflowd, oc3mon, ipma.

  5. challenges - ensuring that the data analysis tool can handle high speeds (OC-12 and above). achieving adequate organization of data and types of information collected.

  6. http://www.merit.edu/ipma
    http://www.rsng.net/pair
    http://www.merit.edu/ipma/docs/sigcomm98.ps

Paul Barford

  1. Paul Barford, Graduate Student, Boston University. Member of the OCEANS and Commonwealth research groups at BU and a member of the W3C's Web Characterization Working Group.
  2. The principle passive monitoring activity here at BU is the collection of Web client access traces through the use of non-caching HTTP proxy software which records all requests made by uninstrumented Netscape browsers installed in our undergraduate computer lab.
  3. There are many of plan but nothing concrete at this time other than the continuation of the Web client monitoring.
  4. We use home-grown HTTP proxy software for gathering Web client traces. We frequently use tcpdump. We use a wide variety of statistical analysis software both commercial and home-gown.
  5. Many of the standard statistical analysis methods break down when used with large data sets such as those associated with computer networks. I am most interested in work related to the development of techniques and methods for analyzing large data sets.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    http://www.cs.bu.edu/groups/oceans/
    http://www.cs.bu.edu/groups/cwealth/
    http://www.w3.org/WCA/

Kathy Benninger

  1. Kathy Benninger
    Pittsburgh Supercomputing Center
    Systems Engineer

    My activities in this area are only very recent so please see my answer to 2. below.
  2. Integration of OC3mon to monitor PSC's production network.
  3. Work with production and research networking staff to analyze and understand the implications of data collected from PSC's OC3mon.
  4. OC3mon
  5. From my admittedly brief exposure to working with monitoring via OC3mon, one of the critical issues seems to be deciding what data to look at. Perhaps underlying that issue is identifying precisely what it is that we hope to find out from the data.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

Nevil Brownlee

  1. Dr Nevil Brownlee Director, Technology Development, ITSS, The University of Auckland Co-chair RTFM (Realtime Traffic Flow Measurement)Working Group of the IETF (Internet Engineering Task Force). Author or co-author of 5 Internet Drafts and3 RFCs (2063, 2064 and 2123) on traffic measurement.

  2. Ongoing maintenance and distribtuion of NeTraMet, a free-source implementation ofthe RTFM traffic measurement system. Investigation of new possibilities for traffic measurement, i.e. development of new sttributes within the RTFM model.

  3. Description of future plans in the field of passive monitoring
    Develop TCP-based attributes for flow measurement. Investigate use of RTFM meters as a platform for collecting data from active measurements, e.g. information about 'probe' packets. Investigate use of TCP to carry a stream of compressed SNMP data as a way of retrieving data from RTFM meters. Implement NeTraMet on new hardware platforms as these become available. Develop methods of associating authenticated user ID with particular traffic flows in conjunction with the IETF's AAA Working Group.

  4. Identification of tools currently used, including comments on their strengths and/or weaknesses [if any are available for demo'ing or a short presentation at the ISMA, please describe the tool and space/equipment/network requirements]
    NeTraMet. See above and/or
    http://www.auckland.ac.nz/net/Internet/rtfm
    http://www.auckland.ac.nz/net/NeTraMet

  5. Identification of what you see as the most critical requirements, challenges, or new directions for this field:

    Challenges:

    User (subscriber) based flow analysis
    Deciding on useful metrics for various kinds of user
    (backbone provider, ISP, campus provider, end-user)

    New directions:

    Effect on traffic measurement of the rising use of encryption. This is clearly something which will affect different users in different ways - for example it could lead to a rise of interest in measurement at the network edges, and a change of emphasis on the backbones (more interest in 'where's the traffic flowing' and less in 'what kind of traffic is it?')

  6. Identification of relevant URLs covering tools or initiatives in this sector:
    http://www.auckland.ac.nz/net/Internet/rtfm
    http://www.auckland.ac.nz/net/NeTraMet

Keith Burden

  1. Name: Keith Burden
    Affiliation: MCIWorldcom
    Title: Design Engineer
    Activities: Enhanced functionality of OC12mon

  2. Enhancing functionality of DOS OC12mon.

  3. Continue work on OC12mon and participate in design of an OC48mon.

  4. OC3MON pro: inexpensive ; con: hard to use, medium reliability
    OC12MON ibid. ; ibid.

  5. Economical sniffing hardware for SONET.
    Analysis software that's easy to use.

  6. CAIDA
    http://www.caida.org
    CAIDA NGI
    http://www.caida.org/funding/ngi1998/content/reports/quarterly_1098.xml
    MCI
    http://www.vbns.net/
    NLANR
    http://www.nlanr.net/

Ramon Caceres

  1. Ramon Caceres, Principal Research Staff Member, AT&T Labs

    As a graduate student at UC Berkeley, I was among the first to gather wide-area Internet packet traces (1989-1991) and use them to characterize TCP/IP traffic flows at the application level (SIGCOMM '91 and J. of Internetworking papers with Danzig, Jamin, et al.). This work led, for example, to the tcplib traffic generation tool from USC.
  2. I am part of a group that over the last 2 years has deployed a performance measurement infrastructure within WorldNet, AT&T's ISP business. This infrastructure has both passive and active components. Passive components include: two custom-built packet sniffers we call PacketScopes; an otherwise idle BGP router that we periodically query for routing tables and updates; the systematic collection of Cisco NetFlow statistics from border routers; and the usual gathering of SNMP statistics from network elements. Active components include a mesh of measurement machines located at major router centers. These machines exchange measurement traffic and collect connectivity, delay, and throughput statistics throughout the day.

    We gather the results of all these passive and active measurements in one place, a large data repository we call the WorldNet Data Warehouse. The Warehouse continuously receives feeds from these data sources and stores them in a high-performance database system developed at AT&T Labs.
  3. We are in the process of installing a third PacketScope to capture all the traffic to and from the server farm run by Easy World Wide Web (EW3), AT&T's Web hosting business. Together with EW3 server logs and our other PacketScopes, this will enable Web traffic characterization at an unprecedented detail.
  4. Our PacketScopes are 500-MHz Alpha workstations with 10-GB striped disk arrays and 140-GB tape stackers. They run Unix and a modified version of tcpdump. We have taken numerous precautions to preserve the security of the network and to protect the privacy of the captured data. For example, we encrypt IP addresses as soon as they come off the wire and before they go to stable storage, thus rendering our packet traces anonymous.

    A major strength of our PacketScopes is that they can gather traces continuously and indefinitely as long as someone refreshes the tape magazine when the tapes fill. We have gathered all HTTP/TCP/IP headers on a 100-Mbps FDDI ring for weeks at a time with less than 0.3% packet loss. A weakness of our PacketScopes is that, at least in their current form, they require manual intervention to change tapes after every two weeks or so of active tracing.
  5. I see two difficult challenges ahead: dealing with end-to-end IPSEC and operating at OC12 and higher links speeds. Regarding IPSEC, some flavors of it will encrypt the complete IP payload, including higher-layer protocol headers such as TCP and HTTP. Regarding high-speed links, the time seems fast approaching when we won't be able to capture more than a few minutes' worth of traffic with off-the-shelf hardware. Unless we can come up with solutions to these two problems, they will prevent much of the traffic analysis we find so useful today.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    http://www.research.att.com/~ramon

Kavitha Chandra

  1. Kavitha Chandra
    Assistant Professor, Department of Electrical & Computer Engineering Center for Advanced Computation & Telecommunications
    University of Massachusetts Lowell

    Our research on Internet traffic analysis and modeling addresses the problem of identifying traffic metrics that can be used in the construction of a state table that characterizes the traffic flow state at a given node, router or switch. We propose a taxonomy for separating the aggregate traffic at a node into component flows based on a feature vector that includes the protocol and port identifiers and packet sizes. Under such a classification each flow is attributed a spectral or correlation measure and mapped using nearest-distance metrics to one of a library of traffic patterns in an existing database. Associated with each such pattern is a set of performance metrics providing an estimate of the impact of this particular flow on the average packet loss or delay seen on the link. The state table therefore allows one to select at any given time a subset of the packets for traffic shaping to control the loss or delay characteristics of the node. The time-varying features in the state table will offer a better understanding of the characteristic time-scales in the traffic and also identify the applications or protocols that are the source of these time-scales. The aforementioned techniques are currently being applied on the measurement traces being made available through the NLANR/MOAT traffic measurement archive. A second research focus is on the design of an integrated wire-line and wireless passive monitoring network. The goal is to use wireless network technology to transmit traffic measurements collected from passive monitoring activities on the wide area backbone to a central site and to develop techniques for generating a relational state table across all of the backbone nodes.

    This research is funded by a NSF CAREER grant # NCR-9734585.
  2. Description of current activities in the field of passive monitoring Initial passive monitoring activities have been conducted on segments of the UML campus network. The tcpdump software utility has been the primary tool used for monitoring. This work has primarily focussed on identification of flow patterns and their performance attributes that may be used to create a traffic signal database.
  3. Description of future plans in the field of passive monitoring The UML campus backbone consists of an OC-3 backbone interconnecting three Cisco-5500 switches. We are currently in the process of setting up the OC3MON utility on this backbone.
  4. Visualization tools using AVS , presently under development.
  5. As traffic and link speeds increase, passive monitoring activities will be required to selectively monitor and store only a subset of the packets crossing a network point. One of the challenges will be to put in place network monitors that can predict the traffic features from the knowledge of the sampled packets and dynamically update traffic parameters and state tables. A second challenge for passive monitoring activities is to determine techniques for cross-classification across a set of nodes on an end-to-end path on the Internet. This will require the establishment of a centralized server that integrates passive measurements from various monitoring sites and yields end-to-end performance estimates for any given path.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    http://morse.uml.edu/~kchandra/career.html

k claffy

  1. k claffy
    caida pi
    started caida
  2. helping with caida projects: coral, cflowd.
    PI on NGI darpa funded project that includes oc48mon
  3. leading caida in
    continuing w above
  4. coral
    tcpdump
    cflowd
  5. make coral more usable common interface between coral and cflowd

    deploy monitors in commercial infrastructure as much as possible
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    www.caida.org

Stephen Donnelly

  1. Stephen Donnelly, postgrad student at The University of Waikato, Hamilton, New Zealand. Waikato Advanced Network Dynamics group, Dept of Computer Science.

  2. Writing drivers, firmware monitor, measurement applications for Dag series cards, our custom hardware for passive measurement of traffic. Currently ATM DS3, and OC3. OC12 ATM/PoS, as well as 10/100 ethernet is very nearly ready, and we are looking at OC48.

  3. Collecting some useful traces! Writing OCXmon to run on Dag hardware is something that needs to happen.

  4. See above.

  5. Useful data reduction methods, simply can't go on recording full traces at v. high speeds. Need to identify metrics/stats to collect.

  6. http://wand.cs.waikato.ac.nz/
    http://www.cs.waikato.ac.nz/Pub/Html/ATMDag/dag.html

Phil Emer

  1. Phil Emer Associate Director Advanced Technologoy Development, NC State University Senior Technical Staff, North Carolina Networking Initiative (NCNI)
  2. The NCNI administers the NC Gigapop - by far, the most well instrumented GigaPop in the country. We monitor links at each of the nodes in the distributed gigapop, and several commodity Internet links as well. Our instrumentation includes 10 OC3monitors including several FreeBSD types.
  3. Continue to monitor at all major campus and regional network aggregation points. Stress analysis of the data especially for use in network planning and traffic engineering.
  4. We use OTS tools including OC3mon, TCPdump, and some home grown snoop tools. No demo. All of the tools provide ample data points - though the data doesn't analyze itself:)
  5. Analysis. We have focused on the ability to capture at very high rates - we can now. We now need to decide on statistically relevant trace methods that limit the amount of data captured. Further, we need to build parsing tools that will help us answer network planning questions.
  6. Identification of relevant URLs covering tools or initiatives in this sector:
    http://www.ncni.net

Shobha Erramilli

  1. Shobha Erramilli, Research Scientist in the Computer Communications Research group in Bellcore. Involved in network level performance and QoS measurement in IP and ATM networks.

  2. Co-developed Bellcore's Differentiated Services testbed and tools for measuring Internet service quality measures such as IP throughput, packet loss ratio and one-way packet delay. The testbed consists of passive monitors and analysis software that capture IP packets along a given path and compute the various performance metrics on a per-class basis. A Web-based controller is used to schedule the passive monitors, the traffic streams and the subsequent analysis.

  3. Extending the previously described tool to analyze the service quality metrics on a per-flow basis. Also, plan to integrate the passive monitoring tool into a network traffic management system.

  4. tcpdump for collecting packets, NTP for synchronizing monitor clocks and netperf for generating traffic on the testbed.

  5. Metrics that are easily measureable, give an accurate picture of both network conditions and user-perceived performance. Lighweight monitors , either standalone or integrated with network elements, need to be deployed in the network so that they can be integrated with the congestion management and capacity planning functions

  6. (no URLs given)

Wenjia Fang

  1. Wenjia Fang, Princeton University, Graduate Student, doing network architectual research using Internet statistics.
  2. Exploring Inter-AS traffic spatial locality, and assessing reservations along AS dimensions as a mechanism to support unicast real-time traffic.
  3. Need to use a lot of traces to assess Inter-AS, Inter-network spatial locality.
  4. fs2flows.
  5. Get more OCMONs deployed at more points of the network, so we can a wide range of sample traces. I am particularly concerned that not many places are willing to release their traces without sanitizing them.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    moat.nlanr.net
    www.vbns.net
    www.merit.edu
    www.caida.org

Steve Feldman

  1. Steve Feldman, MCI WorldCom, Network Architect Responsible for overall architecture of the MAE exchange points, including instrumentation for traffic measurement and analysis
  2. Current activities
    Utilizing vendor-provided switch instrumentation to monitor utilization, errors, and the like hosting oc12mon for CAIDA development and research activities
  3. Future plans
    Work with CAIDA to enhance Coral/oc12mon for ATM and MAE-specific measurements, such as atm-level monitoring, switch behavior, ...
    Traffic characterization at packet and cell levels withing the MAEs to assist in future MAE designs, vendor selection, and feedback to switch vendors for use in their designs
    Use of passive measurements as NOC tools
  4. Tools

    SNMP stuff: scotty, mrtg, etc. to pull data from switch MIBs depends on quality of switch MIB implementation
    Cisco/Stratacom accounting files: for per-vc statistics lots of available info, requires Strataview NMS or custom software to decode and archive
    Coral oc12mon: MAE traffic characterization just starting to explore this
    Ping: crude loss measurements depends on router handling of ICMP messages
  5. Critical Requirements

    Common standards for data collection and archive formats
    Convincing network hardware vendors to integrate reliable instrumentation and data collection of useful information
    Availability to researchers of data collected from "real" network environments.
  6. URLs

    My stuff (such as it is) is at
    http://www.mae.net

Anja Feldmann

  1. Anja Feldmann
    AT&T Labs-Research

    The importance of the Internet is rapidly growing. In the Internet the intelligence is in the end-systems. Because of the diversity of these systems, their applications, and the communication protocols the Internet is a beast. Existing analysis is hardly scratching the surface in providing abstractions and understanding that are taken for granted in other contexts. This includes understanding how user behave, how traffic matrices change, and how to provide performance guarantees. Over the last years I have been involved in building and maintaining an measurement infrastructure for collecting data about an ISP. Most of my research has focused on the science of understanding, describing, and modeling this data.
  2. Together with A. Gilbert, and W. Willinger, I am investigating how the state of the network is influencing the spacing of packets and thus leaves a signature on small time scale network measurements. We have empirically established the presence of complex scaling phenomena (i.e., multifractal scaling) in data traffic over small time scales and identified its main cause -- networking mechanisms such as TCP and end-to-end congestion controls. A mathematical framework based on wavelets, cascades, and multifractals builds the foundation for analyzing and describing the observed small-time scaling properties of the measured traffic. We have demonstrate that the new mathematical framework is capable of detecting and identifying local irregularities in a given set of network measurements.

    Regarding the interactions with applications I together with G. Glass, R. Caceres, F. Douglis, M. Rabinovich are using real HTTP traces to evaluate the performance of WWW caching for modem users. The results clearlz demonstrate that the interactions of the network with the application behavior matter.
  3. Understanding which timescale is relevant for which aspect of traffic engineering in the Internet.
  4. Tools: tcpdump, (httptrace extension to tcpdump that can extract full HTTP headers), Perl, Splus, Wavelets, NS, lots of disk space:-)
  5. See above. Understanding the beast of an Internet.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

Vince Fuller

  1. Vince Fuller, Senior Network Architect at GTE Internetworking. One of the members of the GTE-I group which designed and built the current GTE-I backbone and are in the process of building its next-generation replacement. Involvement with Internet stats and metric analysis has been the development of some simple tools for monitoring and charting link utilization on the network and in specifying traffic flow information needed from tools developed by other groups (mainly based on cflowd data collection).

  2. Working with other groups in GTE-I to use information derived from cflowd analysis to do quantify data transport costs and do capacity planning.

  3. Continuation of above.

  4. Am a consumer of reports generated by tools that analyze cflowd data.

  5. It is essential that the costs involved transporting data be accurately quantified and recovered. If this problem is not solved, it will not be possible to continue to scale the Internet as providers cannot indefinitely continue to build expensive infrastructure without the cost of doing so being covered.

  6. (no URLs supplied)

Ian Graham

  1. Prof Ian Graham, Dean, School of Computing and Mathematical Sciences, University of Waikato.

    Hardware/firmware system for passive measurement, use of GPS timing for transit time measurement over international ATM and IP networks.

  2. Development of the Dag series of boards for measurement of ATM, PoS and Ethernet traffic.

  3. Pushing the data rates higher. Looking at more sophisticated methods of data filtering/compression.

  4. Will be talking about Dags and demonstrating Dag2 OC3 version at ISMA.

  5. Clever ways of dealing with extremely high data volumes

  6. http://wand.cs.waikato.ac.nz/

Hervé Guy

  1. Name: Hervé Guy,

    Affiliation: Canarie inc.,

    Title: Applications Engineer,

    Description of activities/accomplishments (relating to Internet statistics and metrics analysis):
    My mandate at the ARDNOC (Advanced Research and Development Network Operations Center for CA*net II/3) is 1) to perform statistical analysis of the output of deployed measurement tools on CAnet II/3; 2) To design, modify, or implement existing measurement result visual presentation formats such that the analysis of the data is intuitive; 3) Design and maintain a database for network statistics and performance measurements as required; and 4) Design the ARDNOC web site interfaces for real time statistics and performance measurement queries for various measurement tools.
  2. In 1998, 2 oc3mon machines have been installed on the CA*net II advanced Networks: one at the ARNOC GigaPop and the second at the Toronto GigaPop. These machines are now running and collecting a huge amount of traffic data, which are stored on ASCII formatted files.
    The purpose of my first project is to take those stored CA*net II traffic data, analyze them and publish the result via an easy-to-use graphical interface on the ARDNOC Web site.
  3. After the "Oc3mon on the Web" project, other passive (such as cflowd...) and active (such as the ping and traceroute family tools...) measurement tools will be implemented on CA*net II/3 Advanced Networks. Once again, my mandate will be to analyze data stored, and publish the results on the Web.
  4. In 1998, the Surveyor project has been realized on CA*net II/3.

    As far as the "oc3mon on the Web" project is concerned, visual data traffic results should be available on our Web site (http://www.canet2.net/) late in January 1999.
  5. Via an automated process, to be able to extract the most useful traffic data information from the huge amount of data generated (and stored) by oc3mon.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    http://www.canet2.net/
    http://www.caida.org/tools/taxonomy/
    http://www.advanced.org/ippm.html
    http://www.caida.org/
    http://www.internet2.edu/html/measurement.html
    http://www.ietf.org/html.charters/ippm-charter.html
    http://laguerre.psc.edu/networking/nimi/welcome.html
    http://moat.nlanr.net/
    http://www.vbns.net/nettraff.html

Sig Handelman

  1. Sig Handelman, Research Staff Member and Manager, IBM TJ Watson Research Center
    Co-Chair of IETF RTFM WG.
    Stephen Stibler has worked with me on implementation of the IBM RTFM Meter. Co-author of IETF Draft on New-Attributes for RTFM. We have published "Internet and Intranet Service Analysis" at the 1st International Conference on Telecom and E-Commerce.

    24 years ago was Research Associate of Benoit Mandelbrot.

  2. Developing RTFM meter and analysis tools, for different environments.

  3. Would like to see how fast we can make RTFM run.

  4. RTFM on AIX and SNMP Mib to Database Conversion tool.

  5. Passive measurement at high speeds, Database collection of large number of meters.

  6. RTFM is described at www.ietf.org in the Working Group pages.

Stan Hanks

  1. Mr. Hanks has been at the forefront of change in internetworking for nearly twenty years, starting with experimental work in distributed computing during the early days of TCP/IP on the ARPAnet. He is responsible for the invention of the GRE tunneling protocol and deployment of the first carrier IP VPN in 1991, development of a metropolitian area Layer Two facility that grew into MAE East, and has been involved in many NSP and ISP start-up efforts. Mr. Hanks graduated from Rice University with degrees in computer science and electrical engineering, which were followed by five additional years there in the Ph.D. program in computer science. He has been active in many user organizations, standards bodies, and task forces. He is currently active in the Optical Internetwork Forum, USENIX, and CAIDA.

  2. Investigating passive monitoring in very high speed networks, and implications of TAG/MPLS on the requirements for passive monitoring software. Working to develop model for flow modeling of streamed media and other non-HTTP Internet applications.

  3. Traffic engineering in the Enron Communications network will largely become dependent on data collected from passive monitoring. Making sure that we have the right tools to do the job is one of my critical path items.

  4. N/A at this point. I'd be happy to talk about what we have a little further down the line.

  5. Passive monitoring at ethernet speeds when you have 400 MHz processors isn't much of a challenge; passive monitoring with anything like real-time analysis for 10 Gb/s networks, particularly for substreams that take a significant fraction of that bandwidth, is going to take significant hardware assist.

    The ability to tie data from passive monitoring into SLA policing will also be an interesting challenge.

  6. N/A

Ted Hardie

  1. Ted Hardie, Director of Engineering, Equinix.

    Ted is currently responsible for the group which handles network management, benchmarking, and research for Equinix. As an Internet Facilities provider, Equinix is interested in understanding the general characteristics of network flows through its facilities and specifically interested in how to enable effective public and private peering.

    Prior to joining Equinix in November of 1998, Ted worked on the NASA Science Internet project. While at NASA, he managed development and deployment of all services offered by the NASA NIC and led several efforts to analyze application layer network traffic over NASA networks. He also served for two years as the elected chair of NASA's webmasters' working group, coordinated adoption of web technologies and protocols. Ted began working with the Internet as a member of the technical staff of the SRI Network Information Systems Center.

  2. Equinix is in the early stages of development of passive monitoring service offerings for its facilities. Current work concentrates on developing systems of sufficient size to store and analyze data which can be derived from OC-12 and OC-48 network flows.

  3. As a neutral party, Equinix does not plan to require that those present at its facilities use any specific monitoring technology for peering sessions with other participants. Equinix does, however, plan to provide the tools needed to monitor both public and private peering sessions and to encourage its customers to use those technologies and share the data thereby derived. Equinix will monitor the use of network services it provides at its exchanges and share that data with the customers using those services.

  4. Equinix is currently investigating both tools based on heuristic sampling of data flows and on processing real-time capture of flows.

  5. The technical challenges in this area are actually fairly standard scalability issues: getting fast backplanes, linking in fast storage devices, writing code for speedy processors. The analytical challenges are far from standard, however, as there isn't a lot of agreement on what we're looking for in all this data. For any dataset of this size, it's very easy to tune out the interesting information accidentally. Our challenge is to pose the right questions as we go through the analysis. What characteristics of the flow are of interest: number of packets, packet sizes, source network/AS, destination network/AS, unicast v. multicast, type of data, type of service, quality of service indicators, route appropriateness, presence of vpn or tunnel, and any of a raft of other possibilities? Before we can say which characteristics we should look at, we need to know what question we're answering. Do we want to know how well routing arbitration works? Do we want to know whether multicast delivery of data type X results in lower overall packet traffic over a link? Or do we want to know simply how much of the traffic flow is a retry, an ACK or a SYN?

  6. No relevant URLs are available at this time.

Rene Hatem

  1. René Hatem is a network engineer, responsable for leading the Statistics and Measurement effort at the CA*net II/3 NOC (ARDNOC). Several passive as well as active measurement tools are being used or are planned to be used.
  2. Recent and current activities include the installation of two DOS-based OC3Mon machines, one at the heart of C2, and the other in the ARDNOC. MRTG and Surveyor box (active) also used. Major activity is monitoring and analysis of tool data.
  3. Several (passive) measurement projects are in the planning stages at for C2 and C3 including:

    - display of results of the OC3Mon (area which has been unfortunately neglected - Herve Guy is a new member of the ARDNOC team which will specifically be looking at this)

    - investigate feasibility and undertake modification of OCxMon code for greater flexibility in prescribing what statistics are collected

    - deployment of OCxMons for each C3 node to GigaPOP link

    - cflowd on C2 (1st quarter), in C3 (by year's end if supported on GSR)
  4. OCxMon: +: In theory can compile stats on any fields of a AAL5 and layer 3 or 4 header. Would like to see a capability for IP over SONET and IP over Gigabit Ethernet.
    -: the stats collected are not customizable (e.g. it should be able to do everything cflowd can collect without the overhead of running netflow). Not the easiest thing to play with.
  5. - More analysis needed. It is nice to measure, if you aren't analyzing the measurements you might as well not measure! (Actually this is a message to myself!)

    - As Bw becomes increasingly available, it will no longer be possible for any cpu to keep up with traffic over any sustained period of time. Sampling technics will be the only way. Do we know what the best sampling technics for fractal traffic and are tools being made where by default proper sampling technics are used. Does usage based billing have a future in such an environment?

    - Using tools such as OCxMon to aid in the identification of network attacks is extremely interesting. I heard this at ISMA 9809 and would like to know more.

    - We need to look into how we can learn more (create synergies) by use of different measurement tools (not restricted to passive).
  6. relevant URLs:

    http://www.canet2.net/Projects/stats/index.html
    http://www.canet2.net/NetTraffic/MRTGC2.htm

Andrew Heybey

  1. I worked on Bellcore's T1 and ATM network traffic recorders. Mark Sullivan and I built the Tribeca network traffic analysis system to be used with those monitors (Sullivan & Heybey, "Tribeca: A System for Managing Large Databases of Network Traffic". Proceedings of the 1998 USENIX Technical Conference).
    At Niksun, I am building passive network monitors as described below.
  2. We have built passive monitors for Fast Ethernets, FDDI, Frame Relay over T1, T3. An ATM OC-3 monitor is currently under development. The monitors act as data recorders as well and record all the data coming in to disk or to fast tape drives. We also have a suite of software tools for filtering, demultiplexing on various attributes such as IP header fields, link level headers etc. Users interface with the system either through a Java interface or programatically via a query language API. Users can define and compile their own queries. This makes the system easily extendable and user customizable and we view this as a key differentiator. The query language is specialized for doing network traffic processing. An installation of the system typically consists of multiple (time) synchronized recorder/monitors. This is allows for one way delay and packet loss computations. Multiple application level performance statistics are collected and correlated to lower level traffic statistics. Relational database access to some higher level aggregates (tcp flows for example) is provided for easy integration into existing corporate and data warehousing/billing environments. Results of user defined queries can be integrated into the relational database system as well.
  3. An ATM OC-12 monitor/recorder is currently in the planning stage. We are also working with other vendors in the area of network planning and optimization in order to provide integrated solutions to network design and operations problems.
  4. We can provide a demonstration of the monitor in action as well as examples of the system could be used (how queries can be written and executed etc.). In terms of any demo requirements, all that we need is a VGA projector.
  5. Defining and benchmarking metrics that directly relate to user-perceived service quality will be a major challenge, if network monitoring is to have significant relevance to the end user. Such metrics could be defined and levels mapped into diffserv codepoints. In terms of challenges, the impact of ipsec on network monitoring will have to be carefully assessed.

Ken Keys

  1. Ken Keys, CAIDA, engineer on CoralReef project

  2. Development and packaging of Coral passive analysis software tools

  3.  

  4. ocXmon hardware for capturing raw packet headers

    Coral libraries developed at CAIDA

Stas Khirman

  1. Name: Stas Khirman
    Affiliation: Narus Incorporated
    Title: System Architect

    Relevant Past Activities:

    NARUS Inc.

    System architect - responsible for platform design and implementation. VdoNet Corp. 1996-1998 Networking Group leader - multimedia streaming protocol, bandwidth control algorithm,H323,RTP,RTSP RDC Communications 1994-1996 Network Engineer - Network Management System, SNMP, NDIS and ODI adapter drivers.

  2. (for answers to 2-6 from Narus, see Robert Williams)

Bo Kim

  1. Bo Kim

    Postdoctoral research fellow, Center for Advanced Computation and Telecommunications, University of Massachusetts Lowell

  2. I am developing time-series modeling methodologies for traffic measurements obtained from passive monitors. Our goal is to determine what class of such models (linear/non-linear, etc.) may be appropriate to capture a parametric representation of traffic flows. By tracking such parameters through traffic monitors one may be able to design traffic shapers to alter the correlation characteristics (ex. whitening filters) of flows in a differentiated services Internet. I am also considering the application of the work developed in my doctoral thesis (Optimal Feedback Control of ABR in ATM Networks) to the problem of feedback control of data traffic represented by time-series models. We are interested in seeing how congestion control protocols alter the traffic correlation characteristics. In our preliminary results, we have demonstrated that TCP congestion control mechanism can cause an iid source traffic to exhibit periodic correlation features during network congestion. In the passive monitoring and modeling context, I am interested in understanding what measurements or network states other than the monitored traffic may be necessary to fully describe the traffic flow at a site.

    Using tcpdump for capturing measurements off the campus lans.

  3. Our group is setting up the OC3Mon utility on the campus backbone.

  4. Perl, statistic tools, visualization tool using AVS.

  5. Reflecting these analysis results of the network traffic measurements on the design and performance of the Internet.

Aaron Klink

  1. Aaron Klink, Developer, Network Monitoring and Performance Group, Qwest Communications

  2. We are working with Cisco engineering to improve Netflow.

  3. In the future, we will be using passive monitoring tools for usage-based billing, traffic flow analysis, and other internal tasks.

  4. We currently use Cisco Netflow. The strength of Netflow, at this point, is Cisco's desire to work with us to improve the product. We also use in-house developed tools to analyze the data collected by Netflow.

  5. We would like the ability to configure what data is provided, as opposed to the current method of collecting all data and parsing out what is not wanted.

  6. (no URLs supplied)

T. V. Kurien

  1. T. V. Kurien
    Niksun, Inc.
    kurien@niksun.com

    I have been involved in the area of data mining of network data for several years. In Bellcore, I was working on efficient classification and clustering algorithms for high dimensional data sets. These algorithms were applied in the area of fraud detection and data visualization for SS-7 traffic. At Niksun, I am working on the definition and monitoring of metrics that directly reflect user-perceived QoS as well as implementing the relational database functionality in NIKSUN's NetVCR product.

Will E. Leland

  1. Will E. Leland is the director of the Computer Communications Research Group in Bellcore's Research Area. He is also co-chair of the IETF IP Performance Metrics (IPPM) Working Group, which is working to establish accepted standard definitions of metrics for end-to-end Internet performance (such as loss, delay, connectivity, throughput, and derived metrics). While Will is best known for Internet traffic characterization, such as self-similarity in traffic, he is also involved in research on scalable discovery of network topology and behavior using end-to-end measurements (including the Felix project). Will got his Ph.D. from the University of Wisconsin in 1982, and has been with Bellcore's Research Area since 1984.
  2. Within the IPPM, defining neutral, consistent metrics for Internet service quality, and working to keep these standards in sync with other efforts in the standards community (such as T1A1.3 and ITU-T Study Group 13).
  3. Future plans in the field of passive monitoring:

    IPPM:


    The current proposed IPPM metrics are intended to be neutral in regard to active or passive measurements. In practice, they have emerged mainly from active measurement researchers. The co-chairs of IPPM are working to ensure that the metrics definitions accommodate the promise of passive measurement techniques for assessing service quality. We welcome increased input from this community.

    Felix:


    We hope to extend the discovery techniques being developed under Felix for topologies and component behaviors to exploit passive as well as the current active measurements.
  4. Tools
    Not applicable. However, I'd be happy to give a brief talk on metrics proposals in IPPM and T1A1.3, and how these emerging standards might relate to and benefit from passive measurement techniques.
  5. Deriving accurate measures of end-to-end and per-hop delay, loss, and other conventional service quality metrics from purely passive measurements. Scaling such solutions in number of paths and traffic intensity.
  6. relevant URLs:
    http://www.ietf.org/html.charters/ippm-charter.html
    http://felix.bellcore.com

Jed Martens

  1. Jed Martens - Research Assistant and PHd student. Member of Waikatos WAND group.

  2. Design and construction of monitoring equipment for high speed fiber optic networks

  3. Developing equipment for OC-12 and OC-48, and adapting current techniques to monitor packet over sonet.

  4. Currently using Dag2.11 for OC-3 measurement, devolping Dag3 for OC-12. A demo would have to be arranged through Ian (ian@cs.waikato.ac.nz)

  5. The biggest challenge in my area is keeping up with the ever increasing speed of networks

  6. http://wand.cs.waikato.ac.nz/

Matt Mathis

  1. Matt Mathis, Pittsburgh Supercomputing Center.

    Author of mping (Windowed ping), treno, RFC2018 (TCP Selective Acknowledgements), and papers on TCP bulk performance.
  2. Testing an improved TCP implementation and drafting a new Bulk Transport Capacity Metric for the IPPM working group of the IETF. More mundane activities include diagnosing and tuning a couple of vBNS based high performance applications.
  3. Develop tools to diagnose network performance by inspecting transit TCP traffic. Although not the primary focus, this tool will also provide some information about tuning individual TCP connections.
  4. The new tool will be a backend to go behind other packet collection frontends (Coral, tcpdump, argus, etc).
  5. Changing the common view of Internet traffic from "open loop statistical models" to "closed loop statistical control system models".
  6. Identification of relevant URLs:

    http://www.psc.edu/~mathis
    http://www.psc.edu/networking
    http://www.psc.edu/networking/papers
    http://www.psc.edu/networking/tcp.html

Sean McCreary

  1. Sean McCreary, CAIDA

  2. Locality analysis of backbone Internet traffic, IPv4 address space utilization patterns in the backbone

  3. Measuring self-similarity in backbone traffic

  4. OC3mon for collecting backbone packet traces, CoralReef for building analysis software

  5. The greatest challenge is the development of monitoring tools that will allow data collection at OC48 speeds and higher. Without the ability to collect data at these speeds, analysis of the characteristics of backbone traffic will become impossible.

  6. (no URLs supplied)

Tony McGregor

  1. Tony McGregor, Senior Lecturer Waikato University and Visiting Researcher NLANR. Lead researcher on NLANR's active measurement program (http://moat.nlanr.net/AMP). Member of Waikato's WAND group.
  2. The WAND group is establishing a network of OC3/12 ATM monitors focusing on the Asia Pacific region. These monitors use Waikato's DAG hardware (a custom ATM interface with special facilities for measurement including a GPS interface) and are suitable for both active and passive monitoring. Some results are available. (see http://wand.cs.waikato.ac.nz/).
  3. ATM and POS OC12 is nearly complete. OC48 is in the planing stages.
  4. Waikato's DAG. Yes we can probably do a demo but Ian would need to sort out the details.
  5. Higher speeds, especially capturing traces at these speeds. Identifying the needs of the industry and those that commercial providers don't (yet) see. Doing meaningful analysis on passive monitor data.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    http://wand.cs.waikato.ac.nz/

Daniel McRobb

  1. Daniel McRobb
    CAIDA
    Software Engineer

    CAIDA employee for less than 1 year, I was previously employed by ANS. At ANS I was a member of the network management systems (NMS) development group. I've been involved in SNMP manager development and other real-time network management development (ICMP pollers, service monitoring (HTTP, NNTP, DNS, etc.), SLA-driven management, etc.). I've also been involved in mass measurement systems (SNMP MIB-II data, RTT data, etc.) and data storage and analysis (developer/maintainer of original ARTS library at ANS, current author of the publically-available arts++ package). I'm also the author of cflowd.

  2. cflowd author

  3. Enhancements to cflowd and arts++

  4. cflowd

  5. Widespread vendor support for measuring what matters to a network service provider. Useful analysis tools for large aggregate datasets.

  6. (no further URLs supplied)

Greg Miller

  1. Greg Miller
    Senior Engineer
    Internet Technology and Architecture/ vBNS Engineering
    MCI WorldCom

    Have published traffic studies on large backbones (vBNS, internetMCI) Responsible for the active performance measurement of the vBNS

  2. Engaged in analysis of OC12MON packet traces from the vBNS backbone. Studying traffic characteristics including burstiness, correlation, self-similarity, protocol composition, etc.

  3. Continued analysis of OC12MON traces and flows data with a focus on application to traffic engineering, MPLS, etc. Expect to deploy monitors within the UUNET backbone.

  4. OCxMON, tcpdump, SNMP

  5. monitor for packet-over-SONET, OC48 monitor

  6. http://www.vbns.net

Sam Mokbel

  1. Sam Mokbel, Chief of Engineering, ONet Networking, Toronto, Canada

    ONet is the Regional Advanced Network (RAN) for the province of Ontario. We provide online real-time bandwidth utilization reports to our members.
  2. Currently we monitor traffic flows between the different ASes served by our GigaPOP in addition to bandwidth utilization on all of our ATM PVCs, TDM, and POS circuits.
  3. Our concrete plans are in the field of active measurement area as part of CANARIE-C3 measurement project which will be cooperating with the IETF IPPM-WG. We are planning on installing a surveyor node at the Toronto GigaPOP.
  4. We use MIB variables, to get counter data and plot graphs or display tables for our circuits or services such as News, AS to AS flows, Routes, Multicast, and bandwidth utilization. We can make available our GigaPOP and RAN diagrams.
  5. The explosive growth of the Internet and its related applications makes it extremely challenging to devise a comprehensive measuring approach.
  6. You may check our GigaPOP web site at:

    http://enfm.utcc.utoronto.ca/c2/

    We do not have a single page for our stats and measurements, the information is rather scattered but should all be accessible from that URL.

Tracie Monk

  1. Tracie Monk, Director, CAIDA
    Manage CAIDA program at UCSD/SDSC
  2. Responsible for external communications regarding the scope and priorities of CAIDA's passive monitoring initiatives; manage collaborations/subcontracts relating to the development and deployment of passive monitors.
  3. Passive monitoring is a key thrust area within CAIDA. CAIDA places high priority on the development of traffic analysis and visualization tools and on public outreach/education concerning the relevance of monitoring/analysis of passive data for networks, users, and hw/sw vendors. Future efforts will include correlation of passive traffic data with active measurements and routing data.
  4. None personally. CAIDA -- Coral monitors (OC3, OC12) and cflowd.
  5. Education regarding the relevance of these data/analyses to network planning and operations; development of more user-friendly, real-world analysis tools and models.
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    www.caida.org/tools/

David Moore

  1. David Moore, CAIDA, lead on CoralReef project
    • packaging and developing coral passive analysis tools, automated report generation, unix ocXmon drivers, and libcoral libraries (perl & C).
    • looking at basic network characterization metrics from passive measurements
    • examining checksum failures and attempting to detect/identify hardware failures in routers, as well as software and end host problems.
    • characterizing network game traffic

    • looking at interarrival time statistics for aggregated traffic mixes

    • working with more network managers to develop tools which can provide better measurements for solving local operational problems
    • tcpdump - for capturing and filtering packet traces on LANs

    • ocXmon - for capturing raw packet header traces, as well as capture platform for running other specific real-time continuous collection/analysis tasks

    • various analysis tools that caida has developed, in particular coral libraries in perl and C (part of coralreef package) which allow quick prototyping of analysis software from multiple trace file formats

    • looking glass route servers such as route-views.uoregon, etc, looking into using mrtd to track announcements/withdrawals directly
    • analysis of large-scale behaviour of tcp dynamics

    • dealing with IPSEC, NAT, caches

    • providing common access libraries to differing data file formats (with different types of elided or merged data) so tools needn't be rewritten for every dataset out there

    • organized data collection in a manner which allows long term trend analysis, both with sumarrized and raw trace data. lots of bookkeeping, storage problems here as well.

Teunis Ott

  1. Teunis J. Ott
    Bellcore
    Senior Scientist

    Started the ethernet measurements that led to the discovery of fractal behavior many years ago.

    Did theoretical studies of TCP (the square root formula, stationary behavior of TCP, now also transient behavior of TCP).

  2. With data from an ISP, produced model for customer behavior in NNTP. Gave a ``proof of concept'' that with sufficient knowledge of the source behaviors it is possible to do reasonably sophisticated link engineering.

    Currently working on models for Internet Games (Quake).

  3. Produce a source model for at least one interactive Internet Game. Update source models for Internet Telephony and Web Traffic. Attempt to produce a classification of source models.

    Translate source behaviors into rules for link engineering.

    As you see, I am more into analyzing the data than into taking the measurements. I want to learn more about taking the measurements!

  4. I do not consider myself an expert on tools. I will come to learn.

  5. Flexible, easy to use tools that can take ``full traces'' (including time stamps. Tools that can do this on a variety of types of links and link speeds. Analysis tools that can scrub out sensitive information yet keep enough of the data fields to allow study of tunneling behaviors.

    In both the NNTP work and the Quake work we had to work with headers only. We found it useful to develop a ``library'' of patterns in foreward and backward directions that make it possible to pretty much reconstruct what the customer did from headers only (headers client -> server AND server -> client). This seems a useful approach whenever only headers are available (i.e. much of the time!).

  6. I am not the right person for this question.

Dave Plonka

  1. Dave Plonka, University of Wisconsin - Madison. I'm a Systems Programmer in Network Engineering Technology, within our Division of Information Technology (DoIT).

    Recently I've released a perl package called "Cflow" which understands cflowd flow files. (This has been announced in "comp.dcom.sys.cisco".) Also, I've released a package called "NetTree" that I've found useful to summarize flow statistics by network or subnet.

    I have experimented with analyzing the flows produced by cflowd to identify machines on our campus which have had their security compromised. (This can sometimes be determined by observing their responses to scans and probes from the outside world.)

  2. We've done bug fixes and enhancements to cflowd (cflowd-1-3b2). These enhancements allow cflowd to be used with version 1 PDU of Cisco's NetFlow export data. Some of this was, unbeknownst to me, executed in parallel with Daniel McRobb's efforts - at the time I was unaware that the cflowd code was being maintained.

    Also, we've built and are using a flow analysis package that can identify inbound and outbound traffic from a campus or site. (In some cases, such as with multipoint ATM interfaces on Cisco routers, it is not sufficient to determine whether or not the traffic is intracampus based solely on the interface ifIndex.) Also, been working with integrating MRTG's graph generation with cflowd so that I can plot the use of services (httpd, nttp, etc.) over time. The resulting GIFs are suitable for a WWW page.

  3. I'd like to build a well-defined perl interface for, and add the continuous graphing capabilities to, passive monitoring tools other than cflowd. My interest in other flow-based systems (such as ocXmon) is so that I can gather statistics where the routers might be from other vendors.

  4. MRTG While it doesn't fit in the same class as flow-based monitoring tools MRTG is a good example of how historic data can be compressed so that it doesn't become unmanageable. cflowd we chose to implement cflowd first, over oc3mon, because we peer with external entities at multiple points (i.e. multiple routers). To analyze inbound and outbound flows to the campus would have required us to build and maintain multiple oc3mons. cflowd uses flow export data from our existing Cisco border routers. oc3mon I found the documentation to be confusing, the perl analysis scripts to be undocumented. While a couple members of our team managed to assemble a working oc3mon, it is usually not used currently.

  5. The formal packaging (to ease implementation) and quality documentation of monitoring and anlysis packages is critical to their success, IMHO.

    Developing flexible analysis tools so that users can reduce the raw data but still be able to answer all sorts of questions about traffic is difficult. It's what led me to using perl to parse the flows so that the powerful scripting language can be used to crunch the data and report upon it.

  6. (no URLs supplied)

Marc Pucci

  1. Marc F. Pucci
    marc@bellcore.com
    Bellcore
    Senior Scientist

  2. I have designed and deployed pasive network monitors to collect and analyze several terabytes of complete payloads on T1 and OC3 links. Analysis has ranged from protocol summaries to session behavior to delay and loss statistics.

  3. Increasing speeds push the limits on existing tools and monitors. I am still interested in building collectors that gather all traffic and complete payloads.

  4. 8 line T1 HDLC monitor
    1 line OC3 ATM monitor
    both designed and developed at Bellcore
    Various suites of software for prelimainary analysis of collected data.

  5. A) Gathering at higher speeds,
    B) Processing in real time based on what we learn from A).

  6. (no URLs supplied)

Ronn Ritke

  1. My name is Ronn Ritke and I am a 5th year Ph.D. student in the Computer Science Department at UCLA. I advanced to candidacy for the Ph.D. in Feb. 1998. My dissertation topic is Network Measurement and Analysis. I have measured computer network traffic at UCLA for 3 years. I just finished a paper that involved trace-driven simulations (UCLA Technical Report # 980023).

  2. Currently I am taking campus level FDDI traffic traces at UCLA and measuring some of the network traffic into and out of the the Computer Science Department at UCLA. In addition to these efforts, I am in the process of obtaining some Sonet and ATM traffic traces. Some of the traffic traces are used as input to trace-driven simulations (see UCLA Technical Report # 980023). In a cooperative effort, I have supplied raw and processed traffic traces to other researchers at UCLA.

  3. My future plans involve: a) Using network traffic traces as input for trace-driven simulation.
    b) Possible collaboration with Hans-Werner Braun in traffic trace analysis and/or trace-driven simulation.

  4. Tools - tcpdump is the main tool I have been using to get traffic traces. A number of software tools have been created to process the raw traffic traces into the desired format.
    • interp - processes the output of tcpdump and puts it into a very specific format.
    • interpcol - provides the capability to select one or more of the six fields (columns) to be written to an output file.
    • interprow - writes to output only the rows that meet the desired criteria.
    Interprow and interpcol can be used repeatedly to process data into a specific format. For example - the tcpdump output contains the following information for each packet, arrival time, source IP address and port number, destination IP address and port number and packet length. Once interp has formatted the tcpdump output, interpcol can be used to write to an output file only the arrival time and packet length columns. Interprow can be used to have only the packets with lengths greater than 0 or only the packets whose arrival time was less than some value written to an output file.

    A number of other software tools have been developed for differrent traffic trace analysis and processing.

  5. Cooperation of passive monitoring researchers with other researchers. For example I have processed Computer Science dept. level traces, FDDI campus level traces and ATM cell traces for another researcher at UCLA for use in modeling self-similar network traffic.

    Continued use of passive monitoring to monitor traffic on different networks and network types and to obtain accurate characterizations of new and existing network applications, protocols, etc.

  6. The interp, interpcol and interprow software will soon be available via the web.

Jeff Rizzo

  1. Jeff Rizzo is currently employed with Equinix, as Senior Network Engineer, and is the engineering lead for their passive monitoring project. Before joining Equinix, he was senior network engineer at PAIX, where he worked with customers to monitor their switch port usage. Prior to that, he worked at Netcom's Network Operations group where part of his duties included both analysis of Netcom's traffic exchange with other ASes and providing usage statistics to network customers.

  2. Equinix is in the early stages of development of passive monitoring service offerings for its facilities. Current work concentrates on developing systems of sufficient size to store and analyze data which can be derived from OC-12 and OC-48 network flows.

  3. As a neutral party, Equinix does not plan to require that those present at its facilities use any specific monitoring technology for peering sessions with other participants. Equinix does, however, plan to provide the tools needed to monitor both public and private peering sessions and to encourage its customers to use those technologies and share the data thereby derived. Equinix will monitor the use of network services it provides at its exchanges and share that data with the customers using those services.

  4. We are currently in the process of evaluating tools which might benefit us, including the possibility that some may need to be built in-house.

  5. One of the more challenging aspects of what we are trying to accomplish is deciding what the 'interesting' things to track actually *are*. And, of course, the analysis and display of the gathered data in a format which conveys meaning.

  6. No URLs at this time, sorry.

Glenn Sager

  1. Glenn Sager is a Staff Engineer at the San Diego Supercomputer Center. Sager conducts research in areas of computer and data network security with emphasis on problems of traffic capture and analysis on high-performance networks. In work with the Pacific Institute for Computer Security (PICS), Sager developed real-time auditing and security compliance monitoring tools for FDDI networks. These tools have been used successfully for intrusion detection and analysis in the SDSC production environment for the last several years.

  2. Currently, Sager conducts network monitoring research in projects PICS and CAIDA. Of relevance to the passive measurement, he is developing auditing extensions of the cflowd (McRobb, CAIDA) NetFlow collection system. These extensions are being developed to concurrently detect network security policy violations. Historical data analysis capabilities are being developed to detect long-term network host-port scanning activities which have been heretofor difficult to detect. Other applications are being explored including statistical analysis of network connectivity data for trending and anomaly detection, and support for Layer-2 tracing of forged-source denial-of-service attacks.

    In research with CAIDA, Sager is developing driver and kernel extensions to the Coral OC12mon to support in-kernel packet reassembly and filtering for security-related applications and other, more general monitoring application where low-level filtering is desired. Sager is also a contributer to the OC48mon preliminary design work.

  3. In work with PICS, Sager plans to perform statistical analyses of SDSC external connectivity (e.g. by protocol, remote host) to assess predictability of such with a goal of performing traffic anomaly detection. He also plans to deploy Coral packet filtering and analysis tools at the SDSC ingress link to perform security monitoring for the SDSC production environment. Sager will also contribute to OC48mon design and driver support.

  4. Tools used: Several modified versions of tcpdump. Coral OC12mon (prototype phase with some hardware compatibility issues). Cflowd plus filtering extensions and database support. Cflowd is stable and quite useful. Filtering extensions are in prototype phase and database support is in design phase.

  5. Link capacity is increasing much more quickly than our measurement and analysis capabilities. Current Coral monitors and tools will not scale beyond OC-48 (if even that!). It isn't clear that router vendors are getting the input (from ISPs, researchers) concerning the measurements that might be needed for network engineering. It isn't clear that routers are the best choice for the measurements anyway. Then, there is the question of what analysis is feasible (scalable) near the core.

  6. http://www.caida.org/funding/ngi1998/content/security/

Scott Shoaf

  1. R. Scott Shoaf, Senior Member Technical Staff at SBC Technology Resources, Inc. working in the Internet Technologies group.

  2. Providing ADSL traffic analysis to the SBC Internet subsidiary. This analysis is to determine ADSL network growth, application usage, and traffic flow patterns as related to the traditional Internet traffic trends.

  3. Future plans include the integration of passive monitoring devices with active network/application monitoring tools to provide an end-to-end coverage of network performance.

  4. New to high speed data collection, mostly experienced with enterprise collection tools

    Compuware EcoScope - good tool for application discovery and reporting, but limited to LAN and low-speed WAN topologies. Provides good visualization of the discovered network. No ability to collect TCP flow data.

    RMON probes

    snoop w/ basic analysis

    HP Internet Advisor and NAI Sniffer analyzer tools

  5. A method of standardized performance metrics that allow for cross-platform verification of data analysis. Also, I hope to see the coupling of passive and active monitoring devices to provide a robust end-to-end analysis of network traffic along with the correlation/analysis tools to pool and reporting on captured information.

  6. (no URLs supplied)

Michael Tesch

  1. Michael Tesch, CAIDA, developer(?), FreeBSD Oc3mon and Oc12mon work. coral reef stuff.
  2. FreeBSD Oc3mon and Oc12mon work.
  3. Do some analsis.
  4. Coral Reef.
  5. ?
  6. ?

Kevin Thompson

  1. Kevin Thompson Senior Engineer Internet Technology and Architecture/ vBNS Engineering MCI WorldCom

  2. Published traffic studies on large backbones (vBNS, internetMCI), responsible for flows monitoring on vbns, snmp monitoring on vbns, starting to look at packet and cell traces from OC-12 trunks

  3. Future concentrations on packet/cell traces on non-flow-controlled Internet applications, on-line gaming apps, self-similarity, changing protocol composition

    • Automated flows and trace collections from 12-15 OC12MONs on the vBNS strengths: low-cost, truly passive monitoring equipment, extremly versatile, fully programmable weaknesses: DOS monitors difficult to manage in volume, software maintenance required, flows monitoring mode suffers from an expired-flow paradigm, etc.
    • in-house-developed SNMP monitoring systems, including use of MRTG

  4. ISPs will need a low-cost plug-n-play monitoring function easy to manage, flexible, reliable, capable of real-time reporting on active flows, especially those flows requiring diffserv/QoS. should include powerful easy-to-use user interface. coral/oc12mon could serve this role with further development including a fully functional Unix implementation that does more than discrete sampling in packet traces

  5. http://www.vbns.net

Marcelo Torres

  1. Marcelo Torres, Network Engineer at GTE-Internetworking. Use network traffic statistics to produce traffic models used in capacity planning activities. Investigation into the area of statistical inference for traffic models.

  2. Consumer of cflowd data for constructionn of traffic models

  3. Interested in developing traffic models that capture short time scale behavior. Would like to learn more about cflowd and the tools used to gather data and their limitations and how those limitations are changing over time.

  4. We rely on cflowd data, and the output of in-house tools that process this data. We have a difficult time handling the quantity of data generated and have to limit our measurement periods as a consequence.

  5. From a capacity planning standpoint, the ablility to sample intelligently and capture traffic statistics on time scales of interest is quite a challenge and of increasing importance. Would also like to see monitoring accurate monitoring of queue occupancy at router interfaces and tools that make it easy to use this data in non-real time and in a real-time control setting.

  6. Can't think of anything new to add here

Brynjar Viken

  1. Brynjar Viken
    Visiting researcher NLANR/MOAT, Sept. 98 - Feb. 99.
    ph.D student at Norwegian Univeristy of Science and Technology, Department of Telematics. Trondheim, Norway
  2. Distributed real-time visualization of measurement data from an oc3mon based on the cichlid tool. Have implemented some visualization servers running on an oc3mon, that collect traffic traces and process the measurement data to generate data matrices. Data matrices are displayed real-time in 3D on a remote visualization client.
    http://moat.nlanr.net/Software/Cichlid/
    http://moat.nlanr.net/SC98Demos/

    During the supercomputing'98 conference NLANR/MOAT placed an OC3mon monitor in the SCinet98 network operation center that monitored the traffic on the OC-3 link connecting the conference network to the vBNS. Currently I'm doing some analyses based on these measurements.
  3. Plan to include analyses of packet traces as a part of my ph.d dissertation.
  4. Oc3mon/Coral

    Used Cichlid to develop servers for distributed real-time visualization of abstracted measurement data from an oc3mon. For further info, see http://moat.nlanr.net/. This stuff could be available for a demo. Need network connection, a server running cichlid connected to a projector or so ..

Jonathan Wang

  1. Name: Jonathan L. Wang
    Organization: Bellcore
    Title: Senior Research Scientist
    Activities relating to Internet statistics: Jonathan currently manages a research program in Bellcore that aims at providing Bellcore's clients a comprehensive and state-of-the-art solutions to Broadband and Internet traffic management. The research activities include traffic data collection, traffic/performance characterization and modeling, and practical traffic management solutions. In the past several years, this research program has been involved in the data collections and traffic analyses from several working networks including SS7/AIN, Frame Relay, and ATM.

  2. Current activities: continuing development of passive monitoring tools and data analysis methodologies for high-speed networks and broadband applications.

  3. Future plans: research and development of a network (traffic) management architecture solution and its associated set of software/hardware tools for supporting next generation networks.

  4. Tools used:
    • Monitoring tools: custom-built Frame Relay and ATM traffic data vacuum
    • Data analysis tools: custom-built statistical analysis software
    • Simulation: custom-built simulation software in various languages
    Strengths: custom-built, relatively easy to modify, relatively fast
    Weakedness:
    • The need to develop new monitoring tools (maybe, from stretch) and adapt to newer technologies and higher speed
    • Statistical analysis tool is not capable of handling large volume of data due to software limitation
    • Simulation software becomes very complex and slow if detail (protocol) modeling is required/implemented

  5. Critical requirements:
    • New monitoring tools/architecture
    • Modeling capability/physical explanations of models
    • Model/traffic predictability
    • Bridge the gaps between research and reality: what needs to be collected regularly through routers, etc.
    • Information sharing

  6. No public URL is available for Bellcore's research programs at this time

David L. Wasley

  1. David L. Wasley, UC Office of the President; Co-Chair, Internet2 Measurements Working Group; Project Director, CENIC CalREN-2 Network.
  2. The I2 Measurements WG is preparing a requirements document for both passive and active measurements across the Internet2 national infrastructure.
  3. See above for a start. I am personally planning to develop models for network costs allocation based on passive monitoring data.
  4. We are recommending to the I2 technical community that all gigaPOPs install OCxMON, Surveyor, and some form of netscarf-like monitor.
  5. Collecting sufficiently granular data at OC12 and above; verifying QoS/CoS parameters as requested by flows; understanding multicast behavior; ...
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    None for the I2MWG yet -- 'real soon now' !

Robert Williams

  1. Name : Robert Williams

    Affiliation : NARUS Incorporated

    Title : VP Engineering

    Relevant Past Activities :

    Microsoft Corp 1988 - 1996
    - Development Manager - Microsoft NetMeeting
    - Development Manager - Remote Network Access - Windows 95
  2. Current Activities :

    NARUS is developing a network analysis platform primarily designed to analysis ISP customer traffic trends. The current version of the product relies on passive probes deployed at POPs and similar locations which capture detailed session and application information. Aggregated data is analysed using a centralized database with a web based front end.
  3. Future Plans :

    We are currently researching probe technology to support higher-bandwidth links and addressing scalability and robustness issues within the database and aggregation model.
  4. Current Tools :

    The current platform is based on Windows NT using Oracle 8 as the database. Solaris and LINUX versions of the platform are under development. To date, all monitoring and analysis software has been developed in-house. We would be happy to demonstrate or discuss as appropriate.
  5. Requirements and Issues :

    1. Providing a more robust monitoring solution.
    2. Scalability to very high aggregate bandwidth (i.e. dealing with many probes at once)
    3. Extracting detailed information on very high bandwidth links (i.e. building a solution that can extract significant information from within the packet rather than just at the packet header level)
  6. Relevant URLs:

    http://www.narus.com

Matt Zekauskas

  1. Name: Matt Zekauskas
    Affiliation: Advanced Network & Services
    Title: Internet Engineer

    Affiliation2: Co-Chair of the Internet2 Measurements Working Group

    Affiliation3: Co-Chair of the IETF IPPM Working Group

    In my main role at Advanced, over the last year I have been leading the deployment of the Surveyor probes and beginning analysis of active one-way delay and loss data collected by those probes. We currently have 38 probes actively measuring 1093 paths, with more on the way.

    In my role as co-chair of the Internet2 Measurements Working Group, I will, along with my co-chair David Wasley, be recommending standard measurements for at least Internet2 gigaPoPs, with the recommendations spreading to Internet2 universities. We intend for these measurements to include passive measures, using some combination of Coral, RTFM, and MRTd tools.

    In my role as co-chair of the IETF IPPM working group, I (along with Will Leland) guide the development of objective performance measurements to measure Internet paths. Personally, I have been involved in developing the one-way delay and loss measurements. While I have implemented these metrics as active measurements, one-way delay in particular is amenable to passive measurement.
  2. I am not performing passive measurement today, but am actively monitoring the passive measurement activities.
  3. In addition to the activities described in point 1, I will be installing a RTFM and Coral (albeit Ethernet-based) monitor in our Armonk NY offices. The principle reason for installing these monitors is to get a feel for how they work, and also to do some flow measurements in our office.
  4. .
  5. Some thoughts -

    a. Usage/deployment for wide-area analysis
    - is there a methodology for consistent measurement over a distributed set of nodes that makes sense to analyze, other than flows? (In addition, flows done in this sense would be useful)
    - are there basic analyses that should be available everywhere? (flows, security (intrusion/"hacking"))
    - set up needs to be easier (I know CAIDA is working on this for Coral); Many of the passive analysis tools don't seem "ready for prime time".

    b. Higher speeds. OC48 is here, and OC192+ is coming. What makes sense given the flood of data? Are there sampling methods that we should use?

    c. Correlation with active
    - sanity checking (one checks the other)
    - "drill down": active detects problems, and passive debugs
    - correlation with protocol impacts?
    (for example, given one-way delay and loss, can we use passive measurement of real traffic to develop analyses that predict "real" traffic behavior)

    d. Security/privacy
    - Often addresses have to be scrambled, and "user" data thrown away to protect privacy. However, by doing this we restrict the analyses that can be done. How can we protect privacy while increasing the number of useful analyses? (This might be especially tricky in coordinating distributed measurements.)
  6. Identification of relevant URLs covering tools or initiatives in this sector:

    You know them better than I. I would say, at least:

    http://moat.nlanr.net/ and all the coral links off that http://www.caida.org/ and the coral and cflowd tools off of http://www.caida.org/Tools/

    Ian Graham's tools off of http://wand.cs.waikato.ac.nz/

    Nevil Brownlee's tools off of http://www.auckland.ac.nz/net/Internet/rtfm and http://www.ietf.org/html.charters/rtfm-charter.html

    Various tools for routing off of http://www.merit.edu/ipma/ and the windmill tool, which I don't believe has a Web page, but is described in http://www.eecs.umich.edu/~rmalan/publications/mjSigcomm98.ps.gz

    One product - HP's Internet Reporter http://www.tmo.hp.com/tmo/datasheets/English/HPJ3307A.html