• predict in order to maximize scarce resources
• facilitate network operation, planning, design
• competitive advantage in service provision
• support billing/usage-based services
• ISPs should be active in defining metrics and developing analysis techniques, as well as collecting data and educating users on correlation between metrics and actual network performance  

 what investments would yield maximum value for the ISPs?

• traffic engineering mechanisms (e.g., CflowD, AS matrices)
• ways to characterize/track path performance wrt QoS/bottlenecks
• better integration of measurement tools into networking equipment
• closer cooperation/collaboration among communities (R&E, ISP, vendor, customer) wrt designing tools, data acquisition efforts (better coupling with what problems are being solved), postprocessing/analysis
• development of high fidelity multi-platform simulation tools that permit real time feedback from multiple ISPs while ensuring data confidentiality

 what investments would yield maximum value for end users?

• performance monitoring tools/data that assist in making competitive vendor selection / contract negotiation and in ongoing monitoring of ISP services
• tools that assist in identifying/diagnosing problems, e.g., id of bottleneck on lan vs. isp vs. general Internet
• end users are most interested in metrics observed at the edges of the network & should be involved in the definition of these metrics

 research/higher ed communities and government are engaged in efforts to develop more robust tools and deploy a measurement infrastructure across the Internet. do you have any recommendations for how these efforts can help to produce meaningful insights into current traffic conditions and scalability issues, including what aspects of measurement/analysis these communities should pursue jointly with ISPs?

• before you collect, clearly articulate what question you are trying to answer, or what hypothesis you are testing
• collection, aggregation, correlation, dissemination of data on traffic loads, performance levels, network outages
• in addition to new tools (e.g., pathchar, tcpanaly), need better interpretation of existing tools (traceroute, ping, ttcp, dig), perhaps augment their functionality/deployment so end-users can offer ISPs more useful feedback
• vendor access to data for equipment design/implementation (but vendors should take advantage of the wealth of data currently available, e.g., RA BGPstuff)
• confidentiality/privacy, tools to anonymize collected data
• improve the ability of network elements (routers, switches, etc.) to accurately measure and report network performance data, e.g., embedded RMON technology

 what role should other users (e.g., content providers, companies with mission-critical networking requirements) play in this process?

• monitor end-to-end performance, react/communicate w. their ISP when not meeting expectations/guarantees
• regular feedback to tool developers to review tools' goals, purposes, users
• content providers should be directly involved in traffic measurements; results applied to understanding customer needs and to improve protocols and implementation of products

• end-to-end traffic measurement and analysis
• identifying appropriate metrics/meaningful measurement (IPPM has lead in the IETF, but other communities should take a more proactive role in this process)
• address evolving nature of the Internet, e.g., measurement implications of IPSEC, higher speeds (modem and wires), queuing, scaling issues...
• developing tools/simulators that reflect real-world situations, e.g., brownouts and similar 'soft' outages, and tools that are not overly invasive to the net (passive monitoring techniques where feasible)
• better realtime diagnosis of problems, esp routing-related
• standardization of methodologies, protocols, data formats, etc. making tools widely available

• little agreement here... except wrt significant challenges
• coordination of disparate efforts
• techniques as passive as possible
• distributed acquisition, warehousing, analysis
• information sharing/privacy
• communities working together

• very different opinions depending on community, e.g for....
• ISPs - homegrown measurement/analysis tools the norm, albeit increasing interest in some flow measurement tools (CflowD; OC3mon)
• end-users - ping and traceroute are important (however, accurate interpretation of results is very difficult!)
• for researchers - new tools such as pathchar and tcpanaly have promise; distributed, scalable collection of measurement data, warehousing and meaningful analysis and simulation of traffic behavior are challenges
• commercial & research measurement tools are being summarized in CAIDA Tool Taxonomy ( www.caida.org/Caidants/meastools.html ) -- send inputs to INFO@CAIDA.ORG

from workshop conclusions panel
2 May 1997
Questions or comments should be directed to Tracie Monk at tmonk@caida.org