DOE SCiDAC Bandwidth Estimation Project: Goals and Objectives

We propose to improve existing techniques and tools, and to test and integrate them into DOE and other network infrastructures. The proposed effort will overcome limitations of existing algorithms whose estimates degrade as the distance from the probing host increases. The experience of the investigators with both VPS (Variable Packet Size) and PTD (Packet Train Dispersion) probing techniques makes us uniquely qualified to take on this research challenge. As we improve the algorithms for both techniques, we will incorporate our knowledge into an integrated tool suite that offers insights into both hop-by-hop and end-to-end network characteristics. We also will investigate mechanisms for incorporating bandwidth measurement methodologies into applications or operating systems, so that the applications quickly reach the highest throughput a path can provide. Finally, we will examine ways in which routing protocols, traffic engineering, and network resource management systems can use accurate bandwidth estimation techniques in order to improve overall network efficiency.

Goals and Objectives

[1] k claffy - CAIDA at SDSC University of California, San Diego

Year 1 Focus: Develop and implement a bandwidth measurement methodology:

Task 1 Perform overall coordination and integration of project pieces.
Task 2 Establish and maintain database of actual bandwidth characteristics of measured links.
Task 3 Calibrate tools and techniques for bandwidth estimation of Internet paths, considering their ability to measure bidirectional stability, link capacity, link available bandwidth, and path available bandwidth.
Task 4 Build a single-source testbed for implementing a bandwidth measurement methodology.
Task 5 Begin to build an all-paths testbed for implementing a bandwidth measurement methodology.
Task 6 Collaborate with Georgia Tech group in the development and testing of pathload . CAIDA will be primarily responsible for porting the code to different OS platforms and optimizing the code.
Task 7 Create a front-end GUI for pathload. This interface will generate graphical views of available bandwidth in a path, similar to time-series graphs created by MRTG and RRDtool.
Task 8 Disseminate research results at relevant scientific conferences.

Year 2 Focus: Pilot bandwidth measurement methodology in CAIDA and DOE measurement infrastructure:

Task 1 Perform overall coordination and integration of project pieces.
Task 2 Continue to build an all-paths testbed for implementing a bandwidth measurement methodology.
Task 3 Deploy and test developed modules on CAIDA and DOE measurement infrastructures.
Task 4 Correlate active measurements with passive measurements for estimating available bandwidth.
Task 5 Use resulting tools to upgrade Mapnet to dynamically measure backbone bandwidth.
Task 6 Integrate resulting tools and ensure technology transfer to the DOE community as well as other research and commercial organizations.
Task 7 Investigate algorithms and techniques for measuring reverse path bandwidth and apply the results to other tools being developed for this project, as well as traffic engineering software that can, e.g., use reverse path bandwidth information to select servers.
Task 8 Disseminate research results at relevant scientific conferences.

Year 3 Focus: Implement bandwidth measurement methodology (capacity and available bandwidth) on the high-performance network infrastructures that are available to scientists.

Task 1 Perform overall coordination and integration of project pieces.
Task 2 Refine developed modules to facilitate control and management of high-performance network infrastructures such as DOE 's ESnet.
Task 3 Integrate resulting tools and ensure technology transfer to the DOE community as well as other research and commercial organizations.
Task 4 Collaborate with Georgia Tech group to incorporate available bandwidth measurements into a networking management framework to support engineering, e.g., auto-optimization of paths.
Task 5 Disseminate research results at relevant scientific conferences.

[2] Constantinos Dovrolis - Georgia Institute of Technology

The major goal of our research is the development of bandwidth estimation techniques and tools that will allow scientists to effectively use high-performance network infrastructures. The bandwidth estimation techniques will be used in two areas. First, transport protocols and applications at the end-points will use available bandwidth measurements to achieve higher performance transfers. Second, control mechanisms at the network core will use available bandwidth measurements to dynamically distribute the traffic load effectively among different routes and classes of service. These goals will be realized with the following timeline.

Year 1: Focus: Develop an available bandwidth measurement methodology. Such a methodology does not exist today, and will require a significant amount of basic research. The methodology will be implemented in a measurement tool that will be easy to use by scientists without networking expertise.

Task 1: Experiment with the Variable Rate Path Loading technique (this is an idea that we have been working on during the last year) for the estimation of the available bandwidth in an Internet path.
Task 2: Implement this technique in a new measurement tool, called `pathload’.
Task 3: Test and verify pathload. The verification will be performed using SNMP data from the path routers. The Abilene routers provide us with such data. We will explore whether we can get similar verification data from other networks (e.g., ESnet).

Year 2: Focus: Incorporate the previously developed measurement methodology into transport protocols and applications. Specifically, we will explore ways in which a bulk-transfer application can capture all the available bandwidth in a path. Possible strategies include use of existing protocols in unconventional ways (e.g., multiple parallel TCP connections), modification of existing protocols (e.g., modify the TCP congestion avoidance algorithms for higher performance), and prototyping of a UDP-based large-file transfer application.

Task 1: Explore the use of multiple parallel TCP connections to capture all the available bandwidth of a path. Develop a methodology that dynamically determines the optimal number of parallel connections.
Task 2: Explore modifications in the TCP protocol stack that will allow a single TCP connection to capture all the available bandwidth in a path. These modifications will be both on the TCP congestion control algorithms, and on the TCP implementation and tuning of its parameters.
Task 3: Prototype a UDP-based rate-controlled file transfer application. The application will use its own available bandwidth measurements to determine the transmission rate. Scientists will be able to use this application, instead of a TCP-based FTP application, to transfer large files over high-performance networks.

Year 3: Focus: Incorporate bandwidth estimation methodologies (capacity and available bandwidth) in the control and management of the high-performance network infrastructures that are available to scientists. By "control and management" we mean: intra-domain routing and traffic engineering, provisioning of different classes of traffic, and traffic management.

Task 1: Incorporate available bandwidth measurements in intra-domain routing protocols and/or in the auto-optimization of paths or routing within infrastructure. The basic idea is to use available bandwidth information (collected dynamically through edge-to-edge measurements) in the setup of explicit routes or paths.
Task 2: Incorporate available bandwidth measurements in the provisioning of multi-class differentiated services. Specifically, the bandwidth allocation between classes of service will be dynamically adjusted using edge-to-edge load measurements.
Task 3: Incorporate available bandwidth measurements in traffic management. Two specific ideas that we plan to pursue are, first, to determine whether the traffic entering a path should be shaped (i.e., smoothed) depending on the statistical variations of the available bandwidth, and second, to apply admission control at the ingress points of a path when the path is overloaded (i.e., the available bandwidth is zero).