Slide 1: bandwidth

bandwidth estimation:
measurement methodologies and applications

february 2003
 
kc claffy, ucsd/sdsc/caida
kc@caida.org 

constantinos dovrolis, georgia institute of technology
dovrolis@cc.gatech.edu

Slide 2: motivation

motivation


who needs better bandwidth estimation?
 	
high throughput data-intensive applications

scientific visualization

colaboratory interactions

remote sensor analysis

Slide 3: background: metrics definitions

background: metrics definitions



capacity
narrow link

available bandwidth
tight link

TCP throughput

Slide 4: background: existing tools

background: existing tools

capacity (end-to-end)
pipechar (?)
sprobe
pathrate

capacity (per hop)
pathchar
pchar
clink

available bandwidth
pathload
pipechar (?)

TCP throughput
netest-2  (?) 
treno
iperf

Slide 5: goal: calibrate existing tools

goal: calibrate existing tools


how do tools perform against cross-traffic?

what are requirements for running tools?
logistical 
security

what is the overhead? 
time-to-measure
injected traffic

are the results valid?

Slide 6: testing methodology

testing methodology


install tools on two freebsd end hosts

use calngi lab
fully controlled environment
varying test topology
100M links
GigEther links
baseline measurements with no cross-traffic
simulating cross-traffic as pseudo-random traffic flows
SmartBits hardware
SmartFlow software

Slide 7: lab testing: simulation of cross-traffic

lab testing: simulation of cross-traffic 


simulated cross-traffic by SmartBits/SmartFlow

basic parameters:
number of flows, N
packet size for each flow (constant), p
order of flows round-robin to approximate pseudo-random traffic
percentage of the link utilization, 10% < u < 90%
burst size, b <= N 

application 
sends b flows back-to-back (burst)
takes time t
waits for time d = t*(100-u)/u
repeats the burst, cycling the pre-determined flows

link transmits bits for u% of the time on average

Slide 8: lab testing results

lab testing results

layer 2 store-and-forward devices affect tools accuracy
not handled by pathchar, pchar, clink

C. Dovrolis, R.S. Prasad and B. Mah,
"the effect of layer-2 store-and-forward devices on per-hop capacity estimation"(ieee infocom 2003)

not all routers are created equal
internal switches
different buffer configurations
use of "slow path" for ICMP traffic

accuracy of tools deteriorates at high loads
we have shown that per-hop capacity estimation tools (pathchar, pchar, clink) produce consistent and significant errors when path includes layer-2 store-and-forward devices.
all paths we have experimented with include one or more such devices
real and important problem
paper available on const web page and at infocom 2003

upshot: testing bandwidth estimation tools is nontrivial

Slide 9: lab testing: layer 2 devices foil bwest tools

lab testing: layer 2 devices foil bwest tools 


100Mbps


accuracy deteriorates at high loads

Slide 10: lab testing: layer 2 devices foil bwest tools

lab testing: layer 2 devices foil bwest tools 


gigE

pathchar, pchar, and clink fail
pipechar underestimates by 30%

Slide 11: lab testing: router configuration matters

lab testing: router configuration matters


host -> JuniperM20 -> Foundry -> host

Slide 12: lab testing: router configuration matters

lab testing: router configuration matters


identical links, different order of routers
host -> Foundry -> JuniperM20 -> host

Slide 13: lab testing: cross-traffic sensitivity

lab testing: cross-traffic sensitivity




pipechar measurements do not reflect cross-traffic

Slide 14: gatech tools: pathrate

gatech tools: pathrate



measures end-to-end capacity (narrow link)
more reliable, accurate than pathchar, pchar, clink

Slide 15: gatech tools: pathload

gatech tools: pathload



measures available bandwidth (tight link), fast w minimal intrusion
uses new self-loading periodic streams (SLoPS) algorithm
C. Dovrolis and M.Jain, "End-to-End Available Bandwidth: Measurement methodology, Dynamics, and Relation with TCP Throughput" SIGCOMM 2002

Slide 16: problems at high bandwidth

problems at high bandwidth 


working on getting pathrate and pathload to work on high-bandwidth paths (600Mbps to 1Gbps)
pipechar measurements have shown clearly that pipechar does not measure available bandwidth, but rather capacity
problematic, since tool claims  otherwise
both pathrate and pathload have been tested successfully in paths limited by OC-12 (640Mbps) link
gigE paths still challenging when network interfaces batch interrupts.  we have ideas on how to fix the problem..
more about this at PI meeting in March


 important for DOE PIs to 
 give us feedback w our tools!

Slide 17: bandwidth estimation tools gui

bandwidth estimation tools gui


ANEMOS architecture
clients (web applets)
select a path to monitor
add new rules for data analysis   
view previous results

workers
execute bwest tools

coordinator
manages workers 
maintains scheduling queue for active measurements
analyzes measurements
issues user alarms
archives results

Slide 18: tools: bwest gui: even a user interface!

tools: bwest gui: even a user interface!


ANEMOS: an Autonomous NEtwork MOnitoring System

gui for network operators and end-users
schedule, perform, analyze bwest-like measurements

Slide 19: tools: bwest gui features

tools: bwest gui features


ANEMOS (to be presented at PAM03)
coordinated measurements of delay/loss/avbw in multiple paths
GUI for monitoring results through MRTG graphs
resulting measurements archived with MySQL, retrieved with GUI-specified queries.
user can specify "rules" on the resulting measurements for automatic processing and alarm detection (e.g., issue alarm if RTT > 100msec and avbw < 2Mbps in paths X and Y).
code available in april 2003

Slide 20: need scidac researchers' help

need scidac researchers' help


use and give us feedback on our tools please!
many recent scidac-related publications cite pathcar or pipechar and not this project's tools.
work on optimizing bulk TCP transfers in high bandwidth-delay product paths:
Dunigan, Mathis, Feng, Tierney, Rice  
why?  they all need better bandwidth estimation
important to integrate w scidac projects
especially for improving TCP or applications

Slide 21: future directions: research

future directions: research


Internet spectroscopy
observe cell or slot-based traffic on broadband links

passive measurement
does not require additional probes

analysis of inter-packet delay distributions
radon transform
entropy minimization

infer specific layer 2 provisioned bandwidth	
based on subtle "noise" features

Slide 22: future directions: engineering

future directions: engineering


use of bandwidth estimation in maximizing bulk TCP throughput in high bandwidth-delay product paths.
we demonstrate that significant gains result when the socket buffer size is set based on bwest measurements.
application-layer technique for automatic socket buffer sizing, based on bwest.  technique does not require any TCP or OS changes
annual bwest workshop planned summer 2003

Slide 23: acknowledgments

acknowledgements

PIs
constontinos dovrolis
kc claffy
key admin and technical personnel
margaret murray
nevil brownlee
awesome grad students 
ravi prasad (pathrate, effects of layer-2 devices on pathchar-like tools)
manish jain (pathload)
antonios danalis (anemos)

http://www.caida.org/projects/bwest/
margaret murray, marg@caida.org