As part of a two-pronged approach to assess Internet service properties from both the end-user's as well as the service provider's point of view, we investigate performance between a set of endpoint pairs that cover a large geographic area. With help from the NSF supercomputing centers and NASA at FIX-West, NLANR has deployed web caching machines at those strategic locations. While the primary use of the machines is to support a joint project with NSF and Digital Equipment to investigate resource and performance issues in information caching, they also provide an ideal platform for distributed probe studies.
Using a modified ping (using microsecond timestamps as packets as they return from the remote locations), we are now pinging 10 locations throughout the country from each cache every 15 minutes, using a fast 10-packet (64+20 bytes) burst. We attempt to sequence these runs in time across the caching machines, to avoid synchronization among tests (particularly since the caches are time-synchronized). A major objective is attempting to obtain service quality information without being invasive to the infrastructure. For this, we accept a wider margin of error than some other tools which measure, e.g., throughput more precisely, but while imposing possibly significant load on the network. As infrastructural weaknesses are observed, other, likely more service provider centric, tools would have to be employed to determine more exact performance, reasons for the weaknesses, and appropriate responses to improve the situation.
PING Mon Jan 22 11:10:52 1996 www1.cac.washington.edu (140.142.3.7): 64 data bytes at 822337852.548258
> 0 822337852.620408
> 1 822337852.623333
> 2 822337852.632108
> 3 822337852.656483
> 6 822337852.674033
> 7 822337852.674033
> 8 822337852.675008
> 9 822337852.682808
----Mon Jan 22 11:11:02 1996 www1.cac.washington.edu (140.142.3.7) PING Statistics----
10 transmitted, 8 received, 20.00% packet loss.
10.134 seconds elapsed, throughput = 0.79 packets/sec; 530.482 bps.
round-trip (ms) min/avg/max = 50.700/57.403/69.225
var/sdev/skew/kurt = 46.020/6.784/0.715/1.722
The timestamps in the PING line,
show the initial starting time of the 10-packet run, and the lines
beginning with ">" are the microsecond-timestamp
of the returning ICMP Echo Reply packets as they arrive.
The resulting files from the caching machines are collected centrally, and can be processed to assess estimates of Internet performance, e.g.,
src time throughput
min avg max min avg max destination timestamp
bo 4 9.65 10 310 7989 9586 ISI 822556264
bo 6 9.47 10 214 7463 8929 NSF 822556264
bo 5 9.52 10 1778 5606 8115 OSU 822556264
bo 4 9.10 10 303 5314 7660 UUtah 822556264
bo 6 9.96 10 1797 7187 9240 UWash 822556264
bo 7 9.73 10 3837 9091 10928 UCB 822556264
. . .
1 day
2 days_
3 days
7 days
14 days
30 days
Standard, easy-to-use, and minimally invasive Internet performance metrics from an end-user perspective are very difficult to obtain. How does a customer know what performance to expect? We try to use these simple ping results to provide a very rough indication. We advocate this only as steps toward thinking in the right direction; we recognize that these assessments are not optimally accurate, but rather an example of what people can accomplish today, and a baseline from which to develop more accurate metrics.
Each point in a given graph below graph represents daily average/peak ratios of loss vs throughput. More specifically, there are about 24 data point loss-throughput pairs (one for each hour), so a single dot represents the mean loss divided by the peak loss on the x-axis, versus the mean throughput divided by the peak throughput on the y-axis.
In other words:
for each 10-packet ping session, x <= 10
packets actually arrive
and those packets get through within a certain throughput y, in bps.
Then we take summaries of these ping sample pairs.
Each dot is based on one day's worth of ping tests
(at one per hour, that's about 24, though many don't even make it
through the test due to, well, the Internet.
Someone should really investigate that.)
So for the 24 tests you get:
X = average(numbers in set (a) ) / max (numbers in set (a)), andso this is some measure of the variability in service, the average service you get, divided by the maximum that you could get.
Y = same for (b)
You can get larger versions of each graph by clicking on them, but we will try to explain implications of some of the data. The legend for each graph shows:
| color | abbrev | src sitesite |
|---|---|---|
| white | sd | san diego |
| red | sv | fix west |
| green | bo | boulder |
| blue | uc | urbana-champaign |
| yellow | pb | pittsburgh |
| brown | it | ithaca |
The dots in the upper right show paths that are
fairly well-connected, i.e., Pittsburgh and
Ithaca seem to have less loss and throughput
variation to ISI than do San Diego, Silicon Valley,
Boulder, and Urbana-Champaign.
Urbana-Champaign has particularly bad connectivity to ISI,
and the connectivity from Northern to Southern
California is quite variable, consistent
with anecdotal evidence (at least mine).
(Now one can go back and find out why with traceroutes and other tools, but that's a step we haven't taken yet, and would be quite tedious, so tools to automate that would be good).
Things aren't great to Utah (on right): Urbana-Champaign,
Pittsburgh and Boulder have the most trouble with it.
But it's the opposite for Wisconsin, OSU
(on right and below).
Urbana-Champaign and Boulder do okay getting there;
Silicon Valley, San Diego, and Ithaca
are unhappy sources.
Florida isn't great from anywhere.
To get a sense of the range of the above data, we can draw diamonds around the mininmum and maximum values for each data, with the averages crossing at the diamond `center'. We'll just show this for a few of the destination sites, to give an idea.
Note the San-Diego-to-Berkeley (right) and
San-Diego-to-Washington (below) paths have
a wide range of performance,
consistent with the data from Example 2.
Utah, to the right,
and UWisconsin, below
Texas to the right, and Oregon State below.
