Slide 1: measuring the `real' Internet:

measuring the `real' Internet:

acquisition of empirical data in support 
of Internet modeling and simulation

19 jul 2000
darpa nms kickoff

kc claffy, UCSD/SDSC/CAIDA
kc@caida.org 
www.caida.org

Slide 2: outline: data sets for NMS research

outline:  data sets for NMS research


where we are

how we got here

what data we have now

what data we can get soon

what we should do next

Slide 3: Internet's resistance to modeling

Internet's resistance to modeling


evolution-based (good!) reasons
protocols, technologies, applications
independently developed and deployed
by no means syngergistic
by all accounts rapid
`punctuated' but no equilbrium 

but simulation/analysis validation requires core data
right granularities hard to come by
measurement technology just not there
argument for it also not there
losing battle?

Slide 4

Internet's resistance to measurement
                

many would benefit
vendors, users, researchers, ISPs

ISPs would bear cost
multiple media: atm, pos, dwdm, mpls
logistics/management
privacy implications
analysis/research obsolete after (before) done


   ........how to justify measurement??

one answer: tools ISPs want or need

Slide 5: measurement tools lack

measurement tools lack


well-defined traffic metrics
e.g supporting SLAs or billing
uniformly applied methodologies
varied topologies, equipment, ISP practices
clear definition of measurement hypotheses or goals
measurement scalability
ability to explain phenomena
topology changes, routing loops, black holes
relevance to actual ISP problems or mechanisms for fixing 
communication of useful results

Slide 6: publically available data sets

publically available data sets


not comprehensive
no such thing as representative 
case study mentality behooves us

data sources, tools
student-compatible

Slide 7: Internet measurement taxonomy

Internet measurement taxonomy


topology  (circulatory/respiratory)
performance (physiology/psychology)
workload  (cardiovascular/GI)
routing   (neuroscience)


   correlation essentially non-explored
     .....(holistic Internet measurement?)

Slide 8: Internet topology data

Internet topology data


why do we care?
simluation and modeling validation
traffic engineering
track global growth/change
arguable: increased potential/manageability

macroscopic, IP layer
Lucent: burch/cheswick maps
CAIDA: www.caida.org/tools/measurement/skitter

Slide 9: topology: caida's skitter

topology:  caida's skitter

track/depict topology cross-sections 
22 monitors (inc. some root name servers)
forward IP path and round-trip delay 
tens of thousands of dst (mult. lists)
remove targets that complain
150k nodes, 270k links in 5 days (11/99)

architecture
continuous, parallel 52-byte ICMP probes
depending on dst list size, O(1) probes/hr/dst
kernel time stamping

correlate path perf. w events, e.g. BGP
identify critical pieces of infrastructure
case studies of relevant cross-sections

Slide 10: other active (probed) data sets

other active (probed) data sets

MOAT: http://amp.nlanr.net 
HPC sites
RTT, traceroutes

I2's Surveyor: http://www.advanced.org/surveyor/
I2 sites
one-way delay, paths

vBNS  http://www.vbns.net:8080/stats/ 

SLAC, NIMI, others  

http://www.caida.org/tools/taxonomy/
http://atlas.caida.org/

Slide 11: skitter: colored by countries

skitter: colored by countries

Slide 12: topology vis: geographic mapping

topology vis:  geographic mapping


difficult data analysis
requires mapping of thousands (millions?) of nodes to latitude/longitude coordinates

NetGeo service designed to help
http://netgeo.caida.org

backbones require company-specific heuristics

DNS registry growth is problematic
no common data formats

Slide 13: topology/perf.: priorities

topology/perf.: priorities 


dynamic feature detection from large, complex datasets
data aggregation/reduction techniques
faster data collection, processing, rendering
meaningful displays, user-friendly tools
correlation with different datasets

large scale public database of performance data
across many sources
comparisons w/topology, workload, routing analyses

Slide 14: topo./performance: obstacles

topo./performance: obstacles


poorly defined user requirements/interfaces
negative perceptions regarding quality and worth driven by explosive growth
uniform methodology impossible
mapping IP addresses to ... anything meaningful
       (not just geography)

  .....things getting worse not better

Slide 15: BGP policy: US as transit

BGP policy: US as transit

transit: neither src nor dst
can only answer for connectivity, not traffic

Slide 16: BGP policy: US as transit

BGP policy: US as transit

US transit for 71.5% of paths (not traffic!)
100% to MX, 98% to peru, chile
80-90% for cn, hk, tw, au, nz
for most paths, US only 3rd party
2nd biggest transit country: canada
AU provided transit for 46% of all paths to NZ
0 means < 1% (blank means 0)

      all   AU    CA    C_H   JP   KR   MX   NZ  SEA  SWA   TW    US
____________________________________________________________________________
US  71.5  77.8 82.0 90  49.5  61.6 100  79.6  63.0  97.8   83.5
CA  13.3   8.3          4.9  37.5   2.1                27.5  22.3   1.3   0.2
AU   2.8                 18.4                        46.1   1.6              0.4
JP    1.2          1.4    7.4        10.5            12.0               0.3
NZ   0.9   3.7
EUR  0.7                2.1        1.7               4.2  27.0
UK   0.7   0.0   0.0         0.1              0.0    5.8  21.1         0.2
SEA  0.3   0.7           5.6
AR   0.1                                                   5.2
AE   0.1                                             1.9
CH   0.1                                                   2.8
MM  0.1                                             1.6

Slide 17: skitter case study: DNS roots

skitter case study: DNS roots

RSSAC, DNS technical advisory committee to ICANN

goal: optimize root nameserver location
co-locate skitter hosts w root servers
demonstrate root server performance in serving target community
develop techniques for evaluating architectual optimality for root server placement 
visualization to correlate data sources/types

use collaborative project to encourage proactive participation (network operators, researchers, others) 

(www.caida.org/tools/measurement/skitter/)

Slide 18: skitter: rtt distribution: tri-modal

skitter: rtt distribution: tri-modal

Slide 19: skitter: rtt vs longitude (light cone)

skitter: rtt vs longitude (light cone)

Slide 20: skitter: rtt vs longitude (light cone)

skitter: rtt vs longitude (light cone)

Slide 21: skitter: dispersion among ASes across paths

skitter: dispersion among ASes across paths

Slide 22: skitter: AS dispersion across paths (sdsc)

skitter: AS dispersion across paths (sdsc)

Slide 23: skitter: country dispersion across paths

skitter: country dispersion across paths

Slide 24: DNS roots study: future

DNS roots study:  future 


get other roots instrumented
gather/analyze client lists 
correlation among different sources
determination of connectivity metrics 
closeness
redundancy
persistence of paths
how many clients not secondaries
skitter to client sets from non-root sources

Slide 25: skitter: other interesting possible studies

skitter: other interesting possible studies

RTT versus distance 
earth circumf., X, X+Y, to-US-fr

Slide 26: skitter: rtt versus distance

skitter: rtt versus distance

london source
lower band: directly connected 
upper band: thru US to rest of Europe

Slide 27: skitter: rtt versus distance

skitter: rtt versus distance

tokyo monitor
european paths `close' but via US

Slide 28: skitter on-going daily summaries

skitter on-going daily summaries


http://www.caida.org/tools/measurement/skitter/summary

path length (in IP hops) distribution
RTT distribution
RTT versus longitude, 
path dispersion 
AS & country granularity

Slide 29: Internet workload

Internet workload 

many uses
capacity planning 
performance and QOS assurance across ISPs
accounting/billing
security management

measurement tools
router-based (cflowd, netflow)
stand-alone monitors (coral,skitter)

visualization huge challenge
too much data 
noone correlates across/with much

evolution requires use
envisioning new methods?
better data correlation tools are essential

Slide 30: available data: (passive) header traces

available data: (passive) header traces


coral: oc3/oc12 `real' networks
HPC sites: http://moat.nlanr.net/Traces/

tcpdump: campus/corporate sites
http://ita.ee.lbl.gov/html/traces.html

Slide 31: public tools: passive data collection

public tools:  passive data collection


 snoop
 netramet
 tcpdump
 tcpdpriv
 tcptrace
 libpcap
 libcoral
 coralreef

Slide 32: public tools: passive

public tools:  passive 

snoop 
ships with solaris
supports filter patterns

netramet
RTFM flow meter	
Coral interface
lots of supporting utilities
used for accounting in NZ, elsewhere

tcpdump
LBL
ships with free Unices
supports filter patterns
understands many network types
well patched, supported, leveraged, evolved
2 tools:  capture  (pcap) and display (text output)

Slide 33: public tools: passive -- (cont.)

public tools:  passive -- (cont.)


tcpdpriv
like tcpdump for privacy-concerned
encodes IP addresses/ports/strip payload, etc
option to exit after N packets or M seconds
deals with pcap/tcpdump files, same bpf filter patterns.

tcptrace
post-processes tcpdump files
examines TCP sessions
measures RTT between endpoints
generates xplot input files (session dynamics)

Slide 34: public tools: passive -- (cont.)

public tools:  passive -- (cont.)


libpcap
device-independent capture library
canonical filter rules
framework for many tools, e.g., tcpdump

libcoral
similar goals to libpcap
supports [ATM] cell-based networks
equivalent perl API

coralreef
built on libcoral
collection of analysis tools
automated reports

Slide 35: data sets: research issues

data sets: research issues


IP address protection

protected environment

payload

disk space

Slide 36: dataset issues

dataset issues


location

clocks

filtering/encoding

size

damage

Slide 37: workload: AIX-MAEW ipsec (ah/esp)

workload: AIX-MAEW ipsec (ah/esp)

almost 10X increase last half of 1999 
then levels/declines relative to elsetraffic

Slide 38: workload: AIX-MAEW fragmentation

workload: AIX-MAEW fragmentation

relevant to recent IP traceback techniques [Savage00]
definitely on rise (from UDP) at AIX
almost no TCP frags (MTU disc + small pkts)

Slide 39: workload: AIX-MAEW online game traffic

workload: AIX-MAEW online game traffic

decline for games included
    Starcraft, Quake II, and QuakeWorld (a variant of Quake II)
popular last year, then other games take over?

Slide 40: workload: online gaming cont.

workload: online gaming cont.

looking at new games plus old
    Half Life, Quake 3: Arena, and Unreal
    median higher -> gaming traffic on rise, but moving target
    increase mostly from new games, older games wane

Slide 41: workload: AIX-MAEW gaming trends

workload: AIX-MAEW gaming trends

clearly more popular on weekends (nearly double!)

Slide 42: workload: AIX-MAEW napster traffic

workload: AIX-MAEW napster traffic

3 ports used in late jan: 6688, 6697, 6699
ports migrated in march as universities blocked
even so, dramatic increase (> 50% in feb->mar)

Slide 43: workload: AIX-MAEW email trends

workload: AIX-MAEW email trends

significant increase around nov/dec, then drops off 
online commerce / holiday shopping?

Slide 44: workload: summary of AIX findings

workload:  summary of AIX findings


packet size distribution stable
TCP:UDP ratio fairly stable
order magnitude increase in IPSEC mid-last year, then level/decline
significant increase in UDP fragments
decrease in active-mode FTP and realaudio
increase in gaming and napster
increase in email during holidays
strong weekday/weekend pattern in gaming

Slide 45: workload data: meta-challenges

workload data:  meta-challenges

splintered & competitive core 
limited access to data
so difficult to argue `representativeness'

network performance impact 
higher b/w increasing difficult to measure
faster speeds and changing transport technologies complicate data acquisition and processing 
e.g. monitor gone when AIX converts to POS

user privacy volatile issue
hard to get data in researchers hands

  CAIDA's UCSD/CERFnet link monitor available:
    https://anala.caida.org/CoralReef/Demos/

Slide 46: workload data: challenges

workload data:  challenges

id and present `useful' workload metrics, particularly given persistence of fire-fighting environment

id significant patterns, timeframes, correlations
vary by user need
change as technologies and 'net change

methodology has many weaknesses 
dynamic port negotiation (napster)
tons of `other' ports unmapped
ports not really assurance/unique anyway
IPSEC blows away ports anyway
need traffic profiling 
things getting worse not better here

Slide 47: routing & addressing data

routing & addressing data


not much real-time instrumentation on routers

UO's route-views  http://www.antc.uoregon.edu/route-views/

Merit's IPMA http://www.merit.edu/ipma/

Slide 48: routing: differencing routing tables

routing: differencing routing tables 

www.caida.org/Tools/Mantra (multicast)

Slide 49: routing: address consumption

routing: address consumption 

prefix length distribution for routes announced by core ISPs, 1-6/1998 (courtesy NLANR/MOAT, Jeff Brown)

Slide 50: routing: address consumption (#hosts)

routing: address consumption (#hosts)

reachable hosts for routes announced by ISPs, 1-6/1998

Slide 51: routing: address usage of *traffic* sample

routing: address usage of *traffic* sample

32x32 `bitmap' matrix of address space 
height is % packets with src IP in that address block

Slide 52: multicast data sets (using mantra)

multicast data sets (using mantra)

http://www.caida.org/tools/
daily updates 
mbgp, msdp statistics, topology maps/diffs

Slide 53: routing: research priorities

routing:  research priorities 

better IP routing instrumentation 
real-time analysis without interfering with performance
realistic inter-domain routing models

tasks
identification/vis of flaps, outages, critical paths
correlation performance problems with some measure of path `length'
comparison of forward path with
BGP path
shortest path
does asymmetry matter? 
effects of unicast/multicast incongruities?

Slide 54

routing:  research obstacles


routes may change faster than ability to measure or analyze
sometimes on purpose (load-balancing)

poorly instrumented infrastructure (new tools needed)

prudent security dictates inhibiting research 

mapping IP address to anything 
  (deja vu)

Slide 55: now what?

now what?  


the ideal:
well-instrumented infrastructure 
seamless integration of variety of data sources
important for simulation/prediction
but unlikely for the foreseeable future

tools still need:
interpret of vast quantities of data in real-time
geographically & logically distributed 
user-friendly integration with network utilities 
       and control systems
inter- & intra-ISP feature detection
new methods for data collection, reduction, 
       aggregation, and mining (GByte or Tbyte datasets)

Slide 56: setting expectations

setting expectations

rule 1: no magic data sets
(not so far anyway)

   the so-called science of poll-taking 
   is not a science at all 
   but a mere necromancy. 
   people are unpredictable by nature, 
   & though you can take a nation's pulse, 
   you can't be sure that the nation 
   hasn't just run up a flight of stairs. 

               --E. B. White New Yorker, Nov 1948.

Slide 57: www.caida.org/Presentations/

www.caida.org/Presentations/

kc claffy
UCSD/SDSC/CAIDA
kc@caida.org
www.caida.org