Slide 1: Internet measurement:

Internet measurement:

abysmal science


       scientific apparatus offers a window to knowledge, 
       but as they grow more elaborate, 
       scientists spend ever more time washing the windows. 
                                     -- Isaac Asimov 



kc claffy, UCSD/SDSC/CAIDA
kc@caida.org 
www.caida.org

Slide 2: abysmal but unsurprising

abysmal but unsurprising


little capacity to predict, depict, or even measure traffic behavior on current and advanced networks 

few tools to engineer/operate networks or identify traffic anomalies in real time

doesn't stop researchers from building junk

doesn't stop random users from doing random junk

increasing cost to infrastructure

Slide 3: no dearth of activity

no dearth of activity




     http://www.caida.org/Tools/taxonomy.html

Slide 4: payoffs

payoffs


insights for vendors re next generation hw/sw requirement

calibration for users, e.g., monitoring service level agreements's 

diagnostic and planning tools for ISPs 

windows into the infrastructure for researchers

Slide 5: four areas of measurement

four areas of measurement


topology

workload characterization (passive)

performance evaluation (active)

routing (dynamics)


  will show examples, priorities, obstacles

Slide 6: infrastructure-wide priorities

infrastructure-wide priorities 

dynamically discover/depict topology (& b/w)
correlate effects of BGP routing changes
correlate path performance with specific events

Slide 7: skitter (jul 1998)

skitter  (jul 1998)

Slide 8: skitter: infrastructure-wide measurements

skitter: infrastructure-wide measurements

6 monitors
23k (eventually 60k) destinations
architecture:
     - parallel ICMP probes
     - 52-byte packets
     - kernel time stamping
     - ssh / Kerberos


www.caida.org/ISMA/isma9808/slides/isma98-skitter/

Slide 9: skitter: colored by hop distance from src

skitter: colored by hop distance from src

Slide 10: skitter: colored by IP octets

skitter: colored by IP octets

Slide 11: skitter: AS interconnectivity

skitter: AS interconnectivity

Slide 12: skitter: 3D hyperbolic (feb 1999)

skitter: 3D hyperbolic (feb 1999)

198.32.130.15, still reflecting large portions elsewhere
tamara munzner (stanford) layout software

Slide 13: skitter: 3D hyperbolic

skitter: 3D hyperbolic 
 
BGP routing table data (connectivity among ASes)

Slide 14

manta: visualizing the mbone
 
geographical placement of mbone using data from 
mwatch/mrinfo utilities
 

 
Bradley Huffaker (UCSD/CAIDA) developer

Slide 15

manta: mbone colored by metric

Slide 16

otter: general purpose viz tool

Slide 17

otter: main display

Slide 18

AS path data by adjacencies vs paths

Slide 19: topology: research priorities

topology:  research priorities 

visualization
latency
key routers/networks
AS granularity
geographic

obstacles: 
     mapping IP addresses to
router
geography
AS
service provider
country 	
anything...

    route changes faster than can measure

Slide 20: workload characterization

workload characterization


workload profiling (s/w & h/w design, architecture optimizing, capacity planning

security
performance analysis

QOS assurance across ISPs
accounting/billing

tools: netramet, netflow, cflowd, coral
some suck less? ...evolution requires use

Slide 21: workload char.: traffic matrix (ASes)

workload char.: traffic matrix (ASes)

12 march 98, 2 min sample, fixwest

Slide 22: workload char.: matrix (country)

workload char.: matrix (country)

12 march 98, 2 min sample, fixwest

Slide 23: workload: IP trade balance (pkts)

workload: IP trade balance (pkts)

12 march 98, 2 min sample, fixwest 
US still net exporter of IP pkts (w/2 exceptions)

Slide 24

Slide 25: key IP protocols (13 apr 98): bytes, pkts

key IP protocols (13 apr 98): bytes, pkts

Slide 26: key applications

key applications 

mci (now c&w) backbone trunk
24 hour period 13 apr 98
bytes (packets similar)

Slide 27: key applications (flows), 4/13/98 mci backbone

key applications (flows), 4/13/98 mci backbone

Slide 28: traffic vs prefix length

traffic vs prefix length

5 minutes, 5 dec 97, domestic mci link
routing table from UO route-views
depends much on ISP aggregation policies

Slide 29: packet sizes, 4/13/98 mci backbone

packet sizes, 4/13/98 mci backbone

half pkts 40 bytes, half payload in 1500 byte pkts

Slide 30: flow length distribution, 4/13/98 mci backbone

flow length distribution, 4/13/98 mci backbone

heavy tail (quite truncated)
if you only learn about one distribution this quarter...
flow defn creates artifacts (100 bytes)

Slide 31: IP fragmentation, 4/[13-20]/98 mci backbone trunk

IP fragmentation, 4/[13-20]/98 mci backbone trunk

Slide 32: workload characterization: priorities

workload characterization: priorities

coral/ocXmons up to OC12 (OC48)

persistent, real-time, full-frame collection 

dynamic packet filtering triggered by attack precursors 

security policy 
compliance auditing (passive) 
enforcement (active) 

 obstacles
hardware expensive
privacy issues 
IPsec

Slide 33

workload char: working w/vendors


Cisco routers (cflowd) - up to OC3 speeds

http://www.caida.org/Tools/Cflowd 
used primarily for capacity planning and trend analysis 

AS-to-AS matrices 
net-to-net matrices 
port and protocol tables 
forward IP path 

measurement specification
   http://www.caida.org/Docs/meas_spec.html

Slide 34: performance evaluation (active)

performance evaluation (active) 
 


network engineers to diagnose problems

users to verify SLAs

designers of real-time apps to predict software HCI

Internet weather reports

Slide 35: perf. eval: skping (www.freebsd.org)

perf. eval: skping (www.freebsd.org)

Slide 36: perf.eval: routing (path change)

perf.eval: routing (path change)

Slide 37: perf.eval: sktrace (www.cnet.com)

perf.eval: sktrace (www.cnet.com)

Slide 38: performance eval.: priorities

performance eval.: priorities


definitions & metrics
bandwidth assessment techniques
correlation with workload data 
large scale deployment
user interface to measurements


obstacles

core infrastructural access
mathematicians needed
statisticians needed

Slide 39: routing dynamics

routing dynamics


1970's technology

admittedly seems like pretty good stuff

sausage/laws....

incredibly inflexible, inefficient, incantation-driven

Slide 40: routing: example (instability)

routing: example (instability)


RTT data changes color if path changes
10 unique paths over 24 hour period
lots of jitter in data
unlikely to be intentional 
heavy tails predominate

Slide 41: routing: example (load balancing)

routing: example (load balancing)


RTT similar over predominantly two paths 
likely intentional load balancing

Slide 42: routing: research priorities

routing:  research priorities

effects of outages on surrouding ISPs
effects of topology changes on Internet performance
unintended consequences of new policies
detect areas for improving an individual networks' ability to respond to congestion and topology changes
vulnerabilities created by dependencies on critical paths
utilization of address space
efficiency of routing table
asymmetric, unstable routing by provider
incongruity of unicast with multicast

Slide 43: routing: research obstacles

routing:  research obstacles 

canonical BGP (route table) data	(not so much anymore)

mapping IP address to anything 
   (we've been here before)

prudent security dictates making research difficult

Slide 44: now what?

now what?

`seamless': no such thing 
measurement tools/infrastructure 
well-considered
strategically deployed
collaboratively maintained 
more infrastructure-relevant research on resulting data
feedback into tool design 
correlation among data sources/types, simulation, visualization 
proactive participation
    top-down (app devel's scope constr.) 
    bottom-up (ISP cooperation)

Slide 45: researchers

researchers

in difficult position

responsible for
   creating windows into infrastructure
no access to glass 
poke holes as methodically as possible
much activity w/much less integrity by others

without trust & collaboration from ISPs,
   field not going anywhere,
   tools have to help them first

funding sources should push relationships

Slide 46: www.caida.org/Presentations/

www.caida.org/Presentations/




kc claffy
UCSD/SDSC/CAIDA
kc@caida.org
www.caida.org