Slide 1: preparing for the future

preparing for the future 
in the midst of a typhoon:

measurement and analysis 
of the Internet

       scientific apparatus offers a window to knowledge, 
       but as they grow more elaborate, 
       scientists spend ever more time washing the windows. 
                                     -- Isaac Asimov 



kc claffy, UCSD/SDSC/CAIDA
kc@caida.org 
www.caida.org

Slide 2: CAIDA's evolution

CAIDA's evolution 


created late 1997 at ucsd/sdsc 
seed grant from NSF 
sponsorship from Cisco

goals
foster engineering-level cooperation among industry, research, government 
tools/analysis for robust, scalable Internet 
topics beyond purview of single networks/vendor

Slide 3: CAIDA's evolution

CAIDA's evolution 


1998
hardware (OC3/12 monitors with MCI)
viz, analysis of commercial infrastructure
(Nature of the Beast - iMCI)
cflowd; active measurement (skitter)
education (IEC), outreach (ISMAs), training

1999   
measurement, analysis and viz tools
analysis results (active measurement & multicast) 
skitter, mantra, cflowd, coral
education, outreach & training (EOT)
ITL

Slide 4

outline of today's talk


strengthening academic Internet engineering education 

priorities for Internet measurement 
   and analysis

synergies in the Cisco / CAIDA relationship

Slide 5: [EOT]: Internet Eng. Curriculum

[EOT]: Internet Eng. Curriculum 


initiated Jan'98 w NSF & Cisco support 
lead: Prof. Evi Nemeth
http://iec.caida.org

problem: need to improve quality of graduating Internet engineers

approach: 
broad university-industry collaboration
dynamic repository of state-of-the-art Internet engineering curriculum materials for university coursework & continuing education

Slide 6: [EOT]: Internet Eng. Curriculum

[EOT]: Internet Eng. Curriculum 


activities:
advisory group 
(aiken, bellovin, bradner, bush, connolly, crowcroft, kurose, wilder)
workshops for faculty
august '99, faculty (Cisco-supported - routing, tcp, ns, traffic analysis)
june '00, faculty
traffic analysis training CD 
lectures, exercises, data sets
beta released January 00
web version: http://traffic.caida.org
Cisco-supported

Slide 7: [EOT]: Internet teaching labs (ITL)

[EOT]: Internet teaching labs (ITL)
                

problem: univ.'s lack hands-on facilities for teaching Internet engineering

approach
jan 99: Cisco (Estrin) commits ~100 trade-in routers 
aug 99: RFP announced
        round 1: twelve universities selected  (38 applied)
        round 2: pending
dec 99: NSF commits funds for glue 
            (training, collaboration, support)
today: blocked on 7000s' removal from C&W POPs

Slide 8: [EOT]: other workshops

[EOT]: other workshops 
                

goal: bring together key players in communities 

ISMAs: passive meas't, viz, analysis 
SDNAP: BGP/MBGP for ISPs (SDNAP)
ISP tools: cflowd, RRDtool

future workshops:  
correlation among datasets 
routing, topology, performance, workload
active measurement (SLAs)

Slide 9: priorities in measurement & analysis

priorities in measurement & analysis


topology (mapping)

workload characterization (passive)

performance evaluation (active)

routing (dynamics)


  will show examples, priorities, obstacles

Slide 10: topology: skitter

topology:  skitter

macroscopic, infrastructure-wide 
dynamically discover/depict topology (& b/w)
correlate path perf. w events, e.g. BGP
identify critical pieces of infrastructure

Slide 11: skitter: infrastructure-wide measurements

skitter: infrastructure-wide measurements

17 monitors (inc. root name servers)
multiple dst lists (29k servers, 36k dns)
architecture:
     - parallel ICMP probes
     - 52-byte packets
     - kernel time stamping
     - ssh / Kerberos

Slide 12: skitter: AS interconnectivity

skitter: AS interconnectivity

Slide 13: GTrace: geographic traceroute

GTrace: geographic traceroute 
 
   www.caida.org/Tools/GTrace/

Slide 14: skitter: 3D hyperbolic

skitter: 3D hyperbolic 
 
BGP routing table data (connectivity among ASes)



    %%  

      %%layout: geographical
      %%layout: semi-geographical
     %%

Slide 15: mantra: analyzing multicast

mantra: analyzing multicast 

www.caida.org/Tools/Mantra

Slide 16: logical vs geographic topology

logical vs geographic topology

2-dimensional, hierarchical

geographic

Slide 17: semi-geographical (otter)

semi-geographical (otter) 

www.caida.org/Tools/Otter

Slide 18: topology: research priorities

topology:  research priorities 

accurate measurement & visualization
latency
key routers/networks
AS granularity
geographic
integration w mgt tools

obstacles: 
     mapping IP addresses to
router
geography  
AS
service provider
country 	
anything...

    route changes faster than can measure

Slide 19: workload characterization

workload characterization


insights for
usage profiling 
h/w, protocol, application design
architecture optimizing
capacity and peering planning 
network control and management 
security
performance analysis
delay, loss, jitter?
QOS assurance across ISPs
accounting and billing

tools: netramet, netflow, cflowd, coral
some suck less? ...evolution requires use

Slide 20

workload char: working w/vendors


cflowd
www.caida.org/Tools/Cflowd 
primarily for capacity planning and trend analysis 
Cisco's netflow export  

AS-to-jS matrices 
net-to-net matrices 
port and protocol tables 
forward IP path 
 ==> line rate

measurement specifications to vendors

Slide 21

workload char.: protocol 

19 aug 99, ucsd-cerfnet
https://anala.caida.org/CoralReef/Demos/

Slide 22

workload char.: protocol (proportion)

19 aug 99, ucsd-cerfnet

Slide 23: workload char: applications (ucsd-cerfnet)

workload char: applications (ucsd-cerfnet)

Slide 24: workload char.: mantra

workload char.: mantra
18 oct 99, fix-west.mbone.nasa.gov
http://www.caida.org/Tools/Mantra/

Slide 25: packet sizes, 4/98.mci vs 7/99 AIX

packet sizes, 4/98.mci vs 7/99 AIX

Slide 26: packet sizes by prefix length

packet sizes by prefix length

larger packets from shorter prefixes (why?)

Slide 27: favoritism/locality by AS

favoritism/locality by AS

80% of traffic from < 5% of ASes
60% of reachability from < 7% of ASes

Slide 28: flow length distribution, 4/13/98 mci backbone

flow length distribution, 4/13/98 mci backbone

heavy tail (quite truncated)
if you only learn about one distribution this quarter...
flow defn creates artifacts (100 bytes)

Slide 29: workload char of high perf. networks

workload char of high perf. networks

coral/ocXmon testing (oc3,12,48,gE)
persistent real-time full frame collection
integration w coralreef analysis s/w
dag4.0 testing planned 1/00

Slide 30: security of high perf. networks: coralreef

security of high perf. networks: coralreef


CoralReef 3.2 release 12/16/99
ATM: Applied Telecom and Fore OC3 (OC12 App. Tel. only)
ATM/POS:OC3/12 (DAG3.2 testing 1/00)

crl_portmap s/w module
listens for RPC portmap access
adds suspicious probing host to list
records all send-rec'd packets
full payload capture (default)
tcpdump output format

Slide 31: workload characterization: coralreef

workload characterization: coralreef


crl_filter
ATM reassembly
tcpdump output format
pipes to security tools

future
OC12 (full line rates), OC48 need card support
         [obviously way (& getting further) behind switching...]
active enforcement module (yr 2000)
getting ISPs to use them


    www.caida.org/Tools/CoralReef

Slide 32: workload characterization: priorities

workload characterization: priorities

id and present `useful' workload metrics, particularly given persistence of fire-fighting environment
                
id significant patterns, timeframes, correlations
vary by user need
change as technologies and 'net change

obstacles:
limited access to commercial networks
network performance impact
faster speeds and changing transport technologies complicate data acquisition and processing

Slide 33: workload characterization: priorities

workload characterization: priorities

coral/ocXmons (OC3,12,48, gigE)
persistent, realtime, full-frame collection 

security policy 
compliance auditing (passive) 
enforcement (active) 
dynamic packet filtering triggered by attack precursors 

SLA support

obstacles
hardware expensive
privacy issues 
IPsec

Slide 34: performance evaluation (active)

performance evaluation (active) 
 


network engineers to diagnose problems

ISPs & users to verify SLAs

designers of real-time apps to predict software HCI

Internet weather reports

Slide 35: perf.eval: skping (RTT, loss, analysis)

perf.eval: skping (RTT, loss, analysis)

Slide 36: perf.eval: routing (path change)

perf.eval: routing (path change)

Slide 37: perf.eval: sktrace (www.cnet.com)

perf.eval: sktrace (www.cnet.com)

Slide 38: skitter: rtt distribution: tri-modal

skitter: rtt distribution: tri-modal

Slide 39: skitter: rtt vs longitude (light cone)

skitter: rtt vs longitude (light cone)

Slide 40: dispersion among ASes across paths

dispersion among ASes across paths

Slide 41: skitter: correlation (perf., topology, routing)

skitter: correlation (perf., topology, routing)


preliminary findings
~1% IP destinations disappearing monthly (re-addressing, firewalls)
route announced path not matching forward path
indication of potential routing configuration errors
    (by no means automatic)
persistence of paths
methods to identify critical infrastructure
is there an Internet "core"?

datasets available to researchers

Slide 42: performance eval.: priorities

performance eval.: priorities


faster collection, processing, rendering

bandwidth assessment techniques

need intuitive graphic presentations correlating:
performance across sources
comparisons w/topology, workload, routing analyses
        
obstacles 
poorly defined user requirements/interfaces
negative perceptions regarding quality and worth driven by explosive growth

Slide 43: routing dynamics

routing dynamics


[nothing this room doesn't know...]

15-year-old technology

well, it works... 

not much instrumentation/diagnostics
really need real-time
without interfering w performance
(or other engineering priorities)

makes analysis hard

Slide 44: routing: example (instability)

routing: example (instability)


RTT data changes color if path changes
10 unique paths over 24 hour period
lots of jitter in data
unlikely to be intentional 
heavy tails predominate

Slide 45: routing: example (load balancing)

routing: example (load balancing)


RTT similar over predominantly two paths 
likely intentional load balancing

Slide 46: routing: RouteViews table analysis

routing: RouteViews table analysis

Slide 47: routing analysis: research priorities

routing analysis:  research priorities


real-time identification & vis of flaps, outages, critical paths
unintended consequences of new policies, topology
propagation of change across ISPs
realistic inter-domain routing models

==> requires better instrumentation
      (w/o interfering with forwarding..)
      ideal: route-lookups in real-time w/o kernel

Slide 48: routing: research obstacles

routing:  research obstacles 


canonical BGP (route table) data	(not so much anymore)

routes may change faster than ability to measure or analyze
        
mapping IP addr to anything (deja vu)

prudent security dictates making research difficult

Slide 49: overall: meas't & analysis challenges

overall:  meas't & analysis challenges


new methods for data collection, reduction, aggregation, mining, viz 
large, complex datasets ( ~Pbyte )
geographically and logically distributed 
dynamically changing
enable inter- and intra-ISP analysis 
       and feature detection
correlation among data sources/types
user-friendly integration with network utilities and control systems
proactive participation
top-down (app devel's scope constr.) 
bottom-up (ISP cooperation) 
vendors in middle (to right of research)

Slide 50: summary: CAIDA/Cisco relationship

summary:  CAIDA/Cisco relationship


education, outreach and training
IEC, ITL, ISMAs
complementary tools/products 
OC3/48mons
router-based statistics 
netflow, cflowd
route-views (Cisco/UO [Meyer])
SNMP management tools 
infrastructure insights (multicast)
traffic analysis (patterns, trends)

Slide 51: www.caida.org/Presentations/

www.caida.org/Presentations/

kc claffy
UCSD/SDSC/CAIDA
kc@caida.org
www.caida.org

Slide 52: skitter: macroscopic study

skitter: macroscopic study 

DNS f root server (pv's):  path wingspans 
www.caida.org/Tools/Skitter

Slide 53: skitter: rtt vs hopcount (correlation?)

skitter: rtt vs hopcount (correlation?)

Slide 54: skitter: rtt distribution: tri-modal

skitter: rtt distribution: tri-modal

Slide 55: skitter: rtt vs longitude (light cone)

skitter: rtt vs longitude (light cone)

Slide 56: dispersion among ASes across paths

dispersion among ASes across paths

Slide 57: dispersion among ASes across paths (sdsc)

dispersion among ASes across paths (sdsc)

Slide 58: dispersion among countries across paths

dispersion among countries across paths

Slide 59: skitter analyses - preliminary findings

skitter analyses - preliminary findings


~1% IP destinations disappearing monthly (re-addressing, firewalls)
route announced path not matching forward path
indication of potential routing configuration errors
    (by no means automatic) 
persistence of paths
methods to identify critical infrastructure
is there an Internet "core"?

datasets available to researchers

Slide 60: CAIDA workscope summary

CAIDA workscope summary


research pushing boundaries
analysis of complex conditions
management of large datasets
correlation among different datasets
development of timely, insightful visualization

recognized needs
tool integration
user interface improvements
networked collaborative environments

Slide 61: summary: CAIDA/Cisco relationship

summary:  CAIDA/Cisco relationship


education, outreach and training
IEC, ITL, ISMAs
complementary tools/products 
OC3/48mons
router-based statistics 
netflow, cflowd
route-views (UO/Meyer)
SNMP management tools (new - dwm)
infrastructure insights (multicast)
traffic analysis (patterns, trends)

Slide 62: www.caida.org/Presentations/

www.caida.org/Presentations/

kc claffy
UCSD/SDSC/CAIDA
kc@caida.org
www.caida.org