Slide 1: CAIDA projects related to

CAIDA projects related to
Internet data visualization


// What information consumes is rather obvious: 
it consumes the attention of its recipients. 
Hence a wealth of information creates a poverty of attention, 
and a need to allocate that attention efficiently among the
overabundance of information sources that might consume it. //
-- Nobel laureate Herbert Simon, 1995.


ucsd/sdsc/caida
bradley@caida.org 
http://www.caida.org/publications/presentations/

Slide 2: caida

caida  (cooperative association for internet data analysis)

research programs
active: macroscopic topology project 
passive: (real-time) traffic workload characterization
analysis of DNS root and gTLD server performance
security issues, e.g., DOS measurement/analysis/modeling
routing analysis and modeling
bandwidth estimation methods and tools
Internet Measurement Data Catalogue (IMDC)

other areas
tool development
new network visualization metaphors
empirically guided Internet public policy 
outreach & education

Slide 3: visualization

visualization


 
     Scientific visualization is the process by which 
     some attributes or relationships are highlighted 
     at the expense of others.  

     It is the selection of which attributes or 
     relationships are to be preserved and which are 
     to be hidden or distorted that drive the 
     visualization task.

Slide 4: terminology

terminology

IP - Internet Protocol 
protocol used to connect different IP-based data networks together
IP address is fundamental
this presentation drops the word "address"

AS - Autonomous System 
collection of contiguous IPs which can announce IP prefixes
AS number uniquely identifies an AS (ISPs may have more then one)
this presentation drops the word "number"

BGP - Border Gateway Protocal
primary protocol used by ASes to exchange routes
BGP table is a collection of BGP routes seen at a single router

RTT - Round Trip Time
time to send and receive a response between two IP addresses 

protocol
collection of rules that define a service (HTTP, TCP, etc)
defined by port address

Slide 5: analysis of Domain Name System (DNS)

analysis of Domain Name System (DNS) 

DNS is an indispensable Internet component 
two sets of 13 machines = root & gTLD name servers
CAIDA has done several studies of multiple dimensions of DNS performance 
passive monitoring of performance
active probing of latencies
analysis of root name server logs
the infrastructure is changing now...
use of multicast, anycast

visualization
DNS server root groups
effect of removing a DNS root server
effect of moving a DNS root server

Slide 6: DNS server root groups

DNS server root groups



presentation
roots grouped by "RTT client difference" 
axis
color - average absolute difference between clients' RTT
X - group and root server
Y - root server
highlights
color helps the eye to cluster values into groups

Slide 7: effect of removing a DNS root

effect of removing a DNS root



presentation
number of clients whose delta RTT was greater, when the DNS root was removed, than the RTT on the X-axis
axis
X - delta RTT
Y - number of clients
highlights
which server had the largest number of clients who would be most affected by the server's removal

Slide 8: effect of moving a DNS root

effect of moving a DNS root



presentation
distribution of clients' delta RTT
axis
color - shows the top 90%, 75%, 50%, 25%, and 5% points of the distribution
X - individual DNS root servers
Y - delta RTT
highlights
human eye has difficulty with width differences, color helps make the different areas more visible.

Slide 9: traffic workload characterization

traffic workload characterization 

passive measurements of Internet data streams
develop techniques of high speed traffic sampling
measure busy OC3, OC12 and OC48 backbone links
CAIDA has the only publically available OC48 backbone IP flow data in the world
note: sprintlabs has private oc48 data
study how user activities produce torrents of bytes
testing TCP models in presence of bursty cross traffic
detection of long running streams
tracking Internet usage patterns
IETF WG developing flow measurement standards 
Nevil Brownlee (CAIDA/U.Auckland) and David Plonka (U.Wisc) co-chairs

visualization (next slides)
continent to continent matrix
breakdown by protocol

Slide 10: continent to continent matrix

continent to continent matrix


presentation
traffic aggregated by country of announcing AS's headquarters
axis
color - protocol of traffic
left - source continent (sorted by total bits sent)
right - destination continent (sorted by total bits sent)
z - logarithm of the number of bytes sent
highlights
continents with low and high bit rates
values near front and/or high values are easily seen
effective for general public presentations

Slide 11: continent to continent matrix

continent to continent matrix

Slide 12: traffic breakdown by protocol

traffic breakdown by protocol


presentation: flowscan tool  -- flexibly shows traffic by protocol 
color - protocol of traffic
X - time ; Y - bits seen in given protocol plus the sum of those below in graph
highlights
gives total for subsets of protocols
relative sizes of different protocols

Slide 13: interdomain routing

interdomain routing

routing dynamics analysis
evolution of global routing system
{ AS, prefix, IP address } level granularities
prerequisite to improvement of routing infrastructure 
robustness, performance, integrity
analyzing two-hour snapshots of BGP tables

visualization
geopolitical distribution of Internet resources

Slide 14: geopolitical distribution of Internet resources

geopolitical distribution of Internet resources 


presentation
geographic breakdown of Internet space by headquarters of announcing AS
color - country
CIA Factbook - presented in the online CIA Factbook
BGP table - headquartered in a given country (RouteViews)
highlights 
disparity among Internet assignments across countries

Slide 15: geopolitical distribution of Internet resources

geopolitical distribution of Internet resources 


highlights: disparity among Internet assignments across countries

Slide 16: macroscopic topology project

macroscopic topology project

connectivity analysis
massive macroscopic traceroute data -  largest IP topology data set in world
funding from NSF, DARPA, commercial sources
establishing framework for topology analysis
combinatorial structures (trees, acyclic part, core, giant component)
description by Weibull distributions
mapping IP -> AS -> city, country -> latitude, longitude
BGP routing tables: IP -> prefix -> AS 
whois registry/hostnames: AS, IP -> city,country -> latitude,longitude
note: there is no public trusted source of this data at this time 
               (even CAIDA's has not been funded in 2 years)

visualization 
IP paths: single source, single destination
IP paths: single source, few destinations
AS/country paths: single source, many destinations
IP paths RTT: single source, many destinations
IP topology: many sources and destinations
AS topology: many sources and destinations
geopolitical distribution of Internet resources

Slide 17: IP paths single source, single destination

IP paths single source, single destination




presentation
IP path and intermediate RTT changes over time

axis
X - time in minutes
Y - RTT in milliseconds

highlights
path changes
points of large latency

Slide 18: IP paths: single source, few destinations

IP paths: single source, few destinations




presentation
bidrectional IP path between single source and few destinations
axis
X - AS that announces the IP address
Y - hop count from source
highlights
path symmetry 
shared path from source to multiple destinations

Slide 19: AS paths: single source, many destinations

AS paths: single source, many destinations


presentation
proportion of traces passing through a given AS path from a single source
axis
color - AS of a given hop.
black means the path has ended
grey is multiple tiny paths
X - distance in terms of the number of IP hop count
Y -  number of paths which followed from the previous hop
highlights
key peering points and major providers

Slide 20: country paths: two sources, many destinations

country paths: two sources, many destinations


presentation
proportion of traces passing through a given country path from two sources
axis
color - country of a given hop.
black means the path has ended
grey is multiple tiny paths
X - distance in terms of the number of IP hop count
Y -  number of paths that followed from the previous hop
highlights
key peering points and major providers

Slide 21: IP paths RTT: single source, many destinations

IP paths RTT: single source, many destinations


presentation
shows the maximum latency experienced for destination along a given IP path
axis
color - maximum latency for destinations down stream
X,Y - 3D hyperbolic geometry (fisheye distortions)
highlights
paths for which all downstream destinations had large RTT values.

Slide 22: IP topology: many sources, many destinations

IP topology: many sources, many destinations


presentation (note -- bell labs / lumeta layout algorithm)
IP spanning tree (does not include all links)
axis
color - AS of longest matching prefix
X,Y - set by spring/force layout
highlights
Internet size and major providers

Slide 23: AS topology many sources, many destinations

AS topology many sources, many destinations


presentation
IP forward path -> AS topology 
axis
color,radius - logarithmic of AS outdegree
degree - geographic longitude of AS headquarters
highlights
`large' AS geographic locations and interconnectivity (US remains hub)
2001-2003 (new poster to be released this summer in collaboration w WIDE)

Slide 24: security issues

security issues

global denial of service activity
backscatter methodology (CAIDA invented)
detecting denial-of-service (DOS) activity on the global Internet
monitoring spread of worms in the networks
Nimda, Code Red, Sapphire, ... (to be continued)
first (and remains primary) publically available data quantifying DOS

visualization
number of Code Red infected hosts in 24.0.0.0/8

Slide 25: number of Code Red infected hosts in 24.0.0.0/8

number of Code Red infected hosts in 24.0.0.0/8



presentation
number of infected hosts in BGP prefixes
axis
node - BGP routed prefix, or less specific prefix
link - connects more and less specific prefixes
color - number of infected hosts 
X,Y - 3D hyperbolic geometry (fisheye distortions)
highlights
shows clusters of infected hosts in groups of related prefixes

Slide 26: Number of Code Red infected hosts in 24.0.0.0/8

Number of Code Red infected hosts in 24.0.0.0/8




highlights
shows clusters of infected hosts in groups of related prefixes

Slide 27: summary: visualization efforts at caida

summary: visualization efforts at caida


remark: visualization is a medium, not an end
visualization is tool to increase our ability to think
visualization can help make us smart
it can also make us stupid by misadvised mappings and unworkable user interfaces


CAIDA's focus is on analysis rather than visualization
however, we recognize that the community looks to CAIDA for visualization results
research and operational communities
since we have respected experience with analysis

Slide 28: Bradley Huffaker

Bradley Huffaker 
ucsd/sdsc/caida
bradley@caida.org

http://www.caida.org/publications/presentations/