CAIDA : tools : taxonomy

Performance Measurement Tools Taxonomy

This section concerns tools for measuring Internet performance. A summary is provided along with web page pointers to more detailed information. Review comments are also included when available.

| Index Anonymization Topology Workload Performance Routing Multicast Measurement Infrastructures |

co-sponsored by:

Application or E-business Performance

Tool	Input	Measurement		Output
		Active / Passive	Functions
OPNET IT Guru / ACE	RMON probes, packet traces, direct net capture, router config. files, network management data	both	delay, jitter, loss, app. response time, retransmissions, prediction	Windows / UNIX GUI, graphs, web reports
CompuWare EcoScope	LAN, WAN, GigaEther	both	app distribution across bandwidth; response time; traffic char; client/server usage	Windows GUI
NetPredict	network pkts	passive	captures network pkts; performs statistical analysis	GUI, graphs

Bandwidth / Throughput Measurement

Tool	Input	Measurement		Output
		Active / Passive	Functions
bing	compare RTTs from ICMP/ECHO reply of different size pkts	active	raw bandwidth	raw capacity (bps)
{b\|c}probe	ICMP/ECHO reply	active	bandwidth, congestion	text
clink	UDP w/limited TTL to unused port; ICMP err response or ICMP/ECHO reply	active	bandwidth	text
iperf	TCP and UDP	active	bandwidth, delay jitter, pkt loss	text or GUI
netperf	Load generator	active	uni-directional throughput, end-to-end latency	text
nettimer	pkt collection; pkt tailgating	both	pkt capture server, pkt capture client, active probe (tailgater)	records; space delimited fields in ns trace file format
pathchar	UDP; ICMP	both	bandwidth, throughput, latency, loss, queue chars	GUI
pathload	UDP pkts, running in client-server mode	active	bandwidth (maximum IP-layer throughput that a flow can get in the path from sender to receiver, without reducing the rate of the rest of the traffic in the path)	NetLogger style output
pathrate	UDP based, running in client-server mode	active	absolute capacity (bottleneck bandwidth)	NetLogger style output
pchar	UDP; ICMP	both	bandwidth, throughput, latency, loss, queue chars	GUI
SProbe	IP address of the other endpoint of the network path being measured along with the direction of the measurement (a single network path has two directions).	active	bottleneck bandwidth(both forward and reverse path)	text
TReno	UDP w/limited TTL to unused port; ICMP err response or ICMP/ECHO reply	both	bandwidth	gnuplot and ppmtogif graphs
ttcp and nttcp	load generator	active	throughput	text
Viznet	data in real time from a network or logfile generated by Netlog	passive	bandwidth visualization	real-time animation

Forward Path Probes

Tool	Input	Measurement		Output
		Active / Passive	Functions
GTrace	UDP w/limited TTL; ICMP time exceeded	both	latency, reachability	map, name, addr, RTT of each hop along path
mtr	ICMP ECHO	active	link quality; reachability	graphical(ncurses) or text(Gtk)
Nikhef traceroute	UDP w/limited TTL; ICMP time exceeded	active	latency, reachability, pkt loss	name, addr of each hop along path & stats
pingplotter	ICMP Echo/Reply	active	availability, %pkt loss	windows, graphs
traceroute	UDP w/limited TTL; ICMP time exceeded	active	latency, reachability, pkt loss	name, addr of each hop along path
WhatRoute	UDP w/limited TTL; ICMP time exceeded	both	latency, reachability	MAC window; name, addr of hop, map
Xtraceroute	UDP w/limited TTL; ICMP time exceeded	active	reachability	world globe plot

Internet Cloud Monitoring

Tool	Input	Measurement		Output
		Active / Passive	Functions
Narus Intelligence	SNMP	passive	monitor IP services and usage	flow info
NIKSUN NetVCR, NetReporter, NetDetector	SNMP	passive	traffic flow, per-application response time, retransmission rate, RTT, end-to-end loss, bandwidth, throughput	flow info

Internet Weather

Tool	Input	Measurement		Output
		Active / Passive	Functions
for News hosts	ping	active	Pkt loss from U Oregon eXchange point to >100 news hosts	status, %pkt loss, min/avg/max
Andover Internet Traffic Report	ping	active	Global Internet Traffic, RTT, pkt loss	graphs, index (0-100)

Monitoring Services (Application or E-business)

Tool	Input	Measurement		Output
		Active / Passive	Functions
Gomez GPN	Web site ~53 global agent locations near ISP core routers	active	web site benchmarking & testing; web page access times, latency, transaction sequence, host availability; real-time service alerts; drill-down analysis for tests, comparative performance	GUI, graphical reports, email alerts, FTP or XML feeds
Gomez PeerReview	Web site ~ 10,000 end-user desktop locations at different connection rates	active	end-to-end performance measurement; on-demand diagnostic measurement; network latencies; ISP and backbone Testing; Competitive Benchmarking; real-time service alerts	GUI with extensive reporting on performance metrics and connectivity rates
KeyLabs	Web-site	active	web site benchmarks and testing	GUI
Keynote Perspective	Web transaction script recorder from ~50 global Agent locations	active	access times, latency for: full page; benchmark single Web object; multi-page; interactive transactions; secure pages	MyKeynote web interface
ServiceMetrics SM-WEB	Web-Site (global, regional, or local measurement sites)	active	response time; availability; comparative performance	GUI

Monitoring Services (ISP)

Tool	Input	Measurement		Output
		Active / Passive	Functions
BrixWorx software	measurements from Brix100 or Brix1000 Verifiers	both	end-to-end testing of application traffic between a mesh of Verifiers, or between Verifiers and guaranteed, IP-based network or application services	Web based GUI or software controlled alarms
Brix 1000 Verifier	Active Test traffic	active	Measure performance from ISP POP, data center, or NOC, including round-trip or one-way latency, jitter, and both simulated and actual app throughput (e.g., POP, SMTP, DNS, NNTP, RTP, H.323, HTTP, HTTPS)	BrixWorx web based GUI, cmd line interface, SNMP agent v1, SNMP MIB II, Ethernet MIB, EtherStats
Brix 100 Verifier	Active and Passive Test Traffic	both	measure rount-trip latency, jitter, and both simulated and actual app throughput (e.g., POP, SMTP, DNS, NNTP, RTP, H.323, HTTP, HTTPS) at customer enterprise (network edge router or other location)	BrixWorx web based GUI, SNMP agent v1, SNMP MIB II, Ethernet MIB, EtherStats
Brixnet Managed Service	measurements from Brix 100 and/or Brix 1000 Verifier	both	end-to-end testing of application traffic between a mesh of Verifiers, or between Verifiers and guaranteed, IP-based network or application services	BrixWorx web based GUI or software controlled alarms
MRTG	uses SNMP to read the traffic counters of users' routers	passive	monitors traffic, logs the traffic data, and represents them in graphs; also able to monitor any SNMP variable that the users choose	webpages containing graphs
Nettest	TCP/UDP	active	runs Iperf or user supplied additional tests	NetLogger style output
NetVigil	snmp, tcp, applications	both	delay, bandwidth, server metrics (CPU, memory, etc.)	Web based GUI, real-time status, statistics, graphs
snmpstatd	SNMP	active	monitoring operative network, performing asynchronous SNMP polling to collect usage statistics, memory and CPU utilization from routers	text file

Link Utilization

Tool	Input	Measurement		Output
		Active / Passive	Functions
IPTraf	Linux net stats	passive	TCP connect pkt/byte counts, interface stats/activity, TCP/UDP %traffic	console screen
libpcap	machine-independent network layer	passive	pkt collection library	file
tcpdump	based on libpcap	passive	portable packet collector	file
tcpdpriv	based on libpcap	passive	sanitizes collected packets	file
Traffic Explorer	Netflow and full routing topology history	Passive	Prefix-aggregated flows all links based on mapping flows across routes, aggregate and per CoS volume and utilization	GUI

Network Simulation (Capacity Planning)

Tool	Input	Measurement		Output
		Active / Passive	Functions
dummynet	packet intercept	active	bandwidth mgmt	simulated traffic
OPNET Modeler	RMON probes, packet traces, direct capture, router config. files, network management data	both	delay, jitter, loss, app. response time, retransmissions, utilizations, predictions, performance analysis	Windows / UNIX GUI, graphs, web reports
OPNET IT Guru / ACE	RMON probes, packet traces, direct capture, router config. files, network management data	both	delay, jitter, loss, app. response time, retransmissions, utilizations, predictions, performance analysis	Windows / UNIX GUI, graphs, web reports
ServiceProvider Guru	RMON probes, packet traces, direct capture, router config. files, network management data, SNMP	both	delay, jitter, loss, app. response time, retransmissions, utilizations, predictions, performance analysis, automated network design	Windows / UNIX GUI, graphs, web reports
Traffic Explorer	Netflow and full routing topology history	Passive	Volume and utilization per link, flow, path; Trending, predictions, what-if scenario simulation of routing and traffic changes	GUI

One-way Availability/Latency Tests

Tool	Input	Measurement		Output
		Active / Passive	Functions
echoping	ICMP Echo/Reply	active	availability, latency, pkt loss	text
fping	ICMP Echo/Reply	active	multihost availability, latency, pkt loss	parseable text
gnuplotping	ICMP Echo/Reply	active	multihost availability, latency, pkt loss	gnuplot graph of delay distribution
Imeter	ICMP Echo/Reply	active	long-term ping performance	web page graphs
Nikhef ping	ICMP Echo/Reply	active	availability, latency, pkt loss	text
owamp	IP address	active	latency	text
ping	ICMP Echo/Reply	active	availability, latency, pkt loss	text
sting	TCP behavior	passive	one-way pkt loss rates	text
Traceping	ping & traceroute	active	pkt loss	text