Network operators need to determine the composition of the traffic mix on links when looking for dominant applications, users, or estimating traffic matrices. Cisco's NetFlow has evolved into a solution that satisfies this need by reporting flow records that summarize a sample of the traffic traversing the link. But sampled NetFlow has shortcomings that hinder the collection and analysis of traffic data. First, during flooding attacks router memory and network bandwidth consumed by flow records can increase beyond what is available; second, selecting the right static sampling rate is difficult because no single rate gives the right tradeoff of memory use versus accuracy for all traffic mixes; third, the heuristics routers use to decide when a flow is reported are a poor match to most applications that work with time bins; finally, it is impossible to estimate without bias the number of active flows for aggregates with non-TCP traffic.

In this paper we propose Adaptive NetFlow, deployable through an update to router software, which addresses many shortcomings of NetFlow by dynamically adapting the sampling rate to achieve robustness without sacrificing accuracy. To enable counting of non-TCP flows, we propose an optional Flow Counting Extension that requires augmenting existing hardware at routers. Both our proposed solutions readily provide descriptions of the traffic of progressively smaller sizes. Transmitting these at progressively higher levels of reliability allows graceful degradation of the accuracy of traffic reports in response to network congestion on the reporting path.

Eight real packet traces collected at various times of day from OC-48 links at two different ISPs, combined with various amounts of simulated DDoS traffic.

Measured error and bias of packet and byte counts estimated by Adaptive NetFlow for various traffic aggregates as a function of report size, and compared to a theoretical perfectly-configured "Psychic" sampled NetFlow.

Given equal report sizes, ANF gives estimates as good or better than those of "Psychic" NetFlow with respect to accuracy of byte and packet counts, and can actually be implemented. FCE's estimates are better than those of SYN counting techniques, and also work on non-TCP flows. Unlike existing NetFlow, both ANF and FCE are robust to extreme traffic mixes, gracefully degrading the accuracy of their estimates to stay within resource bounds.

• Cisco NetFlow, http://www.cisco.com/warp/public/732/Tech/netflow
• Sampled NetFlow, http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120s/120s11/ 12s_sanf.htm