Short-term Analysis

By default, FlowScan's window for short-term analysis is 48 hours, allowing easy comparison of traffic parameters on two consecutive days. Standard graphs supplied with the FlowScan distribution provide views of 1) traffic by network subnet, 2) application or service type, and 3) Autonomous System (AS) during the past two days. Standard graphs can present these views using bits-per-second, packets-per-second, and flows-per-second measurement scales. All these graphs use five minute averages, particularly useful for detecting network abuse such as Denial of Service (DoS) attacks.

Experience using FlowScan shows that a discrepancy between the number of inbound and outbound flows or packets is a possible indicator of abusive traffic. Sudden changes in packet counts, especially when constrained to one protocol, are usually indicative of a DoS flood. The graph below highlights a traffic flood that went unnoticed on bandwidth usage graphs and was too subtle to be easily noticed on packet count graphs.

DoS Traffic Flood Detected by FlowScan:

In the example graph above, the traffic responsible for the spikes in inbound TCP flows was determined to be a flood of incoming 40-byte TCP ACK packets directed to a campus host that responded with 40-byte TCP RST packets. If a flood of small packets was sent to dynamically changing campus host addresses, spikes show up better in a flow graph than a packet graph, because the spike is a much larger proportion of the total flows than it is of the total packets.

When visualizing over the short-term, it is important to remember that FlowScan increments traffic counters at the time when the flows are exported. More precisely, it records the values of these counters with the timestamp corresponding to the five minute interval in which cflowd wrote the flow to the raw flow file. As such, the time reported in the graphs is some function of the flow end time, but may also reflect timeouts defined in the NetFlow implementation. Therefore, this timestamp is not necessarily the time when represented traffic was actually observed by the router.

It is possible for a FlowScan graph to report a quantity of traffic in a given five minute period that exceeds traffic actually forwarded by the router. On occasion, the quantity reported may even exceed the physical capacity of the physical link. The reason for these misleading reported results is twofold: First, FlowScan assumes that network traffic occurs within the five minute period in which data was written to the raw flow file. (Without this assumption, no totals can accumulate, and FlowScan can't plot values against time.) Secondly, while NetFlow flows contain start and end time information for the packets in a given flow, they do not indicate the distribution of packet delivery within that time range. Therefore, even if FlowScan attempts to record the real time at which traffic was observed, accuracy can not be improved. Experience shows that the effect of this time granularity inaccuracy is negligible as data is aggregated into coarser grained time samples.

Long-term Analysis

For long-term analysis, one specifies the number of hours over which to plot results. Increasing the hours significantly beyond the default (48 hours) becomes an exercise in customization of date and description annotations. Note also that often used daily averages may hide spurious abuse activity. Short-lived attacks can be severely minimized by daily averaging. Therefore, graphing over extended periods of time using daily averages tends to be more useful as an aid to capacity planning or even traffic shaping efforts.

Long-term FlowScan Graph (Daily Averages Over 550 days):

This long-term graph shows:

• The academic calendar dramatically influences traffic levels, but only with respect to traffic to and from ResNet, the campus residence halls network.
• There has been an increase in outbound ftp traffic from the Computer Sciences department within the past year.
• While outbound traffic levels consistently exceed inbound traffic levels, the degree by which outbound traffic exceeds inbound traffic is increasing.

Stateful Inspection of Flows

FlowScan's CampusIO report uses a number of heuristics to help identify and analyze "elusive" traffic (e.g., Napster, PASV mode ftp file transfers). These heuristics employ a method of stateful inspection similar to that used by many modern firewalls. Such firewalls track the state of an application session by observing information within a packet or series of packets, enabling the firewall to filter packets according to whether or not a session has been established and is still active. Passive inspection of either the packet header or the application payload gleans state information, enabling traffic identification and analysis.

In similar fashion, FlowScan attempts to track the state of an application session, or series of sessions, by observing the information within flows. Using flow-based stateful inspection, FlowScan maintains counters for flow attributes which would otherwise remain unidentified or misidentified. For example, the identification of Napster data based on a simple test of protocol and port number (say, TCP on port 6699), may lead to erroneous matches with applications such as PASV mode ftp that dynamically negotiate for unprivileged port numbers. FlowScan minimizes such errors by using a test based on stateful inspection.

Without the use of stateful inspection, a large percentage of our campus traffic would get labeled as "unknown". FlowScan performs a similar inspection to identify Real Media flows. Even with stateful inspection, more than 30% of our campus traffic remains unclassified.