Realtime Traffic Flow Measurement (RTFM) Architecture

This generalized architecture for measuring traffic flows was developed within the Internet Engineering Task Force (IETF), starting in 1991 with work done by the Internet Accounting Working Group. That group was re-chartered as the RTFM Working Group in 1996, and published five RFC's in October 1999. These are:

• 2721 Informational RTFM: Applicability Statement
• 2722 Informational Traffic Flow Measurement: Architecture
• 2723 Standards track Traffic Flow Measurement: Meter MIB
• 2723 Informational SRL: A Language for Describing Traffic Flows and Specifying Actions for Flow Groups
• 2724 Experimental RTF: New Attributes for Traffic Flow Measurement

This architecture provides for a distributed set of network entities called "meters," that measure traffic flows and build tables of flow data. "Meter readers" read flow data from these meters, while "managers" download configuration data to meters and meter readers.

More specifically, a NeTraMet meter is an SNMP agent which implements the RTFM Meter MIB. Any SNMP Network Manager can (provided it has SNMP read or write access) interact with the meter (e.g., by walking through the Meter MIB). In practice, the Meter MIB is tailored to its tasks of configuring the meter and reading flow data, which means that one really needs to use dedicated application programs (such as NeMaC and nifty, described below) when working with NeTraMet meters.

Traffic "flows," as defined by RTFM, are arbitrary collections of packets specified in terms of their end-point attributes. Furthermore, they are bi-directional, where each row in the meter's flow table has two sets of counters:

• "to" - counters for outgoing packets (Source to Destination)
• "from" - counters for returning packets (Destination to Source)

RTFM defines a set of about 40 attributes which can be used in describing packets on a network. The most important of these are "address attributes," which specify addresses at a particular network layer. For example, for IP packets, SourcePeerAddress and DestPeerAddress define the packets to and from IP addresses.

To configure an RTFM meter, a user must first create a "ruleset." Specified in the ruleset language, SRL, it indicates:

• which flows are to be "counted," causing a row to be added to the meter's flow table
• which end-point is considered to be the flow's source
• the level of detail required for each flow (details follow)

When a meter sees a packet, it extracts from it the values of all the RTFM attributes, then executes the SRL ruleset program. If the packet is to be counted, the program will execute a "count" statement, in which case the packet is said to have been "matched." If the packet is not matched, the meter will rerun the program, but this time with the packet's Source and Destination attributes interchanged. (This is how the packet's direction within a flow is determined).

Here are a few examples to make this more clear:

Example 1: Save Only IPv4 Information

Use the following two rules:

  if SourcePeerType == IPv4 save;
  else ignore;

If the packet's network layer protocol is IPv4, then the meter will save that information and continue processing. All non-IPv4 packets will be ignored.

Example 2: Count Flows Originating From a Specific Address

It is often useful to monitor a specific list of IP addresses. For example, to count flows originating within UCSD, use the following rules:

  if SourcePeerAddress == (
    128.54/16, 132.239/16, 132.249/16, 137.110/16,
    192.135.237/24, 192.135.238/24, 192.172.226/24, 199.105.0/26
    )
      {
      ...
      count;
      }

Here, the "if" statement tests whether the packet's SourcePeerAddress (its IP address) belongs to one of the networks within UCSD. The networks in the list above are written in CIDR notation (e.g. 132.239/16 means 132.239.0.0 with a 16-bit network mask (255.255.0.0). If the packet's source address falls within one of the listed networks, then the statements within the braces will be executed, causing the packet to be counted. If, on the other hand, the above test fails, the meter will retry the match while interchanging source and destination. Therefore, this statement specifies which address is the flow's source as well as deciding whether the packet should be counted.

Example 3: Specify Level of Detail to be Saved

Examples 1 and 2 specify only one flow. If we want the meter to subdivide that flow into smaller sub-flows, we must specify the detail to be used in creating flow table entries. This is done using "save" statements, as follows:

  if SourcePeerAddress == (
    128.54/16, 132.239/16, 132.249/16, 137.110/16,
    192.135.237/24, 192.135.238/24, 192.172.226/24, 199.105.0/26
    (
      {
      save SourcePeerAddress;  # IP addresses
      save Dest PeerAddress;
      save SourceTransType;  # IP protocol number
      if SourceTransType == TCP || SourceTransType == UDP {
    save SourceTransAddress; 
    save DestTransAddress;
    }
      count;
      }

The five "save" statements in this example will produce separate flows (entries in the meter's flow table) for every distinct 5-tuple of IP addresses, protocol and port numbers seen by the meter. Note that the Transaddresses (IP port numbers) are only saved for TCP and UDP flows, other IP protocols don't use port numbers.

Flows for 5-tuples like this are commonly referred to as "micro-flows". Within the RTFM architecture we refer to them as "streams", since they are specified by the attributes which the TCP/IP stack uses to identify each separate (bi-directional) connection.