libcoral - a C API for CoralReef

Introduction
- Compiling C and C++
The API

Introduction

Libcoral provides a C API for reading network interface devices and several different trace file formats, and for writing trace files. The installed Coral/lib/example directory contains an example application and Makefile that will help you get started writing your own code. Other realistic examples of libcoral use can be found in apps/utils/*.c in the distribution.

Libcoral operates on "sources", which include several types of live network interfaces/devices, and several types of tracefiles containing data recorded from one or more network interfaces/devices. The naming scheme for sources is described in the command usage document.

A typical program using libcoral has this basic structure:

  configure libcoral and one or more sources
  open sources
  start sources
  while (read block, cell or packet != NULL) {
      process the data we just read
      if (stop condition occurs) {
	  stop sources
      }
  }
  close sources

In the descriptions below, there are several pairs of functions with the same base name, with or without an "_all" suffix. The function suffixed with "_all" operates on all sources, and the other one operates on a single source indicated by the src parameter.

Compiling C and C++

All C/C++ programs built on CoralReef should include these headers:

#include <stdlib.h>
#include <stdio.h>
#include <sys/param.h>
#include <libcoral.h>

Note that <libcoral.h> will define fixed-width integer types (like uint32_t) if your system does not, since they are needed by many libcoral structures.

The following optional headers are also available:

#include <coral-config.h> /* configuration information used by coral */
#include <crl_byteorder.h> /* byte order macros */

CoralReef programs should link with -lcoral, -lpcap (if libcoral was compiled with pcap support), -lz (if libcoral was compiled with zlib support), and possibly -lstdc++ (if needed for Crypto-PAn), in that order. The installed Coral/lib/example/Makefile will link with the correct set of libraries.

The API

Command-line Configuration

The following functions read coral commands from the command line or config files. The coral commands are described in the command usage document. These functions should only be called before coral_open() or coral_open_all().

You should call coral_set_api() before calling these functions to enable certain configuration commands. Also, you should set the defaults for configurable values with the coral_set functions before calling any of these functions, and (if needed) get the value after. This way, the option and its default value will appear in the coral_usage() message, and the user can override the defaults with command line options. If your application does not use a particular configurable value, you should leave it set to its initial value, so it will not appear in the coral_usage() message, and the coral_config functions will not allow it to be changed on the command line.

int coral_config_arguments(int argc, char *argv[]);

This function processes "-C" options and sourcename arguments contained in argv. Argv is a pointer to an array of strings containing the command line; argc is the number of strings in argv. For a description of Coral options, see the Command Usage documentation. A CoralReef source is created (with coral_new_source()) from each sourcename argument. If an option syntax error is encountered, this function prints a usage message. If other arguments are desired, use coral_config_options() instead and parse the arguments yourself; if other options are desired, parse the options and arguments yourself and call coral_config_command() for each "-C" option. Returns a non-negative integer for success, -1 for failure.

To limit the number of sources that will be allowed, call coral_set_max_sources() before calling this function.

For an example, see the end of this section.

int coral_config_options(int argc, char *argv[], const char *argmsg);

This function is like coral_config_arguments(), except it processes only the options, not the non-option arguments. If argmsg is NULL, this function assumes that arguments (other than Coral options) are not allowed, and considers it an error if other arguments are found; otherwise, other arguments (but not other options) are allowed. If successful, this function returns the index of the first non-option argument (so it would return 1 if there were no options, since argv[0] is the application name). It is the programmer's responsibility to handle the remaining command line arguments. If an error is encountered, this function returns -1, and prints a usage message with coral_usage(argv[0], "%s", argmsg). If other options are desired, use coral_config_command() instead. Returns a non-negative integer for success, -1 for failure.

int coral_config_command(const char *command);

Processes a config command. This function should be used instead of coral_config_options() or coral_config_arguments() if you need to accept your own options in addition to CoralReef options, or you want to accept CoralReef commands from inputs other than the command line. Returns 0 for success, -1 for failure.

int coral_config_file(const char *filename);

Process CoralReef commands in file filename. This is not usually needed if coral_config_arguments(), coral_config_options(), or coral_config_command() is used.

int coral_usage(const char *appname, const char *argmsg, ...);

Prints a message describing the command line syntax of a CoralReef application named appname. If argmsg is not NULL, it is used as a printf-style format for the arguments that follow, and the formatted result is appended to the syntax line. This is followed by descriptions of the CoralReef (-C) command line options.

Some example values for argmsg:

for an application that takes CoralReef options and no arguments:: NULL
for an application that takes CoralReef options and an optional argument: "[<argument>]"
for an application that takes -f, -bN, and CoralReef options, and one or more source arguments:: "[-f] [-b<N>] <source>..."

Examples

All these examples assume you will be using the packet API; you should change the argument of coral_set_api() as needed.

If only CoralReef options and sourcename arguments are needed, this simple code will do:

    coral_set_api(CORAL_API_PKT);
    if (coral_config_arguments(argc, argv) < 0)
        exit(-1);

If other options are desired, you must parse the options and arguments yourself, like this:

    int opt, foo_flag = 0, bar = 42;
    extern char *optarg;
    extern int optind;

    coral_set_api(CORAL_API_PKT);

    while ((opt = getopt(argc, argv, "C:fb:")) != -1) {
        switch (opt) {
        case 'C':
            if (coral_config_command(optarg) < 0)
                exit(-1);
            break;
        case 'f':
            foo_flag = 1;
            break;
        case 'b':
            bar = atoi(optarg);
            break;
        default:
            coral_usage(argv[0], "[-f] [-b<N>] <source>...\n"
		"-f      foo flag\n"
		"-b<n>   bar value (default %d)\n",
		bar);
            exit(-1);
        }
    }

    while (optind < argc) {
        if (!coral_new_source(argv[optind]))
            exit(-1);
        optind++;
    }

If a bad option is given, this code will print a message like this to stderr, followed by descriptions of the CoralReef commands that are allowed for the configured API:

    Usage: app [-C <coral_command>]... [-f] [-b<N>] <source>...

Global Configuration

The following functions provide a way for the programmer to set global configuration values, including default values for the configuration of sources that have not yet been defined. The coral_set functions can be called before the coral_config functions to set default values that can be overridden on the command line or config file, and will be displayed in the coral_usage message. If the integer configuration values are set to a negative value, they will not be displayed in the coral_usage message. The coral_set functions can also be called after the coral_config functions so their values can't be overridden. They must not be called after sources are opened. The coral_set functions return 0 if successful, or -1 for failure; the coral_get functions return the value of the requested datum.

void coral_config_defaults(void);

Resets libcoral configuration to its default state. Should not be called while sources are open. The affected configuration values are: max_sources, options, duration, interval, dropinterval, mintime, maxtime, errfile, iomode, comment, gzip level, pcap filter, api, max_blks, max_cells, max_pkts, bandwidth, physical protocol, datalink protocol, firmware.

int coral_set_api(int api);

The api parameter to coral_set_api() must be CORAL_API_BLOCK, CORAL_API_CELL, or CORAL_API_PKT, optionally bitwise ORed with CORAL_API_WRITE. It initializes libcoral for use with the corresponding APIs, and should be called before any coral config, reading, or writing function.

Setting the APIs enables command line and configuration file options and functions that are specific to those APIs. Libcoral will generate an error if any disallowed option or function is attempted.

The api parameter should contain exactly one of the following constants:

API	allowed configuration commands	allowed functions

`CORAL_API_BLOCK`	`blocks`
`CORAL_API_CELL`	`cells` `blocks` `deny`
`CORAL_API_PKT`	`packets` `filter` `deny`	`coral_add_pcap_filter()`

Commands and functions not listed under any API are valid for all APIs.

The api parameter may optionally be bitwise-ORed with this constant, if coral_write or a rotating output file will be used:

API	allowed configuration commands	allowed functions

`CORAL_API_WRITE`	`gzip`	`coral_set_gzip()`

Example: coral_set_api(CORAL_API_PKT | CORAL_API_WRITE);

int coral_set_max_sources(int n); int coral_get_max_sources(void);

coral_set_max_sources() sets the maximum number of allowed Coral sources to n. It should be called before coral_config_arguments(), coral_config_options(), coral_config_command(), and coral_new_source() to restrict them. It returns 0 if successful, or -1 if n is negative or greater than 16. coral_get_max_sources() returns the maximum number of allowed Coral sources. The default max_sources value is 16.

int coral_set_options(int off, int on); int coral_get_options(void);

coral_set_options() disables the options in the bitmask off and enables the options in the bitmask on. coral_get_options() returns a bitmask of currently enabled options. The options are:

CORAL_OPT_SORT_TIME (default: off)

Makes coral_read_cell_all() and coral_read_pkt() read in timestamp order. For this option to make sense, one of the following must be true: all interfaces have start epochs and were started at the same time, or all interfaces have unix epochs, or the CORAL_OPT_NORMALIZE_TIME option is on. If none of these conditions is true, libcoral will automatically enable the CORAL_OPT_NORMALIZE_TIME option. If your application requires time sorting, set this option before the command-line coral_config functions so "sort" will not appear in the coral_usage message.

CORAL_OPT_ALIGNINT (default: off)

Aligns end of first interval to a multiple of the interval length.

CORAL_OPT_PARTIAL_PKT (default: on)

Allows coral_read_pkt() and coral_read_pkts() to handle partial packets.

CORAL_OPT_FATM_RAW_TIME (default: off)

Do not attempt to automatically correct the delayed firmware clock increment when the hardware clock wraps in the fatm card or skip cells prior to a clock reset in the first block. This can be used to improve performance if cell/packet timestamps will not be needed.

CORAL_OPT_POINT_RAW_TIME (default: off)

Do not attempt to automatically skip cells prior to a clock reset in the first block. This can be used to improve performance if cell/packet timestamps will not be needed.

CORAL_OPT_RAW_TIME

Equivalent to (CORAL_OPT_FATM_RAW_TIME | CORAL_OPT_POINT_RAW_TIME).

CORAL_OPT_NORMALIZE_TIME (default: on)

Make the result of the coral_read_clock relative to the unix epoch (1970-01-01 00:00:00 UTC); otherwise, the result will have some interface-dependent epoch. Normalizing is necessary when comparing times from multiple interfaces (this includes using CORAL_OPT_SORT_TIME) with different default epochs. But normalizing may introduce inaccuracy if the interfaces' hosts' CPU clocks are inaccurate, and may introduce additional processing overhead. The default epochs for the various types of interfaces are:

fatm: time of reset (e.g., by coral_start_all());
point: time of reset (e.g., by coral_start_all());
dag: the unix epoch (of the CPU used to do the capture)
pcap: the unix epoch (of the CPU used to do the capture)
tsh: either the unix epoch or the time of reset, depending on the original source of data (libcoral can identify the epoch automatically)

Traces from interfaces that don't natively use the unix epoch can only be normalized if the capture time was recorded in the trace. NLANR and MCI .crl traces do not contain capture time, although their filenames sometimes do, and CoralReef can use that; .crl traces made with CoralReef prior to 3.2.1 contain capture time only to the nearest second. Traces in .tsh format do not explicitly contain the capture time, but if the data timestamps are relative to the unix epoch, libcoral will use the first timestamp as the capture time; otherwise, libcoral will infer a capture time from the filename if possible.

CORAL_OPT_NO_FILTER (default: off)

Do not allow a pcap filter to be set by the coral_config functions. This is useful for applications that need to define their own filters.

CORAL_OPT_IP_CKSUM (default: off)

Tells coral_fmt_get_payload() to print and verify IP checksums.

CORAL_OPT_IP_EXTHDR (default: off)

Tells coral_get_payload() to not skip IP extension headers.

CORAL_OPT_IGNORE_TIME_ERR (default: off)

By default, libcoral reading functions return an error when they encounter a timestamp error they can not correct in a source. With this option enabled, libcoral will continue reading normally, but beware that intervals and duration may not work correctly.

CORAL_OPT_NO_AUTOROTATE_ERRFILE (default: off)

By default, an error file whose name contains "%" is rotated at the beginning of intervals. With this option turned on, automatic rotation is disabled.

int coral_set_iomode(unsigned off, unsigned on, int first, int fixed);

This function sets the default I/O mode of all subsequently defined sources. (To set the I/O mode of sources that have already been defined, use coral_source_set_iomode().)

The arguments to this function are:

off: bitmask of I/O mode flags to disable.
on: bitmask of I/O mode flags to enable.
first: specifies how many bytes to capture from each PDU (layer 2 packet/frame). If this parameter is negative, the configured value will be unchanged. For ATM interfaces, first will be rounded up to a multiple of ATM cell payload size (48 bytes). (In versions prior to 3.2, first was interpreted as cells instead of bytes on ATM interfaces.)
fixed: flag that prevents user from modifying the iomode via the command line or config file (this should be used before the coral_config functions, in applications with a non-configurable iomode).

The I/O mode flags are:

flag	equivalent CLI iomode	description

CORAL_TX	tx	allow transmitting (not implemented)
CORAL_RX	rx	allow receiving
CORAL_RX_IDLE	idle	read all cells, including ATM idle cells
CORAL_RX_OAM	ctrl	read ATM OAM/RM cells
CORAL_RX_LAST	last	read last cell of each ATM AAL5 user PDU (packet)
CORAL_RX_USER_ALL	user	read all bytes of each packet or frame
CORAL_RX_ECHO	echo	retransmit received cells on devices which support it (always enabled on POINT cards)
CORAL_RX_UNKNOWN		for traces, read whatever cells are in the trace file; for devices, use default mode (i.e., rx first=48). Setting this flag clears all other settings.

This function returns 0 for success, -1 for failure. It only affects sources that are created after it is called.

Setting first overrides a previously set CORAL_RX_USER_ALL flag, and setting the CORAL_RX_USER_ALL flag overrides a previously set first value. Attempting to set both at once has undefined results.

The default I/O mode is CORAL_RX, and first==48. Currently, an attempt to explicitly use an I/O mode to read a trace that does not match the I/O mode used to record the trace will generate a warning, but all the data in the trace will still be read. In a future release, the read functions may filter trace files according to the I/O mode (or generate an error if that is not possible).

For a summary of the iomode options supported by the various interface types, see the command usage document.

const coral_io_mode_t *coral_get_iomode(void);

Return the default iomode as set by coral_set_iomode() or command line options.

int coral_set_errfile(FILE *file); int coral_set_errfilename(const char *filename); FILE *coral_get_errfile(void);

coral_set_errfile() and coral_set_errfilename() set the destination for future libcoral error and warning messages. file is an already opened stdio stream pointer, and filename is the name of a file that will be opened by libcoral. Any previously set error file will be closed only if it was set by name other than "-" (i.e., by coral_set_errfilename(), but not by coral_set_errfile()). If file or filename is NULL, stderr will be used. If filename is "-", stdout will be used. coral_get_errfile() returns the current message destination. The default message destination is stderr.

If the name of the error file contains "%", the error file will be rotatable and the actual filename will be generated as described in coral_rf_open. Unlike normal rotfiles, a rotating error file is always writable by coral_diag and libcoral; it does not need to be started. A rotatable error file will rotate automatically at the beginning of each interval of coral_read_pkt or coral_read_pkts, unless then CORAL_OPT_NO_AUTOROTATE_ERRFILE option is set. In any case, coral_rotate_errfile can be called to rotate the error file explicitly. If rotation fails, the error file resets to stderr.

int coral_set_verbosity(int verbosity); int coral_get_verbosity(void);

Set or get the verbosity level of libcoral and coral_diag(). If verbosity==0, only errors are printed; if verbosity==1 errors and warnings are printed. Meanings for values greater than 1 may be assigned by the programmer for use with coral_diag(). The default verbosity is 1. To completely disable coral messages, set the verbosity to -1.

int coral_set_max_pkts(uint64_t n); int coral_set_max_cells(uint64_t n); int coral_set_max_blks(uint64_t n); uint64_t coral_get_max_pkts(void); uint64_t coral_get_max_cells(void); uint64_t coral_get_max_blks(void);

These functions get or set the maximum record count that will be read by the reading APIs. When the limit is reached, reading functions will indicate EOF. If set to zero, there is no limit. All three counts are zero by default.

count	affected APIs

max_pkts	packet
max_cells	cell
max_blks	block and cell

int coral_set_comment(const char *comment); const char *coral_get_comment(void);

Set or get the default comment that will be written in the file header by the coral_write functions. The default comment may be overridden by coral_write_set_comment().

int coral_set_duration(long duration); long coral_get_duration(void);

Set or get the duration value.

If the duration value is greater than 0, it determines how long libcoral will read from sources after they are started; after the duration has expired, the reading function will indicate EOF. Libcoral uses both real time and cell timestamps to test duration, so it will work with tracefiles and devices receiving live traffic.

With the default duration of -1, the coral_config functions will not allow duration to be set. To have your application not use duration by default, but still let the user override it, set it to 0 before calling the coral_config functions.

Before version 3.3.0, libcoral did not test duration when reading; the programmer was expected to handle it himself. Now, the programmer should do nothing other than enable it, and let libcoral's reading functions handle the duration.

Libcoral does not use SIGALRM to implement duration.

int coral_set_interval(const struct timeval *interval); int coral_get_interval(struct timeval *interval);

Set or get the interval value. If the interval has not yet been set, coral_get_interval() returns -1 and fills in *interval with { -1, -1 }; otherwise, it returns 0 and fills in *interval with the nonnegative interval value.

The interval value is never automatically used by libcoral; it just provides a convenient and consistent way to let the user set an interval on the command line or in a config file. The programmer may use the interval by passing the result of coral_get_interval() in as a parameter to coral_read_pkt_init() or coral_read_cell_i(). Libcoral uses cell/packet timestamps, not real time, to test for intervals. Therefore the intervals it reports will be correct for both tracefiles and devices, although the act of reporting may be delayed because of internal buffering.

With the default interval of -1, the coral_config functions will not allow interval to be set. To have your application not use interval by default, but still let the user override it, set it to 0.0 before calling the coral_config functions.

Libcoral does not use SIGALRM to implement interval.

Source Configuration

These functions define and configure sources. They may only be called before opening sources.

coral_source_t *coral_new_source(const char *sourcename);

Create a new coral source from sourcename. Sources are described in the Command usage document. Returns source pointer if successful, or NULL for failure. This function is useful for creating a source directly from the program (instead of through command line and config file with the coral_config functions).

coral_source_t *coral_new_fdsource(int fd, const char *sourcename);

Like coral_new_source, except that it uses the already-open file descriptor fd instead of a filename. If sourcename is not NULL, coral_open() will examine it for a prefix or suffix to determine how to interpret the data read from fd; otherwise, coral_open() will assume data from fd contains a CoralReef (crl) trace. This function is useful for creating a source directly from a file descriptor in the program; for example, a socket connected to another process that used coral_write_fdopen().

Mixing CoralReef operations with other operations on descriptor fd may have unpredictable effects.

int coral_source_set_iomode(coral_source_t *src, unsigned off, unsigned on, int first); int coral_source_set_iomode_all(unsigned off, unsigned on, int first);

These functions modify the I/O mode of the indicated already defined sources. The off, on, and first arguments are identical to those of coral_set_iomode(). These functions return 0 for success, -1 for failure.

To set the default I/O mode of sources that have not yet been defined, use coral_set_iomode().

const coral_io_mode_t *coral_source_get_iomode(const coral_source_t *src); const coral_io_mode_t *coral_iface_get_iomode(const coral_iface_t *iface);

Return the iomode of src or iface, respectively. Note that for live interfaces, it is the iomode of the interface that determines how data is read; the iomode of the source is just a default that an interface may inherit. For file interfaces, the iomode of the interface is that of the original device, and the iomode of the source has little meaning.

int coral_format_iomode(char *buffer, const coral_io_mode_t *iomode);

Prints a human-readable representation of iomode into buffer. buffer must be at least CORAL_FORMAT_IOMODE_LEN bytes long. Returns the length of the formatted string.

int coral_source_set_firmware(coral_source_t *src, const char *firmware); int coral_source_set_firmware_all(const char *firmware);

Set the location of the firmware file of source src, or of all sources. The firmware will be loaded when the source is opened. Returns 0 for success, -1 for failure.

Opening

Sources must be opened and started before they can be read. When they are no longer needed, they may be stopped and should be closed. Note that failing to close a DAG card source may leave it in an unusable state (/proc/dag will report the device is "Busy"); the card must be re-initialized before CoralReef can read from it again.

int coral_open(coral_source_t *src); int coral_open_all(void);

Open the indicated coral sources that have been created by coral_config_arguments(), coral_config_options(), coral_config_command(), and coral_new_source(), but do not start them. If successful, these functions return the number of interfaces opened. If unsuccessful, these functions return -1 and close the source which failed. Note that a single tracefile source may contain multiple interfaces.

int coral_start(coral_source_t *src); int coral_start_all(void);

Start capture on interface(s) of the indicated coral sources which have been opened with coral_open() or coral_open_all(). Additionally, coral_start_all() resets the clocks of all open interfaces that allow it, to synchronize them. A source can not be read until it is started with one of these functions. (Prior to version 3.3, it was not strictly necessary to start tracefile sources.)

int coral_stop_all(void); int coral_stop(coral_source_t *src);

Stop capturing on all interfaces of the indicated device sources. Returns 0 for success, -1 for failure. After a source is stopped, it may still contain buffered data, which can be read until the reading function returns an EOF indication. These functions are not safe to call asynchronously (e.g., from a signal handler).

Packet Reading

The packet reading functions may be used with any CoralReef interface that has been started. All time is measured against packet timestamps, so it works even for trace files that aren't in real time. If pcap filtering is enabled, packets that do not match the filter are discarded. If anonymization is enabled, IPv4 addresses in packets will be anonymized, and non-IPv4 packets are discarded. See crl_print_pkt.c for a good example of using the packet API.

Note that the data in packets returned by the packet API is not guaranteed to have any particular alignment. The functions crl_nptohl() and crl_nptohs() can be used to convert data to host byte order and handle the alignment at the same time.

Single packet reading

int coral_read_pkt_init(coral_source_t *src, coral_iface_t *iface, struct timeval *interval);

Prepare to read packets from an open src or iface with coral_read_pkt(). If iface is not NULL, packets will be read from iface; otherwise, if src is not NULL, packets will be read from all interfaces of src; otherwise, packets will be read from all interfaces of all open sources. There is no restriction on reading multiple pcap sources or mixing pcap and OCx sources, as there was prior to version 3.4. If iface belongs to a source with multiple interfaces, data from the other interfaces of that source will be discarded. Packets can be read from only one set of sources at a time; calling coral_read_pkt_init() invalidates any sources that were previously initialized for packet reading. Interval is the amount of time coral_read_pkt() will wait between returning statistical information (measured against packet timestamps).

Because coral_read_pkt() must be able to compare timestamps, coral_read_pkt_init() will automatically enable the CORAL_OPT_NORMALIZE_TIME option if necessary.

coral_read_pkt_init() returns 0 if successful, or -1 if an error occurs.

coral_iface_t *coral_read_pkt(coral_pkt_result_t *pkt_result, coral_interval_result_t *interval_result);

Read a packet from one of the interfaces which have been initialized with coral_read_pkt_init(). If a packet filter has been set, only packets that match the filter will be read. Packets read are sorted by time only if the CORAL_OPT_SORT_TIME option was set when coral_read_pkt_init() is called (prior to version 3.6, packets were always sorted by time, regardless of the setting of the CORAL_OPT_SORT_TIME option). Even if sorting is disabled, packets will be returned in the correct interval, and no packets will be returned after the duration has expired. If the CORAL_OPT_PARTIAL_PKT option is turned off, only complete packets will be read (but truncated packets are always counted by the truncated field of coral_pkt_stats_t).

pkt_result and interval_result are pointers to structures (allocated by the programmer) that will be filled in by the call. For success, coral_read_pkt() returns a pointer to the interface which was read, and the contents of the structures indicate what happened:

if (pkt_result->packet != NULL) {: A packet was read; information about the packet is contained in pkt_result.
pkt_result->packet will point to a coral_pkt_buffer_t containing the link layer (LLC/SNAP, ethernet, etc) PDU. On ATM interfaces, pkt_result->header will point to a coral_pkt_buffer_t containing the first 4 bytes of the ATM header of the last captured cell of the packet (in network byte order); pkt_result->trailer will point to a coral_pkt_buffer_t containing the AAL5 trailer (in network byte order), if available; and pkt_result->subiface will contain the ATM vpvc (in host byte order) from which the packet was read. On IEEE 802.3/Ethernet interfaces carrying IEEE 802.1Q VLAN traffic, pkt_result->subiface will be the VLAN ID. On other types of interfaces, pkt_result->header and pkt_result->trailer will be NULL, and pkt_result->subiface will be 0.
} else if (interval_result->stats == NULL) {: An interval has begun at time interval_result->begin. (interval_result->end is not valid.)
} else {: The interval begun at time interval_result->begin and ending at interval_result->end has ended. Statistics for the interval are contained in interval_result->stats. All packets read after this point will have timestamps greater than interval_result->end, even if the CORAL_OPT_SORT_TIME option is not enabled.
}

If the interval specified in coral_read_pkt_init() was NULL or 0, interval_result may be NULL, and coral_read_pkt() will never return with pkt_result->packet == NULL. The beginning of an interval is not aligned to a multiple of the interval size. When reading from a live interface, internal buffering may cause reporting of the end of the interval to be delayed, but the data reported will cover exactly the requested interval. (In versions 3.2.x, the reported interval included "quiet" (trafficless) time after the end of the requested time. In versions 3.1 and earlier, the data reported included packets past the end of the interval that fell in the same internal buffer as the end of the interval.) Interval comparisons are made against packet timestamps, not a real clock, so the same results will be produced by reading a live interface or reading a trace taken on that interface in the same period.

The protocol of pkt_result->packet is determined by the interface type. For ATM interfaces, the protocol is determined by the virtual channel and proto configuration rules as by coral_proto_rule(). Note that if the configuration is incorrect, the packet buffer will not make sense. For TSH interfaces, the protocol for a valid packet is always CORAL_NETPROTO_IPv4, and IP options are always zero; the protocol of an invalid packet is CORAL_PROTO_UNKNOWN. Packets read from raw IP sources (i.e., pcap DLT_RAW) will have their protocol set to CORAL_NETPROTO_IPv4 or CORAL_NETPROTO_IPv6 if they contain at least 1 byte of data; otherwise, they will have protocol CORAL_NETPROTO_RAW_IP.

If coral_cell_block_hook points to a programmer-defined function, it will be called when a new block of ATM cells is read.

To indicate EOF, a stopped device, or expired duration, coral_read_pkt() returns NULL with errno == 0. For an error, coral_read_pkt() returns NULL with errno set to indicate the type of error. (Certain types of errors that were reported as EOF in versions prior 3.5 are now correctly reported as errors.)

If coral_get_verbosity() >= 1, then at the end of every interval, if there were any packet errors (nonzero fields in interval_result->stats) during the interval, coral_read_pkt() prints warnings to the error file. If intervals were not used, the warnings are printed only at EOF, covering the entire duration.

Typical packet reading code looks like this:

{
    coral_iface_t *iface;
    coral_pkt_result_t pkt_result;
    coral_interval_result_t interval_result;
    struct timeval interval = {300,0};
    ...
    coral_read_pkt_init(NULL, NULL, &interval);
    while (1) {
	iface = coral_read_pkt(&pkt_result, &interval_result);
	if (!iface) {
	    /* error */
	    break;
	}
	if (pkt_result.packet) {
	    /* got packet */
	    ...
        } else if (!interval_result.stats) {
	    /* beginning of interval */
	    ...
	} else {
	    /* end of interval */
	    ...
	}
    }
}

Callback loop packet reading

int coral_read_pkts(coral_source_t *src, coral_iface_t *iface, coral_pkt_handler pkthandler, coral_pre_interval_handler preinthandler, coral_post_interval_handler postinthandler, struct timeval *interval, void *mydata);

Reads packets from one or more interfaces, and calls a handler function for each packet read and for the beginning and end of each interval. Src, iface, and interval are interpreted as in coral_read_pkt_init(). coral_read_pkts() repeatedly reads link level packets (using coral_read_pkt()) until (coral_pkt_done != 0) or EOF is reached on all interfaces.

Pkthandler is a pointer to a function that will be called for each packet received.

Preinthandler and postinthandler are pointers to functions that will be called before and after each interval of *interval seconds in which packets were received. Preinthandler and postinthandler are not called if they are NULL, interval is NULL, or *interval is less than or equal to 0.0.

Mydata will be passed as a parameter to the handler functions, but otherwise ignored; the programmer may use it as needed.

Postinthandler may set coral_pkt_done to nonzero to make coral_read_pkts() stop looping and return.

coral_read_pkts() returns 0 if it reaches EOF with no errors; or -1 if there is an error, with errno set to indicate the type of error. (Certain types of errors that were reported as EOF in versions prior to 3.5 are now correctly reported as errors.)

typedef void (*coral_pkt_handler)(coral_iface_t *iface, const coral_timestamp_t *timestamp, void *mydata, coral_pkt_buffer_t *packet, coral_pkt_buffer_t *header, coral_pkt_buffer_t *trailer);

coral_pkt_handler is the type of the function called each time coral_read_pkts() reads a packet. Parameters:

iface: a pointer to the interface from which the packet was read.
timestamp: the time at which the packet was read (usable with the coral_read_clock functions).
mydata: the mydata parameter that was passed to coral_read_pkts().
packet: a pointer to a coral_pkt_buffer_t containing a link level packet, not including a trailer.
header: a pointer to a coral_pkt_buffer_t containing the header used at a next lower layer, or NULL. Currently, this is only used for the ATM header of the last received cell in the packet; the ATM header will be in network byte order (in version 3.1, this was in host byte order).
trailer: a pointer to a coral_pkt_buffer_t containing the trailer used at a next lower layer, if the lower layer uses trailers and the packet was not truncated; otherwise, NULL. Currently, this is only used for the ATM AAL5 trailer.

typedef void (*coral_pre_interval_handler)(coral_iface_t *iface, const struct timeval *begin, void *mydata); typedef void (*coral_post_interval_handler)(coral_iface_t *iface, const struct timeval *begin, const struct timeval *end, void *mydata, const coral_pkt_stats_t *stats);

coral_pre_interval_handler and coral_post_interval_handler are the types of the functions called before and after (respectively) each interval of coral_read_pkts(). Parameters:

iface: the interface being read.
begin: the beginning of the interval.
mydata: the mydata parameter that was passed to coral_read_pkts()
stats: a pointer to a structure containing statistics about the interval, summed across all interfaces. (To get statistics for an individual interface, use coral_get_iface_stats().)

Pcap packet filtering

If libcoral was compiled with libpcap, these functions can be used to set pcap filter expressions. Pcap filters can be used with any type of interface, not just BPF devices. The filter will be executed when it is most efficient: in the kernel if the interface is a BPF device, or before AAL5 reassembly (if possible) if the interface is an ATM Coral device or file. The packet reading functions will return only packets which match the filter.

int coral_add_pcap_filter(const char *expr); int coral_add_pcap_prefilter(const char *expr); int coral_add_pcap_ipfilter(const char *expr);

If expr is not NULL, coral_add_pcap_filter() sets expr as a global pcap filter expression. If a filter expression was already set (either by this function or by coral_config "filter" commands), the old and new expressions are joined as "(old_expr) and (expr)". The expression will be compiled and set on every source that is subsequently opened; it has no effect on sources that are already open.

If expr is NULL, coral_add_pcap_filter() clears any previously set global pcap filter expression.

These functions will not work until coral_set_api(CORAL_API_PKT) is called. This function returns 0 if successful, or -1 with errno set if there is an error.

coral_add_pcap_prefilter() is similar to coral_add_pcap_filter(), except it sets a "prefilter", which is used only on ATM interfaces, on the first ATM cell, before AAL5 reassembly. A prefilter can be used to avoid unnecessary reassembly, but only if the first ATM cell contains enough information to apply the filter. It is up to the user to make sure the prefilter does not need information that does not fit in the first cell.

coral_add_pcap_ipfilter() is similar to coral_add_pcap_filter(), except that for each packet CoralReef first skips past layer 2 to find an IP packet. If no IP packet is found, the packet is discarded; otherwise, the filter is tested starting at the IP header. This is useful when packets contain IP encapsulated in some layer 2 protocol that pcap filters can not parse, but CoralReef can, such as MPLS.

See also the section on -Cfilter in the command usage document.

const char *coral_get_pcap_filter(void); const char *coral_get_pcap_prefilter(void); const char *coral_get_pcap_ipfilter(void);

Returns the text of the global pcap filter expression previously set by coral_add_pcap_filter() or coral_add_pcap_prefilter() and/or by coral_config "filter" or "prefilter" commands. Returns NULL if no filter expression has been set.

#include <pcap.h> int coral_pcap_compile(coral_iface_t *iface, struct bpf_program *fp, char *str, int optimize, uint32_t netmask); int coral_pcap_setfilter(coral_iface_t *iface, struct bpf_program *fp); int coral_pcap_setprefilter(coral_iface_t *iface, struct bpf_program *fp);

If you need finer control over pcap filtering than coral_add_pcap_filter() and coral_add_pcap_prefilter() provide, you can use these functions. These allow you to recompile and reset filters at any time (even on interfaces that are already open), and to assign different filters to different interfaces.

These functions are the same as pcap_compile() and pcap_setfilter(), respectively, except that they apply to a coral_iface_t instead of a pcap_t. They are used to compile a tcpdump expression to a BPF program and install it onto a Coral interface. Note that *fp must remain in valid memory until iface is closed or a new BPF program is installed on iface.

To avoid confusion when using these functions, you should set the CORAL_OPT_NO_FILTER option before calling the coral_config functions to prevent the user from specifying a useless "filter" command. Alternatively, you could allow the user to use "filter" commands, use coral_get_pcap_filter() to get it so you can combine it with yours, and coral_add_pcap_filter(NULL) to clear the global expression so it won't be applied by coral_open().

See the pcap and tcpdump documentation for more information.

IPv4 address anonymization

CoralReef can anonymize IPv4 address by zeroing out bits, or by using Crypto-PAn.1.0 which preserves prefix relationships. Anonymization can be enabled on the command line for any application that uses the packet API, or invoked explicitly via the libcoral API described below.

coral_anon_t *coral_get_anonymizer();

Return a pointer to a copy of the global coral_anon_t object created by the "anonymize" configuration command.

int coral_set_anonymizer(coral_anon_t *anon);

Set a global coral_anon_t object to be used by the packet API. If anon is NULL, anonymization in the packet API will be disabled. Anonymization is applied after pcap filtering.

In applications that do a lot of aggregation and need to output anonymized results, it can be more efficient to anonymize after aggregation. To do this, use coral_get_anonymizer() to get the coral_anon_t that was configured on the command line, use coral_set_anonymizer(NULL) to disable automatic anonymization, and call coral_anonymize() or coral_pkt_anonymize() on each address or packet in the aggregated results before outputting them.

uint32_t coral_anonymize(coral_anon_t *anon, uint32_t addr);

Returns an anonymized, host byte order copy of addr, which is also in host byte order. anon must be a pointer to a coral_anon_t object returned by coral_get_PAnonymizer().

Bug: this function does not know if addr is a source or a destination address, so it anonymizes unconditionally, ignoring the src and dst parameters of the "anonymize" configuration command.

Example:

    struct in_addr old, new;
    ...
    new.s_addr = coral_anonymize(anon, ntohl(old.s_addr));
    new.s_addr = htonl(new.s_addr);

int coral_pkt_anonymize(coral_pkt_buffer_t *pkt);

Anonymizes source and destination addresses within IPv4 headers of the packet pointed to by pkt, using the global coral_anon_t set by the "anonymize" configuration command. If the packet contains an IPv4 packet which directly or indirectly encapsulates other IPv4 packets using any protocol that libcoral understands (e.g., IPIP, or the original IPv4 packet inside an ICMP error message), addresses in the encapsulated IPv4 headers are also anonymized. If an address is truncated due to insufficient pkt->caplen, the partial address is still anonymized. IP, TCP, and UDP checksums are incrementally recaclulated to reflect the changed addresses (this is new in version 3.7.4).

If this function does not find a native IPv4 packet within pkt, and the "keep" anonymization parameter was not set, it returns 0; otherwise it returns 1. Thus the caller should keep any packet for which this function returns 1, and discard or apply its own additional criteria to other packets.

By default, IPv6 headers are truncated at 8 bytes, and other Layer 3 non-IPv4 headers are truncated at 0 bytes. These headers may appear because "keep" was set, or because they were encapsulated in IPv4. Truncation is new in version 3.8.0. Truncation is prevented if the "notrunc" anonymization parameter was set.

This function does not anonymize non-IPv4 addresses (e.g., Ethernet, IPv6), nor IPv4 addresses in non-IPv4 headers (e.g., ARP).

Packet reading example

static int count_and_print_pkt(coral_iface_t *iface,
    const coral_timestamp_t *timestamp, void *mydata,
    coral_pkt_buffer_t *buffer, coral_pkt_buffer_t *header,
    coral_pkt_buffer_t *trailer)
{
    long *countp = mydata;

    ++(*countp);
    if (buffer->caplen == buffer->totlen)  /* complete capture? */
        printf("bytes: %d\n", buffer->caplen);
    else
        printf("bytes: %d (of %d)\n", buffer->caplen, buffer->totlen);
    coral_fprint_pkt(stdout, iface, timestamp, 7, buffer, header, trailer);
}

int main()
{
    long count;
    ...
    count = 0;
    coral_read_pkts(src, NULL, count_and_print_pkt, NULL, NULL, 0, &count);
    printf("received %ld packets.\n", count);
    ...
}

Cell/Block Reading

General Description

The block and cell reading functions may be used only with ATM interfaces that have been started. The cell functions are more general than the block functions. For working with layers 3 and above, the packet reading functions are usually better.

After any of these reading functions are successful, *cellp will point to a single cell (for the cell functions) or array of cells (for the block functions); and if binfop is not NULL, *binfop will point to a block info structure describing the block to which the cell or cells belong. All structures will be in network byte order. (In version 3.0, cells read from FATM devices had ATM headers in little endian order.) binfop may be passed a value of NULL if the block info will not be needed.

If timeout is NULL or omitted, these functions will wait until data are available or an interrupt occurs. But if timeout is not NULL, these functions will also return after the amount of time specified in timeout even if there are no data.

If coral_cell_block_hook is not NULL, the function to which it points will be called when a new block of ATM cells is read by any of the cell reading functions.

The deny protocol rules are applied by the cell reading functions to filter the cells and determine their protocol. They are not applied by the block reading functions. In either case, no protocol information is given; use coral_proto_rule to get that. Note: prior to version 3.4.2, the cell reading functions did not apply the the configuration deny rules.

Calls to the different CoralReef reading functions may not be interleaved on a given set of sources.

A call to any CoralReef reading function on a given set of sources invalidates all data in buffers set by previous calls to any reading functions on the same set of sources.

Return values

NULL with errno==0 for EOF, stopped device, or expired duration. (If end-of-file is encountered someplace other than a block boundary, it is considered an error.)
NULL with errno==EAGAIN if timeout is exceeded
NULL with errno==EBADF if sources are not ATM.
NULL with other errno value for other errors
non-NULL interface pointer for success

(Certain types of errors that were reported as EOF in versions prior 3.5 are now correctly reported as errors.)

Reading Functions

coral_iface_t *coral_read_cell(coral_source_t *src, coral_blk_info_t **binfop, coral_atm_cell_t **cellp, struct timeval *timeout); coral_iface_t *coral_read_cell_all(coral_blk_info_t **binfop, coral_atm_cell_t **cellp, struct timeval *timeout);

coral_read_cell() reads a single cell from the indicated source; coral_read_cell_all() reads from all open sources. If the CORAL_OPT_SORT_TIME option is off, coral_read_cell_all() reads cells in the most efficient order (i.e., block order); but if the CORAL_OPT_SORT_TIME option is on, it will read them in time-sorted order (interleaving cells from blocks of different interfaces). If the CORAL_OPT_SORT_TIME is set, timeout is ignored (it is always treated as NULL).

See the section introduction above for more details and return values.

coral_iface_t *coral_read_cell_i(coral_source_t *src, coral_blk_info_t **binfop, coral_atm_cell_t **cellp, coral_interval_result_t *int_result, struct timeval *interval); coral_iface_t *coral_read_cell_all_i(coral_blk_info_t **binfop, coral_atm_cell_t **cellp, coral_interval_result_t *int_result, struct timeval *interval);

These functions are like coral_read_cell() and coral_read_cell_all(), except that there is no timeout and they support intervals.

If the value of interval is greater than 0.0, then time sorting is automatically enabled regardless of the setting of the CORAL_OPT_SORT_TIME option, and whenever *interval seconds have passed or EOF is reached, the functions will return non-NULL with *cellp == NULL to indicate that an interval has ended, and the start and end fields of int_result will be filled in with the boundaries of the interval. *binfop is undefined at the end of an interval. int_result->stats is always NULL. When EOF is reached, the final partial interval is returned, and the next call will return NULL.

See the section introduction above for more details and return values.

coral_iface_t *coral_read_block(coral_source_t *src, coral_blk_info_t **binfop, coral_atm_cell_t **cellp, struct timeval *timeout); coral_iface_t *coral_read_block_all(coral_blk_info_t **binfop, coral_atm_cell_t **cellp, struct timeval *timeout);

The cell reading functions are usually more convenient than these block reading functions.

After a successful read, the number of valid cells in the array pointed to by *cellp is indicated by ntohl((*binfop)->cell_count), the number of bytes (including padding) pointed to by *cellp is indicated by ntohl((*binfop)->blk_size), and the size of a cell is coral_cell_size(iface). The nth cell in the array pointed to by *cellp which was read from interface iface can be found with the expression coral_nth_cell(iface, *cellp, n).

Related Functions

coral_atm_cell_t *coral_nth_cell(const coral_iface_t *iface, coral_atm_cell_t *blk, int n);

Returns a pointer to the nth cell in the array of cells pointed to by blk, which was read from iface. (coral_nth_cell() is actually defined as a macro.)

void (*coral_cell_block_hook)(const coral_iface_t *iface, const coral_blk_info_t *binfo);

If coral_cell_block_hook is not NULL, the function to which it points will be called whenever a cell reading function or packet reading function reads a new block of ATM cells. iface and binfo will point to data about the new block. The default value of coral_cell_block_hook is NULL.

int coral_proto_rule(const coral_iface_t *iface, uint32_t subif, coral_protocol_t *protop);

This function compares iface and subif (ATM vp:vc) against the proto and deny configuration rules. If they are denied by the rules, coral_proto_rule returns 0. Otherwise, it sets *protop to the protocol identifier according to the rules, and returns 1. Cells that would be denied by the rules are never returned by the cell API, but this function is still needed to find the protocol of the PDU to which a cell belongs or to test for denial of a cell returned by the block API.

If there is no matching proto configuration rule for subif, and there is no default protocol for iface, the protocol for 0:0 through 0:15 defaults to CORAL_PROTO_UNKNOWN, the protocol for 0:16 defaults to CORAL_DLT_ILMI, and the protocol for all other virtual channels defaults to CORAL_DLT_ATM_RFC1483.

Although coral_read_pkt puts IEEE 802.1Q VLAN IDs in the pkt_result->subif field, protocol rules should not be applied to VLAN IDs.

int coral_cell_to_pkt(const coral_iface_t *iface, coral_atm_cell_t *cell, coral_pkt_buffer_t *pkt);

Cell is a pointer to the first cell of an AAL5 PDU (in network byte order). coral_cell_to_pkt fills in the object pointed to by pkt with information from cell and iface, for use with functions that expect a coral_pkt_buffer_t argument. Iface and the cell's vp:vc are used to determine the protocol of the data according to the configuration proto rules. If iface is NULL, pkt->protocol will be set to CORAL_PROTO_UNKNOWN.

The return value is 0 if iface and the cell's vp:vc match a configuration deny rule, otherwise 1.

Source and Interface Information

These functions provide information about CoralReef sources and interfaces.

const char *coral_file_version(const coral_source_t *src);

Returns a pointer to a static buffer containing a string describing a file format version. If src corresponds to a tracefile, the string describes the file format of the tracefile; if src is NULL, the string describes the current file format (i.e., the highest format version that this version of libcoral can read, and the format that would be written by coral_write() in this version of libcoral).

coral_iface_t *coral_next_interface(const coral_iface_t *iface); coral_source_t *coral_next_source(const coral_source_t *src);

If iface/src is NULL, these functions return a pointer to the first CoralReef interface/source; otherwise, they return a pointer to the interface/source following iface/src. They return NULL if there is no next interface/source. The order in which interfaces/sources are returned is not necessarily the order in which they were opened. These functions are used to step through all interfaces/sources, for example:

    coral_source_t *src = NULL;
    while ((src = coral_next_source(src))) {
        coral_dump(src);
    }

coral_iface_t *coral_next_src_iface(const coral_source_t *src, const coral_iface_t *iface);

Like coral_next_interface(), except it only returns interfaces of src.

const char *coral_source_get_filename(const coral_source_t *src);

Returns the name of the file or network interface of source src, without a coral type prefix. Returns NULL if src is bad.

const char *coral_source_get_name(const coral_source_t *src);

Returns the name of the source src, including a coral type prefix, if any. Returns NULL if src is bad.

const char *coral_source_get_comment(const coral_source_t *src);

Returns a pointer to the comment string from the file header of source src. Returns NULL if src is bad or there is no comment.

int coral_get_source_count(void);

Returns the number of sources.

coral_source_t *coral_interface_get_src(const coral_iface_t *iface);

Returns a pointer to the source to which interface iface belongs, or NULL if iface is bad.

int coral_interface_get_type(const coral_iface_t *iface);

Return the original (hardware) type of interface iface, or return CORAL_TYPE_NONE if iface is invalid or its type is unknown.

CORAL_TYPE_FATM: FORE ATM adaptor device
CORAL_TYPE_POINT: Apptel POINT device
CORAL_TYPE_DAG: Waikato DAG device
CORAL_TYPE_PCAPLIVE: pcap interface

int coral_source_get_type(const coral_source_t *src);

Return the type of source src, or CORAL_TYPE_NONE if src is invalid or its type is unknown. For live sources, the type will be one of the types listed in the description of coral_interface_get_type(); for file sources, the type will be one of the following:

CORAL_TYPE_FILE: CoralReef trace file
CORAL_TYPE_PCAP: pcap (tcpdump) trace file
CORAL_TYPE_DAGFILE: dagtools trace file
CORAL_TYPE_TSH: NLANR Time Sequenced Header trace file

coral_protocol_t coral_interface_get_datalink(const coral_iface_t *iface);

Return the data link layer type of interface iface, or CORAL_PROTO_UNKNOWN if the protocol is unknown, or -1 if iface is bad. The data link layer type is one of the CORAL_DLT_* (layer 2) constants defined in the Command Usage document.

coral_protocol_t coral_interface_get_physical(const coral_iface_t *iface);

Return the physical layer type of interface iface, or CORAL_PROTO_UNKNOWN if the protocol is unknown, or -1 if iface is bad. The physical layer type is one of the CORAL_PHY_* (layer 1) constants defined in the Command Usage document.

int32_t coral_interface_get_bandwidth(const coral_iface_t *iface);

Return the bandwidth (in kilobits per second) of interface iface, or 0 if the bandwidth is unknown, or -1 if iface is bad.

int coral_source_get_number(const coral_source_t *src); int coral_interface_get_number(const coral_iface_t *iface);

Return a non-negative integer identifier of source src or interface iface, or return -1 if src or iface is bad. The iden