Changes marked "MEMBERS ONLY" are available only to CAIDA members in special members-only releases of CoralReef. You can find information about becoming a CAIDA member at "https://www.caida.org/members/". Version 3.9.5 (2019-03-28) ------------- * Fixed some build problems on Linux. * Fixed build problem in crl_flowest with GCC 6. * Fixed several other build problems on newer platforms. Version 3.9.4 (2014-10-14) ------------- * Fixed build problem in Crypto-PAn with the clang compiler. * Fixed other potential problems with strict compilers. Version 3.9.3 (2014-03-27) ------------- * Fixed build problems in compilers with stricter C++ standard compliance. Version 3.9.2 (2013-08-14) ------------- * Fixed memory leak in printing DNSSEC packets * Fixed memory leak in printing corrupt or truncated DNS packets * Fixed "coral: -: gzip error: " when reading non-compressed data from pipe Version 3.9.1 (2012-12-03) ------------- * Fixed build errors involving "/usr/lib/perl/5.10/CORE/" * Fixed detection of Berkeley DB Version 3.9.0 (2012-09-28) ------------- * Added IPv6 support to crl_flow and crl_anf * Added IPv6 address and packet anonymization, including an option to apply IPv4 anonymization policy to IPv4 addresses embedded within special IPv6 addresses (IPv4-mapped, SIIT, Teredo, 6to4, 6over4, ISATAP). * Added anonymization option "keepmcast" to not anonymize multicast addresses. * Packet anonymization now applies to addresses in IPv4 Record Route options and ICMP REDIRECT. * Added coral_inet_ntop() function * Parses DLT_LINUX_SLL (Linux "cooked" encapsulation), created by Linux raw interfaces (e.g. "if:any"). * Recognizes (but does not parse) EGP, MTP, ENCAP, PIM, SCTP. * Added options to crl_print_pkt to control printing of IPv6 extension headers * crl_print_pkt -s (short format) now includes IPv6 information * crl_to_pcap -Cv=2 prints timestamps of discarded packets * Added preliminary support for IPv6 processing in Tables code: - parses new headers from crl_flow and crl_anf - autoconverts from IPv6 Tuple Table to Proto and App tables * Parses ICMP extensions for MPLS (RFC 4884, 4950) * Improved printing of MPLS * Fixed several bugs in parsing and printing IPv6 extension headers: - printed incorrect value of IPv6 fragment offset - didn't print name of protocol inside a fragment - printed garbage for truncated ROUTING header - parsing of ESP and AH when CORAL_OPT_IP_EXTHDR is off - parsing of multiple extension headers when CORAL_OPT_IP_EXTHDR is off - identification of IPPROTO_IPIP and IPPROTO_IPV6 in extension header * Fixed: rarely, printed garbage when printing truncated/corrupt DNS packets. * Fixed spurious error "dag legacy ethernet format not supported" on some DAG ERF files captured on ethernet links with jumbo frames when user specifies "-Cproto=ether". * Fixed bug in coral_rf_ogz_seek() on some systems when final offset would exceed 2^31-1 bytes. In particular, this could affect crl_to_dag when writing a truncated packet, giving errors like "write: Unknown error: 0" or "write: Value too large to be stored in data type". * Fixed occasional hanging when reading ethernet DAG card. * Changed coral_rf_ogz_seek() offset parameter from a long to an off_t. * Fixed: some errors in gzip input were undetected. * Fixed possible early termination of an interval when reading multiple sources and one had a gap in data larger than an interval. * Documented coral_write_get_name(). * Does stricter checking for Ethernet in MPLS, to avoid ambiguity in packets that are really IP in MPLS. * Fixed compile error in libsrc/IpPrefixPatricia/IpPrefixPatricia.hh involving ptrdiff_t with g++ 4.6. * parse_bgp_dump now removes invalid prefixes unless -a option is used. * Adjusted data storage/retrieval in create_graphs to create more accurate RRDtool graphs when input is sporadic or missing. * parse_bgp_dump no longer converts 4-byte AS number to decimal. * parse_bgp_dump now automatically removes prefixes that are known to be invalid for global routing (such at 0.0.0.0 or RFC 1918 addresses), unless -a option is used. * parse_bgp_dump tries to compensate for BGP dumps where there is no space between the prefix length and next hop IP. * Fixed a bug in printing corrupt DNS packets. * Fixed small inaccuracies in data retrieved from RRDs in create_graphs.pl. * Fixed problem caused by using SWIG versions greater than 1.3.39 which causes memory faults. Version 3.8.6 (2009-06-04) ------------- * Added files missing from 3.8.5: ipset.c and ipset.h. * Fixed error in interpreting DLT_NULL or DLT_LOOP. Version 3.8.5 (2009-06-03) ------------- * Added application crl_ips to efficiently extract IP addresses from trace files. * Anonymization now truncates non-anonymizable packets instead of discarding them. * Added ability to create_graphs.pl to read per-category full names and colors, which deprecates previous long_names and colors config options. * Standardized create_graphs.pl "generated at" timestamp to simpler format. * Updated output of create_graphs and display_report with more table headers and increased precision for different ranges of percentages. * Added -m option to crl_anf to merge interfaces. * Changed verbosity level of forked dumping (with -R) in crl_anf and crl_flow * parse_bgp_dump now parses 4-byte format AS numbers and converts to decimal. * Added support for MPLS in PPP. * Added support for PPPoHDLC in UoPOS. * Coral's PPPoHDLC maps to pcap's DLT_PPP_SERIAL. * On proto=CHDLC links (the default for phy=POS links), coral will warn if a packet appears to actually be MPoFR or PPPoHDLC. * Added ability to convert PPPoHDLC to pcap, so crl_to_pcap doesn't have to strip link layer, and pcap filters work on link layer. * Can read and decode DLT_LOOP interfaces (OpenBSD loopback). * Recognizes (but does not parse) LCP. * Can identify and parse Ethernet in MPLS. * crl_print_pkt now prints e.g. "UDP fragment" for contents of fragmented IP packets. * crl_rate now uses Layer 2 information to estimate bytes of IP packets whose length field was truncated. * Better detection of first packet of a DNS stream in TCP. * Documented new -Cskippackets option and coral_set_skip_pkts() function. * Added a warning when a libpcap bug prevents a filter from working correctly on vlans and other link layers it can't parse. (Workaround: use "ipfilter".) * Eliminated warning about requested options not matching recorded options when trace file's capure len is variable (varlen); added warning when file contains an actual packet whose capture length is less than requested. * Fixed bug in crl_rate that incorrectly counted IPv6 bytes. * Added error message when t2_convert cannot find any requested tables. * Fixed problem on Darwin when Perl looks for .dylib files instead of .so. * Fixed: user-specified "nif" option on live DAG cards was essentially ignored. * Fixed build error involving "assert" in crl_anf in some compilers (including g++ 4.2). * Fixed possible corruption or core dump while printing truncated or corrupt DNS packets. * Fixed bug in crl_rate that incorrectly counted IPv6 bytes. * Fixed: on some interfaces, attempting to use a filter could cause the error "snaplen of 0 rejects all packets". * Fixed: automatic mapping of "PPP" to "PPPoHDLC" did not work on compound sources after the first file, resulting in mis-parsed packets. * Fixed: if crl_to_dag encountered an error on input, it did not always properly terminate its output file. * Fixed problem building AppConfig with Perl 5.10. * Fixed compilation errors with newer GCC (missing headers). * Updated required SWIG version to 1.3.33 to work with newer GCC. * Fixed handling of truncated DAG headers. * Fixed: choked on DAG files that ended 1 to 11 bytes after a 1MB boundary. * Fixed possible loop in DNS printing of rare truncated packets. Version 3.8.4 (2008-06-26) ------------- * crl_anf now supports binary output format with the -b option. * crl_anf and crl_flow now support dumping via forked child processes with the -R option. * Parses new protocol: LLC encapsulation of Bridged Ethernet/802.3 PDUs (RFC 1483 section 4.2, RFC 2684 section 5.2). * Recognizes (but does not parse) new protocols: Reverse ARP, Cisco SLARP in CHDLC, and CLNP/ES-IS in CHDLC. * The coral errfile is unbuffered. * Fixed: read timestamps incorrectly from pcap interfaces and traces on some 64-bit platforms. * Added workaround for libpcap filter bug that caused crl_dnsstat to ignore vlan packets. * Fixed: crl_to_pcap with a DAG ATM file as input would create a pcap file with snaplen=0, which confused most tools that tried to read the file. * Workaround for limitation in Wireshark: when original length of packet is unknown, crl_to_pcap sets it to 65535 (0xffff) instead of 4294967295 (0xffffffff) in its output pcap file. Old files with large lengths can be made wireshark-friendly simply by running them through the new crl_to_pcap. * crl_to_dag now emulates a quirk of DAG cards that sometimes write an incorrect record length, for compatibility with dag tools that expect the quirk. Files produced by older versions of crl_to_dag with payloads stripped may contain records without the expected quirk. The dagbits tool will report "warning: len change 20->28" when it expects the quirk; if that is followed by other errors, the quirk was not found, and subsequent results will be invalid. Current CoralReef apps can read both quirked and non-quirked files; a non-quirked file may be converted to a quirked file for compatibility with dag tools simply by running it through crl_to_dag. * crl_print_pkt prints packet number. * Protocol parser is more forgiving of certain malformed TCP and UDP packets, so they are not unnecessarily truncated during payload truncation (e.g., the -l4 option of crl_to_pcap or crl_to_dag), and they can be decoded by crl_print_pkt. * Protocol printer can decode DNSSEC resource records (DNSKEY RRSIG NSEC DS). * Protocol printer now detects certain kinds of invalid DNS data and prints an error message instead of attempting to decode it. * Fixed: packet printer incorrectly printed DNS TTL values greater than 65535 (~18.2 hours) and several other DNS values greater than 2147483647. * Improved printing of malformed IPv4 packets that appear to contain an IPv6 extension header. * The crl_to_pcap and crl_to_dag apps with an -l option, and the coral_pkt_truncate() function, no longer discard a layer just because it has an incompletely captured header. * Fixed: anonymization on certain rare types of malformed packets or truncated packets with unusual encapsulations could could corrupt subsequent packets. * Packet anonymizer keeps first 8 bytes of IPv6 headers instead of discarding IPv6 packets. * Fixed anonymizer to treat RAW_IP packets as the appropriate IP version, instead of discarding the packet. * Earlier reporting of non-readability of files in compound sources. * Fixed possible corrupt pcap output when reading compound gzipped pcap input (e.g., "crl_to_pcap pcap:[ *.pcap.gz ]"). * Added packet loss counter to crl_anf and crl_flowest/flowbloom output. * Increased hashing efficiency in crl_flowest/flowbloom. * Made random seed in crl_flowest/flowbloom work properly. * Added -P3 option to crl_flowest/flowbloom to output protocol tables separately. * Changed parse_bgp_dump to properly parse both old and new RouteViews formats. * Fixed buffer overflow in crl_flow when using very long command lines. * Added timezone tag to end of date range for RRDtool graphs in create_graphs. * Fixed problem in merge_protos when last_ds value is empty. * Fixed: store_monitor_data not properly updating timestamps due to mmap() in RRDtool 1.3. * Fixed: crl_stats compilation failure without time.h * Changed Tables classes to properly parse and scale packet and flow sampling multipliers (from crl_anf) separately, as well as calculating 'other' flow counts from crl_flowest/flowbloom. * Fixed build error "Undefined symbols: __Unwind_Resume" on OS X with gcc 4.0.1 Version 3.8.3 (2008-02-01) ------------- * Fixed misinterpretation of DAG loss counter that caused CoralReef applications to underreport lost packets or occasionally report a negative number of lost packets from a DAG interface. (The correct values were always recorded in trace files by dagsnap or crl_to_dag, so fixed versions of CoralReef apps will report correct values when run on old traces.) * Fixed: in crl_* commands with rotating output files with a ".gz" suffix, the output was not automatically gzipped (although it could be forced with the -Cgzip option). (broken in 3.8.1) * Fixed bugs in new dropinterval interval option that could kill process in first or second dropinterval. * Fixed 'actual pkts' output of crl_flowest to give proper values. Version 3.8.2 (2007-12-12) ------------- * Added dropinterval option to report dropped pkts on live interfaces more frequently than the normal interval. * Fixed compile error in apps/flowest/mrbmp.h on some platforms * Fixed compile error in ASFinder.so with certain combinations of perl and Mac OS X * Documented lack of IP fragmentation reassembly in flow-counting apps. * Fixed process leak in closing a gzipped pcap file for reading. * Added tutorial on setting up report generator in doc/reporting_tutorial.html * Updated example report generator config files to not require as much editing to work properly. Version 3.8.1 (2007-07-30) ------------- * When rotating files, coral no longer uses the final filename's suffix on temporary filenames, because doing so could cause problems for processes that expected temporary files to be named differently (e.g., the report generator using the results of crl_flow). This required an non-backward-compatible API change in the coral_rf_cb_open_* functions. * Fixed: crl_print_pkt app and coral_*print_pkt functions could loop or crash when parsing a malformed compressed DNS name. * Fixed: output of "crl_flow -m" was not readable by t2_* applications. * added --with-prefix-alias option to configure Version 3.8.0 (2007-06-08) ------------- * Added support for ATM physical layer in DAG ERF devices and traces. * Added applications that comprise a new method of traffic report generation: store_monitor_data, create_report, and display_report, with helper applications config_graphs and create_graphs. They all rely on a support module, CAIDA::Traffic2::ReportSupport * Added crl_flowest, crl_flowbloom, and crl_anf, which are similar to crl_flow, but implement different methods of sampling and/or estimation to reduce memory/CPU overhead at the cost of some accuracy. Note: if the flowest apps fail to build, and you do not need them, you may omit them from your build with the --disable-flowest option to configure. * Added t2_merge application to merge interfaces and subinterfaces together in t2-format files. * Added -m option to crl_flow to merge interfaces. * Added support for new protocols: - GRE (Generic Routing Encapsulation, RFC 2784) - MPoFR (Multiprotocol over Frame Relay / Frame over SONET, RFC 1490 & 2427) - UoPOS (Unknown over PoS) * When anonymizing, non-anonymizable layer 3 headers (which were previously kept only when encapsulated within IPv4) are truncated. New options allow user to keep non-IPv4 packets and not truncate non-IPv4 headers. * crl_guess identifies the number of interfaces recorded in a file. * Added -Cmintime and -Cmaxtime options to most packet applications, to discard packets before mintime and after maxtime. Also added corresponding functions to C and perl APIs. * When crl_stats encounters an error in its input, it prints the statistics for the data seen so far, prints a warning, and exits with nonzero status. * Output of application crl_to_pcap (and any other files open by the API function coral_rf_open_pcap_dump()) can be gzipped. * Fixed crl_flow handling of -Canon option when CryptoPAn failed to build * Fixed several cases of libcoral incorrectly reporting normal EOF or attempting to continue to read packets after an error. * Fixed busywaiting while reading live DAG cards. * Fixed: would sometimes return nothing in unusual case of reading multiple interfaces where one has low or no traffic. * Workaround for FreeBSD lseek() bug that could cause CoralReef apps to fail when reading from a fifo. * Fixed parsing of multiple netmasks in AppPorts rule files. * Changed format of AppPorts rule file to allow for ignoring port matching (protocol and/or IP matching only). * Changed AppPorts match_rule() and get_rule() to allow returning multiple matches instead of just the highest priority one. * Added new applications to example port->application file, and updated older entries. * Updated ISO 3166 country codes to current usage. * Changed t2_convert to allow multiple table conversion. * Changed t2_convert to make ICMP code/type separation in conversion to App_Table not be the default, and added -s option to allow that separation. * Changed parse_bgp_dump's output filename to reflect the input filename. Also allowed uncompressed input files. * Fixed: -Canon=ipzero did nothing if CryptoPAn was not found when CoralReef was built. * Fixed: packet truncation (e.g., crl_to_pcap -l4) didn't always truncate packets that coral could not parse. * Fixed check for large file support on linux systems where it's not enabled by default. * Fixed: when reading from a dag interface, in some situations libcoral would stop reading after the first packet. * Fixed memory corruption in coral_get_anonymizer() function, which may crash crl_flow during startup, or cause anonymization in crl_flow to not work correctly. * Fixed compile error on some systems (linux) missing definitions of ICMP6_MEMBERSHIP_QUERY, ICMP6_MEMBERSHIP_REPORT, ICMP6_MEMBERSHIP_REDUCTION. * Fixed: crl_dnsstat was included in the public release, but did not build by default. * Fixed: crl_dnsstat with infinite interval (-Ci=0) did not print any results. * Fixed crash in packet reading apps when using extremely large intervals. * Changed name of Perl libcoral API from CRL.pm to Coral.pm. * Fixed module loading problem in Traffic_Plot.pm that caused t2_report++ to fail. * Fixed bug with Tables sorting code that caused excessive memory and CPU usage. * Workaround for problem with 'perlio' layer causing various errors while reading t2-format files in CAIDA::Tables library and t2_* applications. Possible manifestations include incorrect data, and the warning message "Warning: Must use positive value to set FlowCounter's bytes". * Fixed crl_info -d on TSH files. * Other bugfixes. Version 3.7.5 (2006-02-28) ------------- * Added -f option to crl_stats to not count flows. * Changed behavior of spoolcat: Added -M, -S, and -d options to specify move directory, statefile, and deletion, respectively. Deletion is no longer the default, and even without the statefile, files will not be spooled twice in the same run. * Added "ipzero" anonymization algorithm, which zeros-out bits of IPv4 addresses in packet-reading applications. Also added parameters to limit anonymization to subsets of addresses or parts of addresses. (Look for "anonymize" in the command usage documentation.) * In libcoral, replaced *_PAnonymize[r] functions with *_anonymize[r] functions, to support new "ipzero" anonymization. * Changed output format of crl_stats. * Packet printer prints NOP in IP and TCP options, and prints window size for SYN packets in addition to ACK packets. * Added -P option to t2_convert to specify port-mapping file for App Table conversion, but defaulting to the CoralReef example. * Added options for Country Table/Matrix conversion to specify 2 or 3-letter abbreviations. * Removed spurious blank line in printing unknown/truncated DNS RR's. * Fixed misinterpretation of file suffixes that were a leading substring of the common file suffixes ".crl", ".pcap", ".dag", ".tsh". * Fixed definition of coral_read_pkt to avoid multiple definition errors when libcoral.h is included in multiple sources (in external code). * Fixed: coral_to_dag produced corrupt output when writing an ethernet file to a pipe. * Fixed bugs in recalculation of UDP checksum during IP packet anonymization: - was not recalculated if UDP datagram was 12 bytes or shorter - was recalculated even if checksum was originally 0 * Fixed fatal bug in handling multi-interface TSH files, introduced in 3.7.4. * Added file rotation, interval support, and -o option for specifying output file to crl_print_pkt. * Fixed bug that caused an infinite loop at end of an interval in rare cases on a live low-traffic interface. * Fixed shared library build problem with gcc on 64-bit architectures ("relocation R_X86_64_32S can not be used when making a shared object"). * Fixed hardcoded references to "/usr/local/Coral" in Countries.pm * Fixed: Countries.pm corrupted the include path of scripts that used it. Version 3.7.4 (2005-04-18) ------------- * Updated installation process to work with DAG software distribution 2.5.x. * Added crl_stats application, which outputs summary information about IP addresses and ports. * Added libcoral functions coral_get_iomode(), coral_source_get_iomode(), coral_iface_get_iomode(), and coral_format_iomode(), and corresponding CRL.pm functions. * Changed arguments and return values of CRL.pm functions Coral::fmt_if_subif(), Coral::fmt_subif(), and Coral::filename(). * IP address anonymization does incremental recalculation of IP, TCP, and UDP checksums. * Improved error messages for gzip errors. * Fixed "undefined versioned symbol name std::basic_string..." error in linking perl modules that occured on some combinations of compilers, libraries, and platforms. * Workaround for perl bug which on some platforms caused the compile error ".../CORE/reentr.h: field `_crypt_struct' has incomplete type". * Fixed problem in "depend" files which on some systems caused build failure involving "/usr/local/lib/perl5/5.8.2/mach/CORE/EXTERN.h". * Fixed: coral_pkt_truncate() (and thus the -l option of crl_to_dag and crl_to_pcap) could generate an invalid packet when the input packet's highest layer was lower than the requested maximum layer. * Fixed potentially fatal bug in handling single-interface TSH files. * Fixed: app crl_print_pkt and function coral_fmt_get_payload() sometimes truncated IP addresses that didn't need truncating. Version 3.7.3 (2004-12-21) ------------- * Fixed missing t2_report++.pl file in public release. * Fixed bug in ParseBGPDump.pm that caused premature script end with a RIB failure (lines starting with 'r'). (This should not occur in tables obtained from Route Views, but can in tables obtained elsewhere.) Version 3.7.2 (2004-11-11) ------------- * Made crl_dnsstat and t2_report++ available in the public release. (t2_report is now just a symlink to t2_report++, for backward compatibility.) * Added functions to convert from appropriate tables into CAIDA::Tables::LatLon_Table, with support in FileReader/etc. * Added conversion diagrams to documentation for CAIDA::Tables. * Fixed bug in ParseBGPDump.pm that caused 'Unset' to appear in output files. * Fixed build error involving coral_rf_cb_open_pcap_dump when linking with old version of libpcap. * Fixed "pcap_if_t" build error with some versions of libpcap (e.g., MacOSX). * Fixed "-undefined dynamic_lookup" error when linking perl modules on MacOSX. * Workaround for perl/swig/g++ 3.3 conflict that caused build errors involving "do_open" in ASFinder.cc, AppPorts.cc, Tables.cc, FlowCounter.cc. Version 3.7.1 (2004-10-07) ------------- * Workaround for gcc/Solaris interaction that disabled Crypto-PAn support. * Added dns_str_to_*() and dns_*_to_str() symbol/number mapping functions to CRL.pm to match those in libcoral. * Fixed compile-time error when building with DAG support (undefined "TRUE" in coral_type_dag.c) Version 3.7.0 (2004-09-23) ------------- Changes since 3.6.3 * Upgraded DAG support: - Can read live DAG 4 cards. - Can read DAG ERF file format. - Added crl_to_dag, which can read from any coral packet source (not just DAG cards) and capture to a DAG ERF format file. - Tracks DAG's packet loss counter. * Added "compound sources": on the command line of crl_* applications, a list of sources enclosed in "[" and "]" are treated as a single source containing the concatenation of the data in the listed sources. * Added prefix-preserving anonymization with Crypto-PAn, configurable on command line in all packet applications with the -Canon option, and accessible through the libcoral API. * Added -Calignint option to align interval boundaries to a multiple of the interval size. * Added crl_ips to extract src and dst IP addresses from coral sources. * Perl Tables API: - Made Perl-only version of AppInfo_Table work correctly. - Added CAIDA::Tables::IP_Proto_Ports_Table and IP_Proto_Port_Table and appropriate converters from other tables, with support in FileReader. - Added CAIDA::Tables::String_Table to allow arbitrary untyped user strings - Added CAIDA::Tables::LatLon_Table, currently without any conversion functions. - Created make_App_Table() as a member function of CAIDA::Tables::Tuple_Table and Proto_Ports_Table. * Can read single-interface TSH files (with interface number 0) in addition to traditional 2-interface files. * Can read gzipped pcap (tcpdump) files. * Ported to the stricter C++ standard compliance of g++ 3.4. * Added -C"ipfilter=" command option and coral_add_pcap_ipfilter() function to apply filter after using CoralReef's parser to find IP layer. * If an IPv4 address is truncated, the crl_print_pkt app and crl_*print_pkt functions print as much as possible. * Interval and duration may be given in "hour:minute:second" format. * Added coral_source_is_block() * Added payload parameter to coral_pkt_truncate() (libcoral) and CRL::pkt_truncate() (CRL.pm) * Eliminated warning about trace file having longer packet capture than requested by application. * Fixed: -Cpackets=N was stopping after only N-1 packets. * Fixed/improved crl_guess recognition of FATM files, files containing RAW_IP and fragmented IP, etc. * Fixed infinite loop in printing (crl_print_pkt app and coral_print_pkt() function) of malformed DNS packets. * Fixed building problems on systems without IPv6 or libpcap. * Fixed: "crl_print_pkt -s" was misinterpreting IPv4 fragments as transport headers (i.e., printing sport and dport when they weren't really present). * Made Perl version of crl_pkt_example.c to demonstrate usage of the Perl API. * Added -b option to t2_convert to output binary format. * Fixed problem in CRL API where Pkt_buffer::protocol() returned a blessed reference instead of a number. Changes since 3.7 pre 4 * Added "compound sources": on the command line of crl_* applications, a list of sources enclosed in "[" and "]" are treated as a single source containing the concatenation of the data in the listed sources. * crl_to_dag preserves loss counter if source is DAG. * Fixed spurious "no support for DAG ETH format with nonzero offset" message. * Fixed/improved crl_guess recognition of FATM files, files containing RAW_IP and fragmented IP, etc. * Fixed handling of mode first=N on DAG ATM (broken in 3.7.pre1) * Fixed: incorrectly disallowed filters and packet count on crl_to_dag (3.7), crl_cut, or any app that set CORAL_API_WRITE. * Fixed infinite loop in printing (crl_print_pkt app and coral_print_pkt() function) of (typically malformed) DNS packets with labels longer than 127 bytes. * Fixed building problems on systems without IPv6 or libpcap. * Fixed: "crl_print_pkt -s" was misinterpreting IPv4 fragments as transport headers (i.e., printing sport and dport when they weren't really present). * Perl Tables API: - Added CAIDA::Tables::IP_Proto_Ports_Table and IP_Proto_Port_Table and appropriate converters from other tables, with support in FileReader. - Added CAIDA::Tables::String_Table to allow arbitrary untyped user strings - Added CAIDA::Tables::LatLon_Table, currently without any conversion functions. - Created make_App_Table() as a member function of CAIDA::Tables::Tuple_Table and Proto_Ports_Table. * Added -Calignint option to align interval boundaries to a multiple of the interval size. * Made Perl version of crl_pkt_example.c to demonstrate usage of the Perl API. * Added -b option to t2_convert to output binary format. * Fixed problem in CRL API where Pkt_buffer::protocol() returned a blessed reference instead of a number. Version 3.6.3 (2004-03-18) ------------- * Fixed build error involving missing ../../coral-config.h. * Fixed compatibility with libpcap-0.8. * Relaced automatic pre-AAL5-reassmbly pcap filtering with a manual "prefilter" command and functions. Version 3.6.2 (2004-03-01) ------------- * Fixed installation of fips.txt, which is required for Countries.pm (Countries.pm is a perl module added in 3.6.0 for finding country name, country code, and continent information) Version 3.6.1 (2004-02-20) ------------- * ASFinder handles improperly masked prefixes in prefix file. * Added -R option to t2_convert to specify routing table for AS aggregation. * Fixed compile error with old or missing libz. * Installs necessary files in $prefix/Coral/include/netinet on linux so standalone applications can build by following $prefix/Coral/lib/example. * Minor documentation fixes/improvements. * CAIDA::Tables now can do Tuple_Table->{src,dst}_AS_Table w/o going through AS_Matrix first. (only in C code) Version 3.6.0 (2003-12-01) ------------- General: * Supports arbitrary (but not variable) record sizes in POS DAG files * Can read POS files written by DAG 4.2 tools. * Fixed: truncated 2 bytes of total length of packets from dag * New protocol decoding: - MPLS (CORAL_DLT_MPLS), can identify encapsulated IPv4 or IPv6 - IPv6 fragments (fixed) - TCP options: window scaling, sack-permitted, sack, timestamp - TCP and IP id, sequence, and ack numbers are printed in hex - ECN flags for IPv4, IPv6, and TCP (RFC 3168) - DNS (CORAL_PROTO_DNS) * Can read/write IPv6 in addition to IPv4 in DLT_RAW pcap (tcpdump) sources; added corresponding CORAL_NETPROTO_RAW_IP pseudo-protocol. * Can recognize AppleTalk and IPX in DLT_NULL pcap (tcpdump) sources. * Fixed potential crashes due to odd-length protocol headers on architectures with strict alignment requirements. * Improved handling of corrupt trace files. Discards all cells before a timestamp error in the first 16s of an FATM trace (not just the first block). Reports an error for any timestamp error past 16s in an FATM trace or the first block in a POINT trace (instead of returning invalid data). New -Cignore_time_err command line option and CORAL_OPT_IGNORE_TIME_ERR libcoral option override this behavior (but may give meaningless results). * Can read and decode DLT_NULL interfaces (e.g., loopback) * Fixed bug in timestamp sorting of multiple live pcap interfaces. * Supports large (>2GB) files on systems that support it (e.g. Solaris 5.8) * Workaround for libpcap bug that caused coral to ignore capture length on bpf interfaces without a bpf filter. Installation: * Removed Perl backend from ASFinder; it now requires a modern C++ compiler to be built. * Fixed error in libsrc/CAIDA when compiling with gcc 2.96. * Added correct namespaces and casts to be properly C++ compliant (for compilers such as gcc 3.0) * Fixed compile problem in Tables.cc involving missing PRIxN with various combinations of C++ compilers, perl, and swig. * Improved detection of Berkeley DB during installation. * Ported to gcc 3. C libraries: * Installs example C application and Makefile in /usr/local/Coral/lib/example. * coral_read_pkt() by default sorts packets at interval granularity, not packet granularity. This improves performance, but may break applications that depend on packet-granularity sorting; such applications should explicitly enable sorting. * libcoral.h defines IP_OFFMASK. * Fixed: coral_fprint_pkt() ignored file argument and always used stdout. * Changed return type of coral_interface_get_{datalink,physical}() and coral_proto_id() from int to coral_protocol_t. * Added coral_config_defaults() to restore default configuration. * Added coral_rf_* functions to libcoral to manipulate rotating output files. * Added coral_write_rfopen() to libcoral to allow rotating coral trace files. * Added coral_pkt_truncate() to libcoral to truncate packets by protocol. * Installs , containing macros for manipulating byte order of 16, 32, and 64 bit integer values. * coral_get_payload() skips IPv6 extension headers unless the new CORAL_OPT_IP_EXTHDR option is on. * New protocol decoding in the coral_*print_pkt and coral_get_payload* functions (see above). * Added coral_f{read,write}_binint() to read and write integers in a portable compact binary format. Perl libraries: * Coral::read_pkt() and Coral::quiet_start() by default sort packets at interval granularity, not packet granularity. This improves performance, but may break applications that depend on packet-granularity sorting; such applications should explicitly enable sorting. * Added Coral::config_defaults() to restore default configuration. * Added Coral::rf_* functions to CRL.pm to manipulate rotating output files. * Added Coral::write_rfopen() to CRL.pm to allow rotating coral trace files. * Added Coral::pkt_truncate() to CRL.pm to truncate packets by protocol. * Added new Table type Proto_Port_Table. * Made ASFinder6, which is just like ASFinder but for IPv6. * FileReader et al, and t2_report deal with 'expired' and 'active' attributes. * FileReader et al deal with new headers in crl_flow output. * FileReader will only return valid tables that can be created from input, not all requested ones. * FileWriter can dump user-created tables. * Changed ASFinder to read both old and new output formats of parse_bgp_dump. * New protocol decoding in the and Coral::*print_pkt and Coral::get_payload* functions (see above). Applications: * Improved performance of packet applications by not requiring time-sorting. * Applications crl_print_pkt and crl_time do not sort packets by default (but do with the -Csort option). * Improved performance of crl_hist. * New protocol decoding in the crl_print_pkt application (see above). * Fixed potential fatal bug in crl_guess. * crl_print_pkt defaults to -l7 * crl_time supports intervals (primarily useful for debugging libcoral). * Added -q (quiet) option to crl_time to print only anomalous records * Fixed spurious "bad wrap" messages from "crl_time -s" on a tsh file. * Added options to crl_rate to print only IPv4 or only IPv6. * When crl_dnsstat runs out of memory or reaches the entry limit specified by the new -N option, it does not die, but just collects less detailed data. * Fixed memory leak in crl_dnsstat. * Added -l and -u options to crl_to_pcap to truncate packets by protocol. * Applications crl_trace, crl_dnsstat, crl_to_pcap, and crl_flow can rotate their output and error files. * Added -ci option to crl_flow to reset flows' counters every interval. * crl_flow prints its command line in the header of the output file. * crl_flow output format changed to incorporate 'expired' and 'active' as part of the description of each table. * Added more compact binary format to crl_flow (experimental) * Fixed: crl_flow "# flows:" line was counting only new flows, not all flows. * t2_convert leaves 'flows:' header untouched when converting tables. * t2_rate deals appropriately with empty intervals and no data. * Added 'extra_html' option to t2_report's configuration file, as a short-term solution to adding arbitrary HTML to generated pages. * Changed 'refresh' command in t2_report output to work properly. * Fixed RRD output bugs with undefined values. * Added -g (guess) and -c (count) options to parse_bgp_dump to deal with multiple origin ASes. Default behavior is now to output a list of ASes instead of 'MultipleOrigins'. * parse_bgp_dump reads bzip2 files as well as gzip files. * parse_bgp_dump reads BGP dumps with linebreaks after long prefixes. Changes since 3.6.pre4 (caida internal only): * Packets read from raw IP sources will now have protocol CORAL_NETPROTO_IPv4 or CORAL_NETPROTO_IPv6 (unlike 3.6.pre4). Version 3.5.1 (2001-11-30) ------------- * Fixed definition of int8_t on platforms without int8_t natively and where unqualified "char" is unsigned. * Fixed "make {clean,distclean,realclean}". * Improved error messages. * Fixed: when reading from a TSH file, packets with more than 4 bytes of IP options contained trailing garbage. * Fixed crl_totsh: interface numbers were off by 1, and IP options were incorrectly interpreted as IP payload. * Fixed --with-incdirs and --with-libdirs options of configure. Version 3.5.0 (members only: 2001-11-20) ------------- general * IPv6 support in APIs libcoral and CRL.pm, and applications crl_rate and crl_print_pkt. * Supports NLANR TSH file format. * Parses IEEE 802.1Q VLAN. crl_print_pkt prints VLAN details. VLAN ID is treated as a subinterface id by applications crl_rate and crl_flow, function coral_read_pkt(), etc. E-RIF is not supported. Filtering by vlan requires libpcap-0.6.2 (or later). * Parses ARP for Ethernet and ATM. crl_print_pkt prints ARP details. * Recognizes and partially parses ILMI, SNMP. * Recognizes (but does not parse) IGMP, IEEE 802.1D, AppleTalk, AARP, and IPX protocols. * On ATM interfaces, the default protocol for virtual channels 0:0 through 0:15 is UNKNOWN, and for 0:16 is ILMI (but the default for all other virtual channels is still ATM_RFC1483). * CoralReef no longer initializes DAG devices, but expects them to be already initialized (using dagtools commands). This allows the user to have full control of initialization. * Normalizing timestamps to the unix epoch (CORAL_OPT_NORMALIZE) is now done by default, but is not required unless interfaces have different epochs. Can be disabled in crl_* apps with -Cnorm=0 command line option. * Packet applications (using coral_read_pkt()) print warnings about packet loss and other errors. * Added -Cgzip option to set compression level or enable gzip on filenames without ".gz" (e.g., "-" for stdout); changed default compression level from 6 to 1 (fastest). * New configuration commands to limit capture by packet, cell, or block count. (Programmers must call coral_set_api() to enable these commands in their own applications.) * When a crl_* application encounters an error in a Coral source, it exits immediately with nonzero exit status, instead of just closing that source and continuing to read other sources (if any). * Can automatically skip cells before an unexpected clock reset in the first block read from point and fatm interfaces. * crl_* applications and libcoral recognize "~" home directory notation in all filenames. * Can read and write non-regular files (e.g., /dev/fd/*, sockets, and FIFOs) as trace files. * Correctly handles reading pcap files from /dev/fd/* on systems where it's a device (like FreeBSD). * Iomode "first" overrides an earlier "user" on the command line (or CORAL_RX_USER_ALL in the C API). * Certain types of errors that were reported as EOF in earlier versions are now correctly reported as errors. * Fixed automatic recognition of empty pcap files. * Improved detection of Berkeley DB 1.85. * Added several new applications to the example Applications port list. * Fixed bugs when building SWIG applications with non-standard Perl configurations. applications * Added crl_guess to identify unknown tracefile and subinterface protocols. * crl_print_pkt can skip low level protocols. * crl_print_pkt: added -c option to print and verify IP checksums. * crl_to_pcap: added options to cut by packet count or timestamp. * crl_to_pcap has the option to discard link layer and write only raw IP. * crl_info: improved speed by several orders of magnitude for -d option on uncompressed regular .crl and .dag files. * crl_flow: added -o and -O options to specify output file. -O is used to create a new file every interval. * crl_flow: doesn't print active flows every interval by default; added -A option to do that. * crl_flow: fixed bug where active flows were not printed in intervals that had no expired flows. * t2_report: Changed handling of USR1 signal to terminate program after the processing of an interval. * t2_report: Added configuration option to allow incremental interval reads. * t2_report: Removed several obsolete configuration options. * t2_report: Made RRD graphs end at the interval time instead of real time (to work with trace data) * t2_report: Restructured RRD database layout to improve performance. If you want to use a pre-3.5 t2_report RRD directory with version 3.5, you will need to convert it; see apps/traffic/Reporting/README.RRDtool. * crl_rate uses 64 bit counters to avoid overflow. * crl_rate: Added rows for totals over interfaces and subinterfaces; added -D option to selectively display information for subinterfaces, interfaces, and totals. * crl_rate_layer2: Added interface information, a check for ATM sources, and cell loss statistics (per interface). * crl_rate_layer2: Made time normalization the default, with -U option to override. * crl_trace uses hostname and command line as the default tracefile comment. * Fixed bug in t2_report, where changing the protocols to be graphed caused the 'other' category to disappear. * Changed t2_report++ to use CAIDA::Tables::AppInfo_Table. (MEMBERS ONLY) * Changed t2_report++ to use AppInfo_Table. (MEMBERS ONLY) * Changed format of example Applications port list (removed quote marks) * crl_flow: Removed startup message about CIDR length. * crl_flow: Changed -b to -B for binary output. * Removed t2_aggregate; use t2_convert and t2_top instead. * Removed t2_IPMatrix and t2_Proto_Port; use t2_convert instead. * t2_ASMatrix: Changed options from -b, -p, and -f to -Sb, -Sp, and -Sf. * Added t2_rate. * Added spoolcat, a helper app for t2_report. * Made parse_bgp_dump no longer require the string "sh ip bgp" in the input file for it to work correctly. installation * Fixed use of --with-libdirs to find libz. C API (libcoral) * Added coral_iface_to_pcapp_raw() to discard link layer when converting packets to pcap format. * Added coral_write_pkt_to_pcap() for convenience. * Added coral_fmprint_pkt() and coral_mprint_pkt() to skip low level protocols when printing. * Added coral_filename() to expand "~" in filenames. * Added coral_inet_pton() for systems without inet_aton() or inet_pton(). * Added coral_new_fdsource() to open a source from an existing file descriptor. * Added coral_write_fdopen() to open a writer from an existing file descriptor. * Added CORAL_OPT_IP_CKSUM option to libcoral. * Added CORAL_FMT_IF_SUBIF_LEN and CORAL_FMT_SUBIF_LEN constants for coral_fmt_if_subif() and coral_fmt_subif(). * Parameters to coral_atm_pkt_hook and coral_cell_block_hook are const. * Changed prototypes of coral_in_cksum_add() and coral_in_cksum_result(). * Made it explicitly legal to call coral_pkt_to_pcap() on a packet from an interface which had not had coral_iface_to_pcapp() called on it. * Replaced last parameter of coral_usage() with a printf-style format and variable arguments (old calls will still work unless the last parameter contains "%"). * Fixed: coral_cell_block_hook wasn't called from read_pkt* or read_cell_all*. * Fixed value of *binfo in unsorted read_cell functions. * Added coral_set_max_{pkts,blks,cells}() to limit reading functions. Perl APIs * Removed CAIDA::Traffic2::Sample and CAIDA::Traffic2::Vpvc_data * Added CAIDA::Tables::AppInfo_Table, which uses application information (such as that returned by AppPorts) instead of just the name as a key. Still only hashes on the 'name' field, however. * Changed CAIDA::AppPorts to use a C++ backend (which is also directly accessible via C++), which increases performance. Also changed API slightly, see CAIDA::AppPorts documentation for more details. * Consolidated multiple .so files in CAIDA::Tables to one file, removing a memory leak (visible when running t2_report). * Made CAIDA::Tables and CAIDA::FlowCounter C++ backends publicly available, not just members only. The perl backend is now deprecated. * Fixed error in Perl API's quick_start(), where setting the libcoral API value failed. * Renamed Coral classes Packet, Buffer, Stats, and Interval to Pkt_result, Pkt_buffer, Pkt_stats, and Interval_results, respectively, to better match the libcoral C API. * Last parameter of Coral::usage() is optional. * Added libcoral parallels to Coral namespace: set_max_{pkts,blks,cells}(), filename(), fprint_data(), fprint_cell(), fprint_pkt(), fmprint_pkt(), file_version(), OPT_IP_CKSUM. Version 3.4.7 (2001-08-15) ------------- * Renamed crl_filter to crl_to_pcap. * Fixed: an error when reading a dagtools file caused an infinite loop * Fixed compile errors in coral_data.c and out_of_mem.cc on some systems (Red Hat 7.1) * Fixed detection of Berkeley DB 1.85 (false positive caused compile error on Red Hat 7.1, Solaris 8) * Fixed problem where 'make clean' would delete SWIG-generated files, causing subsequent compilations to require a SWIG binary. Version 3.4.6 (2001-05-25) ------------- Fixed corruption of first 2 cells of every Nth block from DAG device, where N is number of blocks in the device's buffer (introduced in 3.4.2). (First cell had a timestamp of 4096.0 and corrupt (mostly zero) data; second cell had a timestamp of 0.0, but other data was not corrupted.) Fixed core dump when closing a DAG device (introduced in 3.4.2). Version 3.4.5 (2001-05-04) ------------- * crl_filter's -i option selects the interface * Fixed: coral_read_pkt_init() and coral_read_pkts() ignored iface parameter. * Fixed reading final blocks of second interface of 2-interface trace file. Version 3.4.4 (2001-04-10) ------------- * Fixed timezone bug in t2_report that caused incorrect times to be output. * Fixed never-ending creation of new files with t2_report pie charts. * Fixed off-by-one error with RRD graphs in t2_report. * Fixed error using CAIDA::ASFinder in crl_bycountry. * Fixed erroneous "Couldn't find revision of report file!" in crl_hist. * Fixed graphing problem with t2_report (using RRDtool) and trace files. * Fixed excessive timeouts for country lookups when using CAIDA::Tables or crl_bycountry. Version 3.4.3 (2001-03-21) ------------- * Fixed payload corruption when reading packets from 2 or more DAG interfaces. Version 3.4.2 (2001-03-14) ------------- * Added documentation for Tables and other perl APIs. * Fixed "Proto_Table requires 6 fields (you gave 1)" error in t2_report using Perl version of CAIDA::Tables. * Fixed bug caused by writing a binary file from a Perl-only Length_Table and reading it with a C-backend Length_Table, and vice versa. (MEMBERS ONLY) * Added cell option to crl_info. * Added coral_proto_rule(). * "deny" configuration rule works in cell API, and applications crl_cut, crl_fail, crl_info, crl_print, crl_rate_layer2, crl_time. * Fixed bug in crl_flow where subinterface meta-data was wrong when active flows existed. * Added reset() member function to FlowCounter to return to original state. * Fixed bug in Tables where clear() didn't reset the total counter. * Workaround for g++ library error on FreeBSD 4.1. Version 3.4.1 (2001-02-15) ------------- * Fixed loss of final partial block when duration expires. * Added -s (short format) option to crl_print_pkt. * Fixed -Tm option of crl_flow and crl_rsdos: was flooring expiry time and incorrectly setting gap on interval boundaries. * Added packet, block, and short options to crl_info. * Fixed early stop when reading multiple tracefile interfaces (introduced in version 3.4.0). Version 3.4.0 (members: 2001-01-19) ------------- * Supports pcap traces on stdin with the name "pcap:-". * Supports system network interfaces with the prefix "if:". * Added support for DAG cards (ATM and PoS). * Added support for dagtools file format, making the dagtools "dagcrl" ultility obsolete. * Added support for protocols: Cisco HDLC (over POS) PPP (with PFC) (RFC 1661) PPP over POS (with ACFC) (RFC 1662, 2615) PPP over ATM (RFC 2364) PPP over Ethernet (RFC 2516) Bridged IEEE 802.3/Ethernet over PPP (RFC 1638) * Processing intervals can be specified in fractions of a second on the command line and via the C and perl APIs. * A Coral source on standard input can be gzip'd. * Error file name "-" means stdout. * Eliminated arbitrary limit on number of ATM VPVC's that can simultaneously be in AAL5 reassembly, and reduced memory use for cases where the number of VPVC's is small. * Added the "filter" coral command, to set a pcap filter on any "crl_*" application that operates on packets. * Can be told to search anywhere for extra libraries and include files during installation. Applications * Renamed crl_vpvc and crl_vpvc_layer2 to crl_rate and crl_rate_layer2. * Replaced crl_traffic2 with more flexible crl_flow. * Added crl_dnsstat for collecting statistics on DNS usage. (MEMBERS ONLY) * crl_info -d works on pcap files. * Changed output format of crl_traffic2 to allow for easier reading by CAIDA::Traffic2 libraries (see below). * Fixed crl_traffic2 to not look for layer 4 packet inside IPIP. * crl_print_pkt can print ICMP. * Fixed crl_rate (formerly crl_vpvc) to work on non-ATM links. Output changed slightly. * Added t2_aggregate which turns a tuple table into specified aggregate table type. * Removed t2_top10, use t2_aggregate -n 10 'src_IP_Table' instead. [in 3.5.0, use 't2_convert src_IP_Table | t2_top -n 10'] * coral_print_pkt prints IP and TCP options. libcoral * Added efficient BPF filtering in libcoral. * Improved performance of AAL5 reassembly. * coral_read_pkt() sets the protocol of pkt_result->header and pkt_result->trailer. * Pcap DLT_RAW is equivalent to CoralReef CORAL_NETPROTO_IP when converting to or from a a pcap trace. * Added subiface to coral_pkt_result_t. * Added coral_fprint_data(), coral_fprint_cell(), coral_fprint_pkt(). * Added coral_next_src_iface() to step through interfaces of a source. * Added coral_nth_cell() to find the nth ATM cell in a block. * Added coral_get_iface_stats() to get interval packet statistics for individual interfaces. * Fixed truncation of gigx payload when iomode != user. * Reorganized verbosity levels within libcoral. * Doesn't ignore SIGPIPE. * Added coral_fmt_if_subif() and coral_fmt_subif(). * Added layer 2 PDU counters (l2_recv and l2_drop) to coral_pkt_stats_t. * The coral_get_payload functions, when used on an IP packet that is not the first fragment, will set the payload's protocol to CORAL_PROTO_UNKNOWN. * The coral_get_payload functions can parse ICMP. * Fixed use of "-" as input file. * coral_pkt_to_pcap() skips any encapsulation understood by libcoral but not by libpcap and returns raw IP packet. * Timestamp is optional in coral_pkt_to_pcap(). * The result of coral_iface_to_pcapp() should not be freed by the caller. * Added coral_interface_get_physical(). * coral_fmt_get_payload() prints IP and TCP options. * Removed restriction against CORAL_OPT_SORT_TIME on devices. * Changed prototypes of coral_{pre,post}_interval_handler. perl libraries: * Added make_Proto_Table to CAIDA::Tables::Proto_Ports_Table * Created new modules in the CAIDA::Traffic2 namespace: FileReader, FileWriter, Interval, and SubinterfaceInfo. These are meant to replace Sample and Vpvc_data, and be more flexible, allowing reading and writing of crl_traffic2 'style' output (computer-formatted text or binary output, not human-readable format). * Default behavior when using any of CAIDA::Tables and running out of memory is no longer an abort and core dump, but instead is an error message and exit. (MEMBERS ONLY) * In CAIDA::Tables, renamed top_n_by_* functions to sort_by_*. Defaults to a descending sort (including sort_by_keys()), but can be changed with a second (boolean) argument to specify an ascending sort. * Added sort_by_counter_fields() to CAIDA::Tables as a generalized sort method, to phase out the deprecated sort_by_{pkts|bytes|flows}. * Changed ASFinder::get_as() and ASFinder::get_as_raw() to return the AS (only) when called in a scalar context, to avoid confusion. report generator (t2_report[++]): * Upon receipt of a HUP or USR1 signal, t2_report[++] will reread the configuration file, application ports file, and command line arguments. Note though that the ordering and sequence of initialization remain the same, so there is no way for a new configuration file to override items specified on the command line. To avoid this problem, avoid using the command line for more than minor adjustments. * Changed to use new CAIDA::Traffic2 reading modules (see above) instead of CAIDA::Traffic2::Sample. * Replaced the old calls to the Perl module GifGraph with routines that generate calls to GD::Graph. Updated those routines to create a separate legend so that more pie slices could be displayed. Added new configuration variables to control the pie charts. There is a configuration variable to enable the 3d appearance, but that feature is disabled due to a bug in GD:Graph. * Fixed setting to Nonblocking in netgeo so that it would only be set when netgeo is already active. * Patched the negative values that were being generated for the minimum other applications. The guess is that due to processing glitches, RRDtool is tricked into marking the overall traffic as 0, The applications however, do not undergo such glitches. As a result, the subtraction of the non-zero applications from the overall results in a negative number. Now values less than zero are simply set to 0. * Added support for timeseries graphing the top 'N' applications viewed by t2_report++ per sample interval. RRDtool is used for these plots and for the data storage. The number of applications to be shown is configurable via the configuration file. The applications that can be displayed are those specified in the configuration file (MEMBERS ONLY). * Fixed symlink from X_Y -> X:Y in rrd directory. * Fixed a missing conditional test for indexing the RRDtool graphs of applications. * Added support to disable tracking of Internet Application tracking as provided by the internetapps flag or -i (MEMBERS ONLY) * Removed the hardcoded RRDtool graphs intervals and replaced it with a configuration file option rrd_time_samples. The default intervals are: 1 hour, 1 day, 28 days, and 365 days. libhashtab: * Changed add_hash_entry() to return success (0) or failure (1). Version 3.3.2 (2000-08-17) ------------- * Fixed writing of files larger than 2 GiB (on systems that support it). * Fixed configuration test for perl version (for perl 5.6). * Avoids overflow at 2^31 in perl FlowCounter when using C++ backend, and thus in the t2_* applications. (MEMBERS ONLY) * Fixed reading of pcap files created on machines with different byte order. * Fixed inappropriate alteration of FORE clock after clock reset. drivers: * Fixed cell loss counter in POINT driver. * Recognizes Apptel POINT CPLD version 4.5. applications: * crl_filter aborts if filter expression is invalid. * Eliminated buggy -h option of crl_filter and crl_portmap. * Fixed fatal bug in crl_totsh. libcoral: * Fixed: if coral_pkt_atm_hook returned 0 on a 1-cell PDU and imode > 1 cell, the next PDU on that vp:vc would be incorrectly discarded. * Fixed: libcoral.h (caida_t.h) could incorrectly define SIZEOF_* macros to 0, interfering with user-written apps that used them. * Added "coral_" prefix to pkt_handler, pre_interval_handler, and post_interval_handler to minimize namespace pollution. CAIDA::Tables perl library: * Changed make_Country_byAS to use faster array lookups instead of individual lookups. report generator (t2_report[++]): * Changed t2_report to use faster array lookups in netgeo. Version 3.3.1 (2000-06-26) ------------- * Fixed bug in FATM driver which could cause it to stop returning data on some machines (introduced in 3.3.0). applications: * Fixed ports_script to correctly find installed ports file. * crl_print prints error message and ignores non-ATM sources instead of coring. * Fixed -i option of crl_cut. * Removed obsolete crl_toascii (use crl_print or crl_print_pkt instead). * Changed crl_traffic2 to store flow counts with 64 bit integers and to output data in 64 bit form. report generator (t2_report[++]): * Fixed undercounting of applications in t2_report++ which was a result of a bug in wildcard matching in AppPorts.pm. (MEMBERS ONLY) libcoral * coral_read_pkt() sets the protocol of pkt_result->header and pkt_result->trailer. * Pcap DLT_RAW is equivalent to CoralReef CORAL_NETPROTO_IP when converting to or from a a pcap trace. * Fixed truncation of GIGX payload when iomode != user. * Cell and block reading functions return error if sources are not ATM. * Fixed length of last interval when reading from a pcap file. perl libraries: * Changed to 64 bit integers in Tables save/load functions to match previous change to FlowCounter. applications: * Changed crl_traffic2 to store flow counts with 64 bit integers and to output data in 64 bit form. Version 3.3.0 (2000-05-23) ------------- * Fixed behavior of FATM driver when application doesn't keep up with the traffic: it reports lost cells, and gives the correct cell count when stopped. (In previous versions, incorrect cell counts could cause core dumps.) * Reported intervals now cover exactly the requested period of time, instead of including some "quiet" (trafficless) time. Applications that used intervals (like crl_traffic2) will now report different intervals than previous versions did for the same data, but the aggregate results will be the same. * When reading old tracefile formats that didn't store the capture time, CoralReef can extract capture time from the filename if it's there. * Fixed byte order of ATM headers when reading NLANR and MCI traces. (Incorrect headers caused incorrect vpvc's and discarding of some cells misinterpreted as OAM/RM data). * Fixed conversion of NLANR and MCI traces to current CoralReef file format. * crl_traffic2 does not use mmalloc by default, because it does not work on all systems. Use the --with-mmalloc option of configure to enable it. libcoral: * Duration is now handled by libcoral, so application programmers do not have to handle it (but existing applications that implement it themselves will not have any problem). * Fixed coral_stats() to report current information from driver. * timestamp parameter to pkt_handler prototype is now const. applications: * crl_encode doesn't discard partial blocks. * crl_hist.pl now uses crl_hist_helper instead of crl_portsummary. The new helper program's interface is specific to crl_hist.pl and may be changed in future releases. * Added support for ICMP and IGMP to crl_portsummary. The type and code/max response time values for these protocols are returned in the source and destination port columns, but otherwise the output format has not changed. * Modified t2_IPmatrix and t2_ASmatrix to use CAIDA::Tables instead of local hashes. They accept binary or text input (from crl_traffic2). * Added t2_Proto_Ports. This is another script for summarizing the output of crl_traffic2. * Added -b(ytes), -p(ackets), -f(lows) and -n(umber) options to t2_IPmatrix, t2_ASmatrix, and t2_Proto_Ports. The first three specify the method of sorting (default is just a sorted list of keys). -n specifies how many to output. For example, t2_IPmatrix -b -n 10 will show the top 10 IP matrices, ranked by bytes transferred. * Added -b (binary output) option to crl_traffic2, for use in piping directly to any programs that use the CAIDA::Tables. * Changed output format of crl_traffic2 to not include the cell loss, and thus changed the format version number. * Changed default prefix masklength on crl_traffic2 to be 32 instead of 16. perl libraries: * Added basic support for ICMP to Unpack.pm. The module now supports extraction of type, code, and ICMP checksum data. * Added CAIDA::CRL_report.pm, a new module for reading and writing text summaries output by crl_hist.pl and crl_bycountry.pl * Added get_as_raw() function to ASFinder for accessing binary IP addresses. * Updated Vpvc_data.pm to match the modified output of crl_traffic2. * Changed to 64 bit integers in FlowCounter's C backend to avoid overflow (C backend is MEMBERS ONLY). CAIDA::Tables perl library: * Changed parameters to make_*_Table() functions to accept a hash ref. * Added top_n_by_pkts(), top_n_by_flows(), and top_n_by_keys() * Increased efficiency for sorting many elements. (MEMBERS ONLY) * Added clear() function. * Added save_text(), save_binary(), load_text(), and load_binary(). All take open filehandles as an argument, and save or load tables. (C version (order of magnitude faster): MEMBERS ONLY) * Added make_AS_Table() function to IP_Table. * Added make_Country_Table() function to AS_Table. * Added make_IP_Matrix() and make_AS_Matrix() functions to Tuple_Table. * Added IP_Matrix and AS_Matrix, which have make_AS_Matrix() and make_Country_Matrix(), respectively, as member functions. * Added Country_Matrix. * Added Port_Table. * Added Port_Matrix. * Added make_src_Port_Table() and make_dst_Port_Table() to Tuple_Table, Proto_Ports_Table, and Port_Matrix. * Added make_Port_Matrix() to Tuple_Table and Proto_Ports_Table. * Allowed optional 'protocol' argument to make_src_Port_Table(), make_dst_Port_Table(), and make_Port_Matrix() (for Tuple_Table and Proto_Ports_Table, NOT Port_Matrix) to only aggregate those ports with a specific protocol. * Added make_src_IP_Table() and make_dst_IP_Table() to IP_Matrix. * Added FORCE_C and FORCE_PERL variables to ASFinder and Tables to allow user to specify the backend used in a new(). (MEMBERS ONLY) perl modules: * Created a new Perl module: AppPorts.pm which implements a modest rule-based system for matching application port and protocol number to application name. The module reads in a formatted ASCII text file with the rules and stores them sequentially. The current matching method is return the first matching rule. This allows for some limited flow of control by putting more specific rules before general rules. report generator (t2_report[++]): * t2_report++ is an expanded version of t2_report (MEMBERS ONLY) * Allowed setting of names for each vpvc shown in the top level vpvc summary. * Reports now show tables sorted by packets and flows (tuples) in addition to by bytes. Display has been modified to permit access to all output. * t2_report++ generates new tables indexed by application, unknown TCP ports, and unknown UDP ports. Application is determined by AppPorts.pm using protocol and port information. The default file for rules is ~/.Coral/t2_report.ports, and it can be overriden by the config file parameter AppPorts_rules. (MEMBERS ONLY) * t2_report[++] uses "C" back end for Application tables, ASFinder tables, and Country lookup tables. (MEMBERS ONLY) * Added timeseries graphs for applications as logged by Application ports supplied in t2_report.ports. Graphs are of the stacking bar type, and are accessed by a new list of items in the table menu of report links. * Removed debugging output of (now obsolete) cell loss counter. * Added a command line option -z or --zaptotals which disables the computation of total values for all VPVCs in the data stream. This is useful when the hardware running t2_report is not fast enough to keep up with the traffic2 stream. * Searches for configuration file in $prefix/Coral/etc first, then in the user's home directory. Settings in the user's personal config file override those in the global file. * Fixed report table headers so that the word "encoded" appears only when the IP data is actually encoded. * Fixed a bug that caused the IP encoding to not be applied to the flows display. * Added additional checking for the arguments to configuration settings. * Installed the file example_t2_report.conf in the $prefix/Coral/etc directory. * Fixed a bug that caused the colons between vp:vc pairs to be displayed in hexadecimal. * Fixed a documentation error that caused the configuration table and the command line options table to be omitted from the html format. * Restored the warning statement when t2_report[++] is run without encoding IP addresses. Version 3.2.3 (2000-02-23) ------------- * Fixed bug in CRL API involving cell accessor functions. * Fixed memory leak in *_Table code (Tables were never freed). (MEMBERS ONLY) * Created an App_Table for t2_report. * Fixed hardcoded perl path in building Tables. * Removed suppression of documentation for swigged files. (MEMBERS ONLY) applications: * Fixed handling of null encapsulated IP, and IP in more than one level of encapsulation. libcoral: * Replaced coral_iface_to_pcap() with coral_iface_to_pcapp(), so applications don't need . * Added coral_interface_get_bandwidth(). * Fixed CORAL_RESET bug in driver that appeared when libcoral was optimized. report generator (t2_report): * Added the UserGuide Perl module classes to handle providing diagnostic and usage messages in a modular fashion. The report generator now accepts the command line argument: --help or -h. To obtain a list of all the command line options use --help list. To get information on a particular option use --help