Logo

Upgrading from previous RRD files

Upgrading from previous RRD files (post-CoralReef 3.5)

This guide assumes the RRD files were created by t2_report[++], and additionally that they came from a version of CoralReef after 3.5. If you want to convert from pre-3.5 RRD files, see the instructions for very old files below. In order to convert an existing RRD directory structure from the old format to the new one, several steps are required:
  • Make sure you have the latest version of rrdtool (at least 1.2) installed and in your path.
  • cd into each subinterface of the main RRD directory (eg cd my_RRDs/0[0]/).
  • Run the script rrd_v2_to_v3.sh (found in the installed Coral/bin directory).
  • Returning to the main RRD directory, rename subinterfaces to monitor names, according to the existing subinterface map.
  • You can delete the top-level version file, as one has been created in each monitor subdirectory.
In addition, changes in application names cause different RRDs to be written; if using the CoralReef example applications file, you will want to rename and merge a few RRDs: 
EoA.rrd -> AoE.rrd
WinMX_UDP.rrd -> WINMX.rrd
BLACKWHITE_TCP.rrd -> BLACKWHITE.rrd
BLACKWHITE_UDP.rrd -> GAMESPY_ARCADE.rrd
REALAUDIO_TCP.rrd and REALAUDIO_UDP.rrd -> REALPLAYER.rrd
EDONKEY_TCP.rrd and EDONKEY_UDP.rrd -> EDONKEY.rrd
IMESH_CTL.rrd and IMESH_DTA.rrd -> IMESH.rrd
The RealPlayer, eDonkey and iMesh RRDs can be joined by the same merge_protos script mentioned below (after being dumped to XML), with the caveat that merge_protos will fail if the RRDs were last updated at different times. Other programs exist to merge RRDs, but have not been tested on report generator files.

rrd_v2_to_v3.sh will also will rename the following RRDs: 

OTHER_TCP.rrd -> UNKNOWN_TCP.rrd
OTHER_UDP.rrd -> UNKNOWN_UDP.rrd
OTHER_1.rrd -> UNKNOWN_ICMP.rrd
TCP_NOPORTS.rrd -> NOPORTS_TCP.rrd
UDP_NOPORTS.rrd -> NOPORTS_UDP.rrd
All of the unrecognized protocols, like OTHER_18.rrd, for example, will instead be named UNKNOWN_PROTO_18.rrd.

Upgrading from very old files (pre-CoralReef 3.5)

The very old directory structure consisted of one subdirectory for each vp:vc pair (assuming ATM), like: 
RRD_dir/1:16/
RRD_dir/2:16/
... etc.
Each subdirectory then had its own subdirectories for the actual RRDs: 
RRD_dir/1:16/app/
RRD_dir/1:16/proto/
RRD_dir/1:16/top_n_app_bytes/
RRD_dir/1:16/top_n_app_flows/
RRD_dir/1:16/top_n_app_packets/
The app directory held RRDs for apps specified in the t2_report config file, the proto directory held RRDs for protocols specified in the t2_report config file, and the top_n_app_* directories held all the applications seen while running t2_report. Because the top_n_app_* directories will most likely contain most of the applications seen on the link, you can simply copy all the RRDs from one of them into app: 
top_n_app_bytes/* -> app/*
After copying them, the top_n_app_* directories can be removed. The older format also had no total.rrd, so you must manually create it. To do this, you must dump all of the protocol RRDs into XML format, run a conversion script (named merge_protos.pl, which is installed with the other CoralReef applications) to add them up into a single XML file, and turn that back into an RRD. For example: 
rrdtool dump 1:16/proto/6.rrd > 1:16/6.xml
rrdtool dump 1:16/proto/17.rrd > 1:16/17.xml
rrdtool dump 1:16/proto/1.rrd > 1:16/1.xml
merge_protos.pl 1:16/*.xml > 1:16/total.xml
rrdtool restore 1:16/total.xml 1:16/total.rrd
Due to changes in the application mapping file, the RRDs in the app directory need to be renamed as follows: 
MEDIA_NETSHOW_MS.rrd -> MS_MEDIA.rrd
AIM.rrd -> AOL.rrd
ICMPECHOREPLY.rrd -> ICMP_ECHOREPLY.rrd
ICMPECHOREQUEST.rrd -> ICMP_ECHO.rrd
ICMPTTL.rrd -> ICMP_TIMXCEED_INTRANS.rrd
ICMPNOREACH.rrd -> ICMP_UNREACH_HOST.rrd
At this point, you can follow the same directions as with post-3.5 files.

Related Objects

See https://catalog.caida.org/software/coralreef/ to explore related objects to this document in the CAIDA Resource Catalog.