Passive Monitor: equinix-sanjose
The equinix-sanjose Internet data collection monitor is located at an Equinix datacenter in San Jose, CA, and is connected to a backbone link of a Tier1 ISP between San Jose, CA and Los Angeles, CA. Prior to June 2010 this was an OC192 link (9953 Mbps); currently it is a 10GigE link. This ISP has multiple links between these cities. Load balancing is done per flow.
For the listing of CAIDA's active data and passive data monitors, see the CAIDA Data Monitors page.
Capture suspended on San Jose monitor
At the end of September 2014 the San Jose to Los Angeles links were upgraded to 100Gig. Since our capture hardware is not capable of tapping these new links the San Jose monitor is no longer capturing data. The last monthly trace was taken on September 18, 13:00-14:00 UTC; the report generator stopped at approximately 20:00 UTC on September 18. We are currently investigating our options for reviving this link.
The infrastructure consists of 2 physical machines, numbered 1 and 2. Both machines have a single Endace 6.2 DAG network monitoring card. A single DAG card is connected to a single direction of the bi-directional backbone link. The directions have been labeled A and B. Both machines have 2 Intel Dual-Core Xeon 3.00GHz CPUs, with 8 GB of memory and 1.3 TB of RAID5 data disk, running Linux 2.6.15 and DAG software version dag-188.8.131.52. In a test environment both machines dropped less then 1% of packets with snaplen 48 at 100% OC192 line utilization, using a Spirent X/4000 packet generator sending packets with a quadmodal distribution, with peaks at 40, 576, 1500 and 4283 bytes.
Realtime Traffic Report Generator
Realtime traffic reporting for this monitor was available at
- https://www.caida.org/data/realtime/passive/?monitor=equinix-sanjose-dirA (direction A)
- https://www.caida.org/data/realtime/passive/?monitor=equinix-sanjose-dirB (direction B)
crl_anf -b -A 500000 -f 500000 -Calignint -O %flow_file% -Cm=phy=pos %measurement_card%
-b option outputs a binary format, for increased
efficiency in writing and storage. The
options set the maximum number of entries in the tables for counting using
the ANF and FCE algorithms, respectively. (Note: as of April 20th, 2008,
this functionality is not yet part of a CoralReef release, as it has not
been fully tested and documented.)
-Calignint forces the
(5-minute) intervals to align along 5-minute clock boundaries, so as to
make it easier to compare intervals from different monitors, and
-O specifies the output filename. The rest is for configuring
the DAG card.
The first passive traffic traces on this monitor were taken July 2008. Since then each month a 1-hour trace has been recorded for both directions. The monthly traces from this monitor and the equinix-chicago monitor are available to researchers (in anonymized form) in the CAIDA passive trace datasets:
- CAIDA Anonymized 2008 Internet Traces Dataset,
- CAIDA Anonymized 2009 Internet Traces Dataset,
- CAIDA Anonymized 2010 Internet Traces Dataset.
- CAIDA Anonymized 2011 Internet Traces Dataset.
- CAIDA Anonymized 2012 Internet Traces Dataset.
- CAIDA Anonymized 2013 Internet Traces Dataset.
- CAIDA Anonymized Internet Traces on IPv6 Day and IPv6 Launch Day Dataset.
- CAIDA Anonymized 2014 Internet Traces Dataset.
Both physical machines are configured to synchronize their hardware clock via NTP. The DAG measurement cards have their own internal high-precision clock, that allows it to timestamp packets with 15 nanosecond precision. Every time a traffic trace is taken, The DAG internal clock gets synchronized to the host hardware clocks right before measurement starts. NTP accuracy is typically in the millisecond range, so at initialization the clocks on the individual DAG measurement cards can be off by a couple of milliseconds relative to each other. The precision (ie. timing within the packet trace) within a single direction of trace data is 15 nanosecond for the DAG files, and 1 microsecond for PCAP files. Due to human error the DAG internal clock was not synchronized with the system hardware clock for the first passive traffic trace (July 2008).
Starting in August 2008 the clocks on the DAG measurement cards are synchronized between the two measurement machines. At the beginning of each measurement experiment, the DAG clocks are synchronized to the individual machine clocks once, but one DAG card also feeds a 1 pulse per second signal to the other card, which allows them to be synchronized up with approximately 2 clock ticks precision (30 nanoseconds).