RE: [Cflowd] Re: [flow-tools] Typical Hardware

From: Williams, Neil (neil.williams@boeing.com)
Date: Tue Feb 11 2003 - 08:33:57 PST

  • Next message: Scott Sheppard: "[Cflowd] RE: [flow-tools] Support of sampled CFLOWD"

    Interesting. We collect on about 20 (backbone) routers (out of ~1000 other routers), streaming to 6 sun 280's. We run artnets, artsports, artsases and artprotos reports. Initially it took 12 hours to process the data, but by reading in some key files into memory (/etc/protos, /etc/services) and a couple of other code tweaks to the report engines, we are able to reduce processing time from 12 hours to about 5 minutes.

    We don't use it for real-time either, just for trending and analysis. Actually, I would consider it to be more of an inventory mechanism rather than a diagnostic tool. An analyst needs to apply more intensive (but less scaleable) tools such as ecoScope or sniffers to get better real time and/or behavioral understanding of the applications.

    Unfortunately, the cflowd tools are proving inadequate because they don't allow (us) enough control over how the data gets summarized. We are ultimately going to leverage the Cisco collector/analyzer package (which is relatively cheap) to get better processing control.

    I have resisted posting to this group, but it was interesting to see an architecture similar to ours.

    Neil Williams
    Network Performance Architect
    neil.williams@boeing.com
    Boeing

    -----Original Message-----
    From: Craig A. Finseth [mailto:fin@finseth.com]
    Sent: Tuesday, February 11, 2003 6:47 AM
    To: flow-tools@splintered.net; cflowd@caida.org
    Subject: [Cflowd] Re: [flow-tools] Typical Hardware

    >>I would like to do an informal survey of the type of hardware that
    >>people on this list are using for running flowtools and or cflow

    We collect flows from about 19 routers using about a dozen small Suns.
    Data is forwarded to a single Sun 280 with dual 900 MHz processors, 1
    GByte of memory and a half-terabyte of raid disk.

    Total raw data size is about 4 GBytes/day, compressed.

    It takes 12-18 hours to process one days' data.

    We use the data for usage and performance reasons, not real-time
    attack analysis.

    Craig

    _______________________________________________
    Cflowd mailing list
    Cflowd@caida.org
    http://login.caida.org/mailman/listinfo/cflowd

    _______________________________________________
    Cflowd mailing list
    Cflowd@caida.org
    http://login.caida.org/mailman/listinfo/cflowd



    This archive was generated by hypermail 2.1.4 : Tue Feb 11 2003 - 08:45:39 PST