RE: Cflowd vs Netflow vs ....

From: Martin_Nieuwelaar@infonet-europe.com
Date: Fri Jan 26 2001 - 05:13:14 PST

  • Next message: Alex Shavkun: "Re: Cflowd vs Netflow vs ...."

    > -----Original Message-----
    > From: Andrew Kemp [mailto:andrew_kemp@pacific.net.au]
    > Sent: Friday, 26 January 2001 09:52
    > To: cflowd@caida.org
    > Subject: Cflowd vs Netflow vs ....
    >
    >
    >
    > Greetings,
    >
    > We are currently in the process of spec'ing,
    > designing and implementing and box to collect,
    > measure and analyse our network traffic.
    >
    > I was advocating the use of cflowd and related
    > utils for this project, but a couple of other
    > network engineers has raised a concern with
    > "the summarisation that cflowd performs".
    >
    > Another consequence of this summaristaion is that
    > "cflowd was considered deficient as it throws
    > away too much data".
    >
    > I was wondering if others on the list could comment
    > on these issues, explain if they can be worked around
    > and possibly suggest alternative products.
    >
    > Thanks.
    >
    > Regards,
    >
    > Andrew Kemp

    Hi Andrew,

    I think the answer to whether or not cflowd would be suitable can be found
    by answering the question "does cflowd provide the data required?". As I
    see it, it is indeed true that information is lost when the Netflow Exports
    are stored in the ARTS files as used by cflowd. For example, I can look at
    IP to IP traffic flows using artsnets. I can look at AS to AS flows with
    artsases. There is no way however to look at IP to IP traffic for flows
    originating in a particular AS. The limitations essentially come into play
    when you want to combine restrictions on different aspects of the flows.
    Another example would be combining the tcp/udp port number with AS
    infonrmation. If I want to look at AS to AS flows for traffic originating
    or destined for port 80, I can't. Strictly speaking, by using flowwatch I
    can get at views based on whatever weird expression I choose to dream up,
    but then you're only using a very small part of cflowd, and you're missing
    out on all the other good stuff it does. Also the man page recommends
    against using flowwatch as it has high overhead and can cause flows to be
    missed (on a busy collector presumably).

    Other things to be aware of:

    1) If you find cflowd will provide what the reporting you require, will it
    still do so in one years time? Personal experience has shown that initially
    we only used AS to AS flow information. Then we started using the netflows
    stats. Recently we have started using flowdump. Essentially the more we
    know about our traffic the more we want to find out. All this has happened
    over a period of about one year.

    2) If it is necessary to record more than what cflowd records, then maybe
    it's also worth calculating how much space this will take. I know that if I
    tried to record all our Netflow stats it would amount to many many gigabytes
    per day. Processing or doing something meaningful with that much data
    (especially in real-time) is a challenge to say the least.

    Well, just my 2c worth.

    -Martin

    -- 
    "Buying a car because it's reliable is like marrying
    someone because they are punctual" - Jeremy Clarkson
    

    -- cflowd mailing list cflowd@caida.org



    This archive was generated by hypermail 2b29 : Fri Jan 26 2001 - 05:23:03 PST