Re: [Cflowd] netflow question - Cisco file format

From: gab.seun jones.ewulomi (seun_ewulomi@hotmail.com)
Date: Fri Mar 21 2003 - 02:21:52 PST

  • Next message: Mark Fullmer: "Re: [Cflowd] netflow question - Cisco file format"

    Hi Edwin,

    My apologies for not replying sooner.

    youre correct that it is quite difficult to install a part-by-part netflow
    system. I actually got cflowd, arts++, flowscan to work but ran into memory
    issues cflowd was running at 95% cpu contantly. Im actually building cflowd
    on another linux box. It was quite unfortunate that cflowd doesnt give per
    subnet statistics. I was told flow-tools could do this. I willbe
    investigating further.

    I actually checked out the Aguri netflow tool. Excellent tool.
    Does it actaully read raw netflow exported data/flows?
    do you have to tell it what file to read?

    Any pointers will be graetly appreciated.

    The problem with netflow is the amount of data it generates. What database
    do use to store aggregated flows e.g. PostgreSQL

    regards,
    gab

    >From: "gab.seun jones.ewulomi" <seun_ewulomi@hotmail.com>
    >To: cflowd@caida.org
    >Subject: Re: [Cflowd] netflow question - Cisco file format
    >Date: Fri, 21 Mar 2003 10:07:25 +0000
    >
    >
    >
    >
    >
    >
    >
    >
    >>From: "Edwin D. Vinas" <edwinv@asti.dost.gov.ph>
    >>To: "gab.seun jones.ewulomi" <seun_ewulomi@hotmail.com>
    >>CC: <cflowd@caida.org>
    >>Subject: Re: [Cflowd] netflow question - Cisco file format
    >>Date: Fri, 14 Mar 2003 08:45:28 +0800
    >>
    >>hi gab,
    >> >
    >> > Im currently on the verge of installing cflowd and flowscan. your
    >>website
    >> > and netflow implementation has given a breath of encouragement. I
    >>wanted
    >>to
    >> > give up.
    >> >
    >>Thanks, but don't give up. Its really quite difficult to install a
    >>part-by-part netflow system. I mean cflowd, arts++, flowscan, flowsql,
    >>flow-export configuration, generating summaries etc etc. I think this is
    >>the
    >>price of using all open-source packages instead of commercial netflow
    >>software.
    >>
    >> > 1)Can/Does cflowd/flowscan show per subnet statistics or
    >> > is flow-tools capable of this
    >> >
    >>
    >>I think there are other tools that can do this. Im not sure which one.
    >>However, you can check Aguri netflow tool. It can generate statistics for
    >>each subnet or IPs. I haven't tried generating subnet statistics though.
    >>
    >> > 2)The top summaries how is this generated. what other tools have you
    >> > installed/integarted with netflow to get/generate this data in the
    >>tabular
    >> > format
    >> >
    >>To generate these summaries, I used PHP/Perl to query and summarize the
    >>top
    >>summaries from the netflow database generated by FlowSQL.
    >>Im also still searching for better methods to incorporate in my netflow
    >>system coz it consumes too much disk space. Also, i will still have to
    >>make an automatic netflow analyzer that could detect spamming, dos attacks
    >>and emails a summary report at a regular interval. Right now, Im
    >>just collecting from a main gateway router and a single day database
    >>consumes at least 500MB of my database. This database is the detailed
    >>database which we use for future forensics (i.e., tracing and analyzing
    >>data
    >>sources, protocols, spammers, etc).
    >>
    >>best regards,
    >>edwin
    >>
    >> >
    >> >
    >> >
    >> >
    >> >
    >> >
    >> >
    >> > >From: "Edwin D. Vinas" <edwinv@asti.dost.gov.ph>
    >> > >To: "Vladimir Jirasek" <Vladimir.Jirasek@t-mobile.co.uk>
    >> > >CC: <cflowd@caida.org>
    >> > >Subject: Re: [Cflowd] netflow question - Cisco file format
    >> > >Date: Thu, 13 Mar 2003 08:21:41 +0800
    >> > >
    >> > >MessageHi,
    >> > >
    >> > >If you want collect netflow from routers, it is possible to use
    >>Cflowd.
    >> > >Cflowd has two components -- cflowdmux & cfdcollect. When cflowd is
    >> > >running, it will collect raw flow files version 5 format from
    >> > >flow-exporters and saves the raw flow files in arts++ format. In our
    >>case
    >> > >we are using Cflow to analyze these raw flow files. To graph the data
    >>you
    >> > >can use FlowScan and to database it we used a custom program called
    >>FlowSQL
    >> > >which stores the granular flow fields in a Postgresql database. This
    >>is
    >>an
    >> > >example implemenation: http://noc.asti.dost.gov.ph/netflow/index.php
    >>Docs:
    >> > >http://netmeas.asti.dost.gov.ph/docus/netflow/Netflow.pdf
    >> > >
    >> > >HTH :-)
    >> > >
    >> > >best regards,
    >> > >--edwin
    >> > >
    >> > >-----------------------------------------------------------------
    >> > >If Americans have atomic bombs & the Internet...
    >> > >Filipinos are very far behind to catch up in any field.
    >> > >-Edwin D. Viņas
    >> > >edwinv@asti.dost.gov.ph
    >> > >http://www.geocities.com/edwin_vinas
    >> > >Science Research Specialist I
    >> > >PREGINET Project
    >> > >Advanced Science and Technology Institute
    >> > >UP Technopark Complex, CP Garcia Ave, Diliman,
    >> > >Quezon City Philippines
    >> > >-----------------------------------------------------------------
    >> > >~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    >> > >This communication is intended only for the person or entity to which
    >>it
    >>is
    >> > >addressed and may contain confidential and/or privileged material. If
    >>you
    >> > >are not the intended recipient, please note that any review,
    >> > >retransmission,
    >> > >dissemination, copying or other use of, or taking of any action in
    >>reliance
    >> > >upon, this information by you or by persons or entities other than the
    >> > >intended recipient is prohibited.
    >> > >~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    >> > >
    >> > > ----- Original Message -----
    >> > > From: Vladimir Jirasek
    >> > > To: 'cflowd@caida.org'
    >> > > Sent: Wednesday, March 12, 2003 10:04 PM
    >> > > Subject: [Cflowd] netflow question - Cisco file format
    >> > >
    >> > >
    >> > > Hi,
    >> > >
    >> > > I have developed the tool that can read text files from Cisco
    >>Netflow
    >> > >collector via ftp and analyse it down to application flow level. Now I
    >>want
    >> > >to setup something similar but using Cflowd. I would like to use only
    >> > >collector function and get raw data prefferably in Cisco format.
    >> > > Is this achievable?
    >> > >
    >> > > Many thanks
    >> > >
    >> > > Vladimir Jirasek
    >> > > Mobile: +447956542287
    >> > > Fixed line: +442082142813
    >> > > International Workgroup Corporate network (EU153)
    >> > > T-Mobile International
    >> > > Imperial place, Borehamwood, WD61EA
    >> > > United Kingdom
    >> > >
    >> > >
    >> > >
    >> > >
    >> > >
    >> > > NOTICE AND DISCLAIMER:
    >> > >
    >> > > This email (including attachments) is confidential. If you have
    >>received
    >> > >this email in error please notify the sender immediately and delete
    >>this
    >> > >email from your system without copying or disseminating it or placing
    >>any
    >> > >reliance upon its contents. We cannot accept liability for any
    >>breaches
    >>of
    >> > >confidence arising through use of email. Any opinions expressed in
    >>this
    >> > >email (including attachments) are those of the author and do not
    >> > >necessarily reflect our opinions. We will not accept responsibility
    >>for
    >>any
    >> > >commitments made by our employees outside the scope of our business.
    >>We
    >>do
    >> > >not warrant the accuracy or completeness of such information.
    >> > >
    >> > >
    >> > >
    >> >
    >> >
    >> > _________________________________________________________________
    >> > Worried what your kids see online? Protect them better with MSN 8
    >> >
    >>http://join.msn.com/?page=features/parental&pgmarket=en-gb&XAPID=186&DI=1059
    >> >
    >>
    >
    >
    >_________________________________________________________________
    >Use MSN Messenger to send music and pics to your friends
    >http://messenger.msn.co.uk
    >
    >_______________________________________________
    >Cflowd mailing list
    >Cflowd@caida.org
    >http://login.caida.org/mailman/listinfo/cflowd

    _________________________________________________________________
    Express yourself with cool emoticons http://messenger.msn.co.uk

    _______________________________________________
    Cflowd mailing list
    Cflowd@caida.org
    http://login.caida.org/mailman/listinfo/cflowd



    This archive was generated by hypermail 2.1.4 : Fri Mar 21 2003 - 02:37:16 PST