Re: [Cflowd] netflow question - Cisco file format

From: Mark Fullmer (maf@eng.oar.net)
Date: Fri Mar 21 2003 - 09:42:00 PST

  • Next message: patricia.kan@HydroOne.com: "[Cflowd] Output Bytes from Arts++"

    flow-tools can do per subnet stats. It can also be used as a collector
    for FlowScan. See http://www.splintered.net/sw/flow-tools

    mark

    On Fri, Mar 21, 2003 at 10:21:52AM +0000, gab.seun jones.ewulomi wrote:
    >
    > Hi Edwin,
    >
    > My apologies for not replying sooner.
    >
    > youre correct that it is quite difficult to install a part-by-part netflow
    > system. I actually got cflowd, arts++, flowscan to work but ran into memory
    > issues cflowd was running at 95% cpu contantly. Im actually building cflowd
    > on another linux box. It was quite unfortunate that cflowd doesnt give per
    > subnet statistics. I was told flow-tools could do this. I willbe
    > investigating further.
    >
    > I actually checked out the Aguri netflow tool. Excellent tool.
    > Does it actaully read raw netflow exported data/flows?
    > do you have to tell it what file to read?
    >
    > Any pointers will be graetly appreciated.
    >
    > The problem with netflow is the amount of data it generates. What database
    > do use to store aggregated flows e.g. PostgreSQL
    >
    > regards,
    > gab
    >
    >
    >
    >
    >
    >
    > >From: "gab.seun jones.ewulomi" <seun_ewulomi@hotmail.com>
    > >To: cflowd@caida.org
    > >Subject: Re: [Cflowd] netflow question - Cisco file format
    > >Date: Fri, 21 Mar 2003 10:07:25 +0000
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >
    > >>From: "Edwin D. Vinas" <edwinv@asti.dost.gov.ph>
    > >>To: "gab.seun jones.ewulomi" <seun_ewulomi@hotmail.com>
    > >>CC: <cflowd@caida.org>
    > >>Subject: Re: [Cflowd] netflow question - Cisco file format
    > >>Date: Fri, 14 Mar 2003 08:45:28 +0800
    > >>
    > >>hi gab,
    > >> >
    > >> > Im currently on the verge of installing cflowd and flowscan. your
    > >>website
    > >> > and netflow implementation has given a breath of encouragement. I
    > >>wanted
    > >>to
    > >> > give up.
    > >> >
    > >>Thanks, but don't give up. Its really quite difficult to install a
    > >>part-by-part netflow system. I mean cflowd, arts++, flowscan, flowsql,
    > >>flow-export configuration, generating summaries etc etc. I think this is
    > >>the
    > >>price of using all open-source packages instead of commercial netflow
    > >>software.
    > >>
    > >> > 1)Can/Does cflowd/flowscan show per subnet statistics or
    > >> > is flow-tools capable of this
    > >> >
    > >>
    > >>I think there are other tools that can do this. Im not sure which one.
    > >>However, you can check Aguri netflow tool. It can generate statistics for
    > >>each subnet or IPs. I haven't tried generating subnet statistics though.
    > >>
    > >> > 2)The top summaries how is this generated. what other tools have you
    > >> > installed/integarted with netflow to get/generate this data in the
    > >>tabular
    > >> > format
    > >> >
    > >>To generate these summaries, I used PHP/Perl to query and summarize the
    > >>top
    > >>summaries from the netflow database generated by FlowSQL.
    > >>Im also still searching for better methods to incorporate in my netflow
    > >>system coz it consumes too much disk space. Also, i will still have to
    > >>make an automatic netflow analyzer that could detect spamming, dos attacks
    > >>and emails a summary report at a regular interval. Right now, Im
    > >>just collecting from a main gateway router and a single day database
    > >>consumes at least 500MB of my database. This database is the detailed
    > >>database which we use for future forensics (i.e., tracing and analyzing
    > >>data
    > >>sources, protocols, spammers, etc).
    > >>
    > >>best regards,
    > >>edwin
    > >>
    > >> >
    > >> >
    > >> >
    > >> >
    > >> >
    > >> >
    > >> >
    > >> > >From: "Edwin D. Vinas" <edwinv@asti.dost.gov.ph>
    > >> > >To: "Vladimir Jirasek" <Vladimir.Jirasek@t-mobile.co.uk>
    > >> > >CC: <cflowd@caida.org>
    > >> > >Subject: Re: [Cflowd] netflow question - Cisco file format
    > >> > >Date: Thu, 13 Mar 2003 08:21:41 +0800
    > >> > >
    > >> > >MessageHi,
    > >> > >
    > >> > >If you want collect netflow from routers, it is possible to use
    > >>Cflowd.
    > >> > >Cflowd has two components -- cflowdmux & cfdcollect. When cflowd is
    > >> > >running, it will collect raw flow files version 5 format from
    > >> > >flow-exporters and saves the raw flow files in arts++ format. In our
    > >>case
    > >> > >we are using Cflow to analyze these raw flow files. To graph the data
    > >>you
    > >> > >can use FlowScan and to database it we used a custom program called
    > >>FlowSQL
    > >> > >which stores the granular flow fields in a Postgresql database. This
    > >>is
    > >>an
    > >> > >example implemenation: http://noc.asti.dost.gov.ph/netflow/index.php
    > >>Docs:
    > >> > >http://netmeas.asti.dost.gov.ph/docus/netflow/Netflow.pdf
    > >> > >
    > >> > >HTH :-)
    > >> > >
    > >> > >best regards,
    > >> > >--edwin
    > >> > >
    > >> > >-----------------------------------------------------------------
    > >> > >If Americans have atomic bombs & the Internet...
    > >> > >Filipinos are very far behind to catch up in any field.
    > >> > >-Edwin D. Viņas
    > >> > >edwinv@asti.dost.gov.ph
    > >> > >http://www.geocities.com/edwin_vinas
    > >> > >Science Research Specialist I
    > >> > >PREGINET Project
    > >> > >Advanced Science and Technology Institute
    > >> > >UP Technopark Complex, CP Garcia Ave, Diliman,
    > >> > >Quezon City Philippines
    > >> > >-----------------------------------------------------------------
    > >> > >~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    > >> > >This communication is intended only for the person or entity to which
    > >>it
    > >>is
    > >> > >addressed and may contain confidential and/or privileged material. If
    > >>you
    > >> > >are not the intended recipient, please note that any review,
    > >> > >retransmission,
    > >> > >dissemination, copying or other use of, or taking of any action in
    > >>reliance
    > >> > >upon, this information by you or by persons or entities other than the
    > >> > >intended recipient is prohibited.
    > >> > >~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    > >> > >
    > >> > > ----- Original Message -----
    > >> > > From: Vladimir Jirasek
    > >> > > To: 'cflowd@caida.org'
    > >> > > Sent: Wednesday, March 12, 2003 10:04 PM
    > >> > > Subject: [Cflowd] netflow question - Cisco file format
    > >> > >
    > >> > >
    > >> > > Hi,
    > >> > >
    > >> > > I have developed the tool that can read text files from Cisco
    > >>Netflow
    > >> > >collector via ftp and analyse it down to application flow level. Now I
    > >>want
    > >> > >to setup something similar but using Cflowd. I would like to use only
    > >> > >collector function and get raw data prefferably in Cisco format.
    > >> > > Is this achievable?
    > >> > >
    > >> > > Many thanks
    > >> > >
    > >> > > Vladimir Jirasek
    > >> > > Mobile: +447956542287
    > >> > > Fixed line: +442082142813
    > >> > > International Workgroup Corporate network (EU153)
    > >> > > T-Mobile International
    > >> > > Imperial place, Borehamwood, WD61EA
    > >> > > United Kingdom
    > >> > >
    > >> > >
    > >> > >
    > >> > >
    > >> > >
    > >> > > NOTICE AND DISCLAIMER:
    > >> > >
    > >> > > This email (including attachments) is confidential. If you have
    > >>received
    > >> > >this email in error please notify the sender immediately and delete
    > >>this
    > >> > >email from your system without copying or disseminating it or placing
    > >>any
    > >> > >reliance upon its contents. We cannot accept liability for any
    > >>breaches
    > >>of
    > >> > >confidence arising through use of email. Any opinions expressed in
    > >>this
    > >> > >email (including attachments) are those of the author and do not
    > >> > >necessarily reflect our opinions. We will not accept responsibility
    > >>for
    > >>any
    > >> > >commitments made by our employees outside the scope of our business.
    > >>We
    > >>do
    > >> > >not warrant the accuracy or completeness of such information.
    > >> > >
    > >> > >
    > >> > >
    > >> >
    > >> >
    > >> > _________________________________________________________________
    > >> > Worried what your kids see online? Protect them better with MSN 8
    > >> >
    > >>http://join.msn.com/?page=features/parental&pgmarket=en-gb&XAPID=186&DI=1059
    > >> >
    > >>
    > >
    > >
    > >_________________________________________________________________
    > >Use MSN Messenger to send music and pics to your friends
    > >http://messenger.msn.co.uk
    > >
    > >_______________________________________________
    > >Cflowd mailing list
    > >Cflowd@caida.org
    > >http://login.caida.org/mailman/listinfo/cflowd
    >
    >
    > _________________________________________________________________
    > Express yourself with cool emoticons http://messenger.msn.co.uk
    >
    > _______________________________________________
    > Cflowd mailing list
    > Cflowd@caida.org
    > http://login.caida.org/mailman/listinfo/cflowd

    _______________________________________________
    Cflowd mailing list
    Cflowd@caida.org
    http://login.caida.org/mailman/listinfo/cflowd



    This archive was generated by hypermail 2.1.4 : Fri Mar 21 2003 - 09:56:04 PST