cflowd, cflowdmux running but no data

From: Jana Dunn (jana@scs.unr.edu)
Date: Mon Jun 18 2001 - 17:27:34 PDT

  • Next message: Martin, Jeff: "IP Scaling Issue"

    I have cflowdmux, cflowd, and cdfcollector compiled and running
    under Solaris 2.6, but don't see any data--flowdump doesn't show
    any data in the files created in the flows directory and

            flowwatch all

    shows no output.

    Perplexingly enough, I had flowwatch showing data last night, but
    I can't reproduce this even with the cflowd.conf file restored to
    (what I think was) the way I had it at that point.

    Attached is output from various commands plus my cflowd.conf file.

    I'm sure I've just missed something simple.

    Jana Dunn
    Univ. Nevada System
    jana@scsr.nevada.edu

    -------------

    Syslog output:
    myhost:%29 #> cat cflowd.log

    Jun 18 16:55:54 myhost cflowdmux[903]: [I] cflowdmux (version cflowd-2-1-b1) started.
    Jun 18 16:55:54 myhost cflowdmux[903]: [I] created 2101248 byte packet queue shmem segment {CflowdPacketQueue.cc:247}
    Jun 18 16:55:54 myhost cflowdmux[903]: [I] attached to 2101248 byte packet queue at 0xef300000
    Jun 18 16:55:54 myhost cflowdmux[903]: [I] created semaphore: id 0
    Jun 18 16:55:54 myhost cflowdmux[903]: [I] set UDP recv queue to 261040 bytes for fd 4 (port 2055)
    Jun 18 16:55:56 myhost cflowd[907]: [I] cflowd (version cflowd-2-1-b1) started.
    Jun 18 16:55:56 myhost cflowd[907]: [I] got semaphore: id 0
    Jun 18 16:55:56 myhost cflowd[907]: [I] attached to 2101248 byte packet queue at 0xeed00000
    Jun 18 16:55:58 myhost cfdcollect[911]: [I] cfdcollect (version cflowd-2-1-b1) started with 1 cflowd instances.
    Jun 18 16:55:59 myhost cfdcollect[911]: [I] connected to localhost:2056

    Netstat output:
    myhost:%35 #>
    myhost:%35 #> netstat -na | grep 205
          *.2055 Idle
          *.2056 *.* 0 0 0 0 LISTEN
    127.0.0.1.32849 127.0.0.1.2056 32768 0 32768 0 ESTABLISHED
    127.0.0.1.2056 127.0.0.1.32849 32768 0 32768 0 ESTABLISHED

    Shared memory:
    myhost:%36 #> ipcs -a
    IPC status from <running system> as of Mon Jun 18 16:57:03 2001
    Message Queue facility not in system.
    T ID KEY MODE OWNER GROUP CREATOR CGROUP NATTCH SEGSZ CPID LPID ATIME DTIME CTIME
    Shared Memory:
    m 0 0x5000033d --rw-r--r-- root root root root 1 68 203 203 15:42:41 15:42:41 15:42:41
    m 701 0x00005fe6 --rw-r--r-- root other root other 3 2101248 903 907 16:55:56 no-entry 16:55:54
    T ID KEY MODE OWNER GROUP CREATOR CGROUP NSEMS OTIME CTIME
    Semaphores:
    s 0 0x00005fe6 --ra-ra-ra- root root root root 2 16:57:02 15:42:49

    Process IDS:
    myhost:%37 #>
    myhost:%37 #> ps -ef | grep cflow
        root 903 1 0 16:55:54 pts/5 0:00 /usr/local/arts/sbin/cflowdmux /usr/local/arts/etc/cflowd.conf
        root 907 1 0 16:55:56 pts/5 0:00 /usr/local/arts/sbin/cflowd /usr/local/arts/etc/cflowd.conf

    lsof output:
    myhost:%38 #> lsof -p 903
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    cflowdmux 903 root cwd VDIR 0,1 500 201506756 /tmp
    cflowdmux 903 root txt VREG 79,5 5013312 385816 /opt/local/arts/sbin/cflowdmux
    cflowdmux 903 root txt VREG 79,6 1015768 5401 /usr/lib/libc.so.1
    cflowdmux 903 root txt VREG 79,6 16932 472866 /usr/platform/sun4u/lib/libc_psr.so.1
    cflowdmux 903 root txt VREG 79,6 726968 5492 /usr/lib/libnsl.so.1
    cflowdmux 903 root txt VREG 79,6 19304 5420 /usr/lib/libmp.so.2
    cflowdmux 903 root txt VREG 79,6 105788 5465 /usr/lib/libm.so.1
    cflowdmux 903 root txt VREG 79,6 53656 5432 /usr/lib/libsocket.so.1
    cflowdmux 903 root txt VREG 79,6 4304 5602 /usr/lib/libdl.so.1
    cflowdmux 903 root txt VREG 79,6 181840 5397 /usr/lib/ld.so.1
    cflowdmux 903 root 0u VCHR 24,5 0t2643 234156 /devices/pseudo/pts@0:5->ldterm->ptem->pts
    cflowdmux 903 root 1w VREG 79,0 1242 49119 /var/adm/log/cflowd.log
    cflowdmux 903 root 2w VREG 79,0 1242 49119 /var/adm/log/cflowd.log
    cflowdmux 903 root 3w VCHR 21,0 0t0 234127 /devices/pseudo/log@0:conslog->LOG
    cflowdmux 903 root 4u inet 0x609da148 0t0 UDP *:netflow (Idle)

    myhost:%39 #> lsof -p 907
    COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME
    cflowd 907 root cwd VDIR 0,1 500 201506756 /tmp
    cflowd 907 root txt VREG 79,5 5226028 385817 /opt/local/arts/sbin/cflowd
    cflowd 907 root txt VREG 79,13 1000000 979722 /data/CflowData/cflowd/flows/10.10.200.11.flows.0
    cflowd 907 root txt VREG 79,6 1015768 5401 /usr/lib/libc.so.1
    cflowd 907 root txt VREG 79,6 16932 472866 /usr/platform/sun4u/lib/libc_psr.so.1
    cflowd 907 root txt VREG 79,6 726968 5492 /usr/lib/libnsl.so.1
    cflowd 907 root txt VREG 79,6 19304 5420 /usr/lib/libmp.so.2
    cflowd 907 root txt VREG 79,6 105788 5465 /usr/lib/libm.so.1
    cflowd 907 root txt VREG 79,6 53656 5432 /usr/lib/libsocket.so.1
    cflowd 907 root txt VREG 79,6 4304 5602 /usr/lib/libdl.so.1
    cflowd 907 root txt VREG 79,6 181840 5397 /usr/lib/ld.so.1
    cflowd 907 root 0u VCHR 24,5 0t4005 234156 /devices/pseudo/pts@0:5->ldterm->ptem->pts
    cflowd 907 root 1w VREG 79,0 1242 49119 /var/adm/log/cflowd.log
    cflowd 907 root 2w VREG 79,0 1242 49119 /var/adm/log/cflowd.log
    cflowd 907 root 3w VCHR 21,0 0t0 234127 /devices/pseudo/log@0:conslog->LOG
    cflowd 907 root 4u unix 105,29 0t0 234299 /devices/pseudo/tl@0:ticots->/usr/local/arts/etc/cflowdtable.socket (0x60646958) (Vnode=0x6076d5d0)
    cflowd 907 root 5u inet 0x60d85ed8 0t0 TCP *:cflowd (LISTEN)
    myhost:%40 #>

    truss output; apparently no data received by cflowdmux:

    myhost:%28 #> truss -vall -p 903
    poll(0xEFFFCF50, 1, 2000) = 0
            fd=4 ev=POLLRDNORM rev=0
    semop(0, 0xEFFFEEC0, 2) = 0
            semnum=1 semop=0 semflg=SEM_UNDO
            semnum=1 semop=1 semflg=SEM_UNDO
    time() = 992909132
    semop(0, 0xEFFFEF38, 1) = 0
            semnum=0 semop=-1 semflg=IPC_NOWAIT|SEM_UNDO
    sigprocmask(SIG_UNBLOCK, 0xEFFFEF30, 0x00000000) = 0
             set = 0x00000001 0 0 0
    sigprocmask(SIG_BLOCK, 0xEFFFEF30, 0x00000000) = 0
             set = 0x00000001 0 0 0
    poll(0xEFFFCF50, 1, 2000) = 0
            fd=4 ev=POLLRDNORM rev=0
    semop(0, 0xEFFFEEC0, 2) = 0
            semnum=0 semop=0 semflg=SEM_UNDO

    Router output:
    my-router-gw#show ip flow export
    Flow export is enabled
      Exporting flows to 10.10.1.235 (2055)
      Exporting using source interface Loopback0
      Version 5 flow records
      346175 flows exported in 11539 udp datagrams
      0 flows failed due to lack of export packet
      11539 export packets were sent up to process level
      0 export packets were dropped due to no fib
      0 export packets were dropped due to adjacency issues
      0 export packets were dropped due to fragmentation failures
      0 export packets were dropped due to encapsulation fixup failures

    Snippet of router config:

    ip flow-export source Loopback0
    ip flow-export version 5
    ip flow-export destination 10.10.1.235 2055

    Notes:
    "10.10.1.235" is "myhost's" IP address.
    Loopback0 is "10.10.200.11"

    ###########################################################################
    # cflowd.conf - cflowd configuration file
    # $Name: cflowd-2-1-b1 $
    ###########################################################################
    # THIS IS JUST AN EXAMPLE!!! IT MUST BE MODIFIED TO WORK WITH
    # YOUR CFLOWD INSTALLATION!!!
    ###########################################################################

    #---------------------------------------------------------------------------
    # OPTIONS stanza
    # --------------
    # The OPTIONS stanza contains global cflowd options. It must be the
    # first stanza in the configuration.
    #
    # Option fields:
    #
    # LOGFACILITY (Optional, default local6)
    # The syslog facility to use when logging.
    #
    # TCPCOLLECTPORT (Optional, default 2056)
    # The port on which to listen for connections from cfdcollect.
    #
    # PKTBUFSIZE (Optional, default 1048576)
    # The length (in bytes) to use for packet buffering in
    # shared memory.
    #
    # TABLESOCKFILE (Required)
    # The full path to be used for the named socket on which cflowd
    # will listen for connections from local clients (cfdases, et. al.)
    #
    # FLOWDIR (Required if storing raw flows, no default)
    # The directory in which to store memory-mapped raw flow files.
    # These files tend to have high I/O requirements.
    #
    # FLOWFILELEN (Optional, default 1048576)
    # The maximum length of an individual flow file. You should
    # be careful with this value; the file is memory mapped and
    # hence should not be too large (1-2M is reasonable in most
    # cases).
    #
    # NUMFLOWFILES (Optional, default 10)
    # The number of raw flow files to retain per router.
    #
    # MINLOGMISSED (Optional, default 300)
    # The minimum number of perceived dropped flows to cause a
    # syslog() message from cflowd.
    #
    #---------------------------------------------------------------------------

    OPTIONS {
      # syslog to local6 facility.
      LOGFACILITY: local6

      # Listen for connections from cfdcollect on port 2056.
      TCPCOLLECTPORT: 2056

      # Use a 2 megabyte packet buffer in shared memory.
      PKTBUFSIZE: 2097152

      # Use /usr/local/arts/etc/cflowdtable.socket as named stream socket
      # for connections from local clients (cfdases et. al.)
      TABLESOCKFILE: /usr/local/arts/etc/cflowdtable.socket

      # Keep raw flow files in /usr/local/arts/data/cflowd/flows directory.
      FLOWDIR: /usr/local/arts/data/cflowd/flows

      # Each raw flow file should be 1000000 bytes in length.
      FLOWFILELEN: 1000000

      # Keep 10 raw flow files per router.
      NUMFLOWFILES: 10

      # Log total missed flows from a router if it exceeds 1000 between
      # connections from cfdcollect.
      MINLOGMISSED: 1000
    }

    #---------------------------------------------------------------------------
    # COLLECTOR stanza
    # ----------------
    # The collector stanza is used to control access from collector
    # clients (e.g. cfdcollect). Typically you have only one instance
    # of cfdcollect and hence only one COLLECTOR, but you can have as
    # many as you want (for example, if you have a backup host to run
    # cfdcollect when the primary cfdcollect host is down).
    #---------------------------------------------------------------------------

    COLLECTOR {
      HOST: 127.0.0.1 # IP address of central collector
      ADDRESSES: { 127.0.0.1 }
      AUTH: none
    }

    #---------------------------------------------------------------------------
    # CISCOEXPORTER stanza
    # --------------------
    # The CISCOEXPORTER stanza contains information about a Cisco that
    # is expected to export flow data to cflowd.
    #
    # CISCOEXPORTER fields
    # --------------------
    # HOST - The IP address of the exporting Cisco. This is essentially
    # used as an indexing mechanism, to differentiate one Cisco
    # from another.
    #
    # ADDRESSES - addresses of individual interfaces on this Cisco. This
    # allows cflowd to accept packets with a source address of
    # one of the interfaces, but still map the data to this
    # Cisco.
    #
    # CFDATAPORT - the port to listen on for packets arriving from the
    # Cisco via flow-export. This should match the port
    # argument of the 'ip flow-export ...' config line on
    # the Cisco.
    #
    # LOCALAS - This is used to substitute an AS number when cflowd gets
    # data with an AS number of 0. This is a kludge workaround
    # due to prefix cache misses on the Cisco and should be used
    # carefully (set it to 0 to not do substitution).
    #
    # SNMPCOMM - SNMP community for the router. This is used by
    # cflowd to get interface names and IP addresses via SNMP.
    # The community should be enclosed in single quotes.
    #
    # COLLECT - What to save from the flow-export data received by the
    # Cisco. The possible collect options:
    #
    # protocol - IP protocol table (pkts/bytes per protocol...
    # ICMP, UDP, TCP, IGMP, etc.)
    #
    # portmatrix - port matrix. Pkts/bytes from port A to port B.
    #
    # ifmatrix - interface matrix. Pkts/bytes from interface A
    # to interface B.
    #
    # nexthop - nexthop table. Pkts/bytes to each IP next hop.
    #
    # netmatrix - network matrix. pkts/bytes from
    # network A to network B.
    #
    # asmatrix - AS matrix. pkts/bytes from AS A to AS B.
    #
    # tos - TOS (Type Of Service) table. pkts/bytes vs. IP TOS.
    #
    # flows - raw flow data.
    #
    #---------------------------------------------------------------------------

    CISCOEXPORTER {
      HOST: 10.10.200.11 # IP address of Cisco sending data.
      ADDRESSES: { 10.10.200.11, # Addresses of interfaces on Cisco
                      10.10.11.251, # sending data.
                      10.10.220.206, #
                      10.10.220.226, #
                      10.10.220.234, #
                      10.1.4.2 } #
      CFDATAPORT: 2055 # Port on which to listen for data.
      SNMPCOMM: 'xxx' # SNMP community name.
      LOCALAS: NNNN # Local AS of Cisco sending data.
      COLLECT: { protocol, portmatrix, ifmatrix, nexthop, netmatrix,
                      asmatrix, tos, flows }
    }

    --
    cflowd mailing list
    cflowd@caida.org
    



    This archive was generated by hypermail 2b29 : Mon Jun 18 2001 - 17:36:41 PDT