NFDUMP(1) | General Commands Manual | NFDUMP(1) |
nfdump
— flow
display and analysis program
nfdump |
-r flowpath
[-w outfile]
[-f filterfile]
[-C config]
[-R filelist]
[-M dirlist]
[-O order]
[-t timewin]
[-c num]
[-a ] [-A
aggregation] [-b ]
[-B ] [-I ]
[-D nameserver]
[-G geoDB]
[-s statistic]
[-n num]
[-o format]
[-6 ] [-q ]
[-N ] [-i
ident] [-v
flowfile] [-E
flowfile] [-x
flowfile] [-z ]
[-y ] [-j ]
[-J num]
[-X ] [-Z ]
[-T ] [-V ]
[-h ] [filter] |
nfdump
reads the flow data from one or
more binary files, created by any nfdump collector nfcapd,
nfpcapd and sfcapd. It processes and lists the
flows in many different output formats and can create a wide range of
statistics.
nfdump
has a very powerful flow filter to
process flows. The filter syntax is very similar to tcpdump, but adapted and
extended for flow filtering. A flow filter may also contain arrays of many
thousand IP addresses etc. to search for specific records.
nfdump
can aggreagte flows according to a
user defined number of elements. This masks certain elements and allows to
sum up flow records matching the same values.
The combination of flow filtering and aggregation as input for any
flow statistics allows complex flow processing. Pre-filtered and aggregated
flow data may also be written back into a binary flow file, which again may
be processed with nfdump
nfdump
can enrich the listing of flows
with geo location information and AS information, unless AS information is
already available in the flow records. IP addresses can be tagged with a two
letter country code, or with a longer location label containing the
geographic region, country and city. The geo location and AS information is
retrieved from the optional geoDB database, created by
the geolookup program from the nfdump tools.
geolookup uses the Maxmind
database GeoDB or GeoLite2 to
create a binary lookup database for nfdump
Please
check the geolooup(1) man page for more details.
The options are as follows:
-r
flowpath-w
outfilenfdump
This can be useful
to limit flows according to a flow filter and/or specific flow
aggregation.-f
filterfile-C
confignfdump
tries to read by default
%prefix/etc/nfdump.config. This may be overwritten
by the environment valiable NFCONF which again may
be overwritten by this option -C.
In order to
prevent reading any config file, even if it would exist set
-C
none. A config file is not
required, but may be handy for often used output formats etc.-O
orderflows
packets
ipkg
opkg
bytes
ibyte
obyte
pps
ipps
opps
bps
ibps
obps
bpp
ibpp
obpp
tstart
tend
duration
-t
timewin-c
num-a
-A
-A
aggregationproto
srcip
dstip
srcip4/net
srcip6/net
dstip4/net
dstip6/net
srcnet
dstnet
srcport
dstport
srcmask
dstmask
srcvlan
dstvlan
srcas
dstas
nextas
prevas
inif
outif
next
bgpnext
insrcmac
outdstmac
indstmac
outsrcmac
tos
srctos
dsttos
mpls1
mpls2
mpls3
mpls4
mpls5
mpls6
mpls7
mpls8
mpls9
mpls10
router
odid
opid
xsrcip
xdstip
xsrcport
xdstport
nfdump
automatically compiles the
appropriate output format for the selected aggregation elements unless
an explicit output format -o
is given. The
automatic output format is identical to
-o 'fmt:%ts %td <fields>
%pkt %byt %bps %bpp %fl'
where <fields> represents the selected aggregation tags.
-b
-o
format option.-B
-b
but tries to guess the
correct client to server direction. Automagically swaps flows if src port
is < dst port for TCP and UDP flows and src port < 1024 and dst port
> 1024. Some exporters do not really care sending the flows in proper
order. It's considered to be a conveniency option.-I
-r
flowpath.-g
-r
flowpath a one line summary, which can be easily
used by gnu plot.-D
nameserver-G
geoDBnfdump
tries to read the environment
variable NFGEODB for the path of
geoDB. The option -G
overwrites NFGEODB. In order to prevent reading any
geoDB file, even if it would exist set
-G
none.-s
statistic [:p
[/orderby]]statistic can be:
record
srcip
dstip
ip
nhip
nhbip
router
srcport
dstport
port
tos
srctos
dsttos
dir
srcas
dstas
srcgeo
dstgeo
as
inif
outif
if
inam
onam
srcmask
dstmask
srcvlan
dstvlan
vlan
insrcmac
outdstmac
indstmac
outsrcmac
srcmac
dstmac
inmac
outmac
mask
proto
mpls1
mpls2
mpls3
mpls4
mpls5
mpls6
mpls7
mpls8
mpls9
mpls10
sysid
nbar
ja3
odid
opid
vrf/ivrf
evrf
ivrfnam
evrfnam
NSEL/ASA statistics
event
xevent
xsrcip
xsrcport
xdstip
xdstport
iacl
iace
ixace
eacl
eace
exace
NAT statistics
nevent
nsrcip
nsrcport
ndstip
ndstport
% nfdump -s srcip -s
ip/flows/bytes -s record/bytes
-n
num-s
statistics as well as to
ordered output -O
-or
-aggreated
-records
-a
The default is set to 10 for statistics and
unlimited for the other use cases. To disable the limit, set
num to 0.-o
formathas many
different output formats already predefined.
format
may be one of the options below:
raw
fmt:
userjson
csv
pipe
Already predefined fmt formats:
line
long
biline
bilong
gline
glong
extended
nsel
nel
The nfdump
config file may contain
additional formats. If you want to add new formats or change existing
ones, check the config file.
IPv6 addresses are printed condensed in any
fmt defined
format to prevent cluttering the output with large blank blocks. A
condensed IPV6 uses max 16 characters. If it is longer, then the middle
part of the IP is cut out and replaced be "..". For previewing
an output, this fits most needs. For a listing with the full IPV6
addresses add option -6.
-6
-q
-N
-i
ident-r
to ident-v
flowfile-E
flowfile-x
flowfile-z
-y
-j
-J
num-r
flowpath num: 0
uncompress, 1: LZO1X-1, 2: bz2, 3: LZ4 compression. This option may be
used for archiving flow files and changing the compression to use less
disk space.-X
-Z
-R
filelist-M
dirlist-T
-V
nfdump
version and exit.-h
filter selects, which records will be further processed. If no filter is given, all records will be processed. Otherwise, only those flows matching the filter will be processed. Any IP address in a filter may be specified as IPv4 or IPv6.
The filter syntax is similar to tcpdump but adapted and extended for flow records. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case insensitive.
A single filter primitiv filters a single element of a flow record. A filter consists of one or more primitives, which are linked together:
expr
and expr
expr
or
expr
not
expr and
(expr)
Possible filter primitives:
@include
filecount
comp numberident
stringinet
ipv4
inet6
ipv6
proto
protocoltcp
,
udp
, icmp
,
ah
, esp
,
ipip
, and many more or a protocol number, such as
6, 17 for protocol
tcp and
udp.
tun
proto
protocolip
ipaddrsrc
ip
ipaddrdst
ip
ipaddror
chain. (IP or IP
or IP). If ip
is not specified with
src
or dst
the source or
destination IP may match.host
ipaddrhost
is just a synonym for ip
(See above)
ip
in
[iplist]src in
ip
[iplist]dst
ip
[iplist]or
chain, otherwise use an iplist If
ip
is not specified with
src
or dst
the source or
destination IP may match.
net
network netmasksrc
net
network netmaskdst
net
network netmasknet
network/netbitssrc
net
network/netbitsdst
net
network/netbitsnet
is not specified with
src
or dst
the source or
destination IP may match.
geo
geolocsrc
geo
geocodedst
geo
geocodegeo
is not specified with
src
or dst
the source or
destination geo location code may match.
tunip
ipaddrsrc
tunip
ipaddrdst
tunip
ipaddrtunip
is not
specified with src
or dst
the source or destination tunnel IP may match.
port
comp numsrc
port
comp numdst
port
comp numport
is not specified with
src
or dst
the source or
destination port may match.
port
in
[portlist]src port
in
[portlist]dst port
in
[portlist]port
is not specified with
src
or dst
the source or
destination port may match.
icmp-type
numicmp-code
numproto icmp.
engine-type
numengine-id
numsysid
numnfdump
collector
internal assigned number. See also option -E
above.
if
numin if
numout
if
numif
is
not specified with in
or
out
the input or output interface may match.
as
comp numsrc
as
comp numdst
as
comp numprev
as
comp numnext
as
comp numnfdump
supports 32-bit AS numbers every where.
Without or the source or destination AS may match. See
comp for the comparator details.
as
in
[aslist]src as
in
[aslist]dst as
in
[aslist]prev as
in
[aslist]next as
in
[aslist]as
is not specified with
src, dst, prev
or next
the
source or destination AS may match.
mask
bitssrc
mask
bitsdst
mask
bitsmask
is not
specified with src
or dst
the source or destination mask bits may match.
vlan
numsrc
vlan
numdst
vlan
numvlan
is not specified
with src
or dst
the source
or destination vlan may match.
flags
tcpflagsflags S and not flags
AFRPU
router
ip
ipaddrnext
ip
ipaddrbgpnext
ip
ipaddrmac
macaddrin
mac
macaddrin src
mac
macaddrin dst
mac
macaddrout
mac
macaddrout src
mac
macaddrout dst
mac
macaddrmac
with any combination of a direction specifier as defined by CISCO v9 the
test is limited to those mac
addresses only.
Otherwise multiple matches are possible. Without any specifiers any
mac
address is tested against
macaddr
mpls
labelN
comp numberN
with
N
as mpls label number 1..10 matches
number Filters according a specific number in the
mpls label stack.
mpls
eos
comp numbermpls
expN
comp numberN
with N
as mpls label
number 1..10 matches number
packets
comp numin
packets
comp numout
packets
comp numpackets
and in
packets
is equivalent and describes the number of packets from
source to destination. In case of a bi-directional flow (sent by an
exporter or combined by option --B
) the packet
counter for the reverse flow can be tested with out
packet
bytes
comp numin
bytes
comp numout
bytes
comp numbytes
and in bytes
is
equivalent and describes the number of bytes from source to destination.
In case of a bi-directional flow (sent by an exporter or combined by
option --B
) the byte counter for the reverse flow
can be tested with out bytes
flow
comp numtos
numflowdir
directionduration
comp timepps
comp numbps
comp numbpp
comp numobservation
domain id
comp numberobservation
point id
comp numberyaf
or the nfdump
collector nfpcap can send payload data along the netflow
information. If such
payloads are sent it can be filtered according
the filter primitives below:
payload
content
'string'payload
regex
'regex'payload
regex
'regex' flagsregex
searches over the full payload length. A
' ' byte does not stop the match process.
regex must be quoted with single or double quotes:
'regex' or “regex” The regex engine understands the
following reduced syntax:
flags are otional can be:
payload
ja3
md5stringpayload
ja3 defined
-s
ja3 statistic.
client
latency
comp timeserver
latency
comp timeasa
event
eventasa
event
comp numberasa event
denied
reasonasa
xevent
comp numxip
ipaddrsrc
xip
ipaddrdst
xip
ipaddrxip
is
specified without src
or
dst
both IP addresses may match.
xport
ipaddrsrc
xport
ipaddrdst
xport
ipaddrxport
is
specified without src
or
dst
both ports may match.
xnet
network/masksrc
xnet
network/maskdst
xnet
network/maskxnet
is specified without
src
or dst
both IP
addresses may match.
ingress
ACL
comp numberingress
ACE
comp numberingress
XACE
comp numberegress
ACL
comp numbernat
event
event
nat
event
comp numbernip
ipaddrsrc
nip
ipaddrdst
nip
ipaddrnip
is specified
without src
or dst
both IP
addresses may match.
It Cm nport Ar number
src
nport
numberdst
nport
numbernip
is specified
without src
or dst
both
ports may match.
ingress
vrf
numberpblock
start
comp numberpblock
step
comp numberpblock
end
comp numberport in
pblock
src port
in pblock
dst port
in pblock
=,
==, >, <, >=, <=
To prevent collisions with bash
interpretion, alternative comparators are available: EQ,
LT, GT, LE, GE
If comp is omitted, '==' is assumed.This section describes how output formats are compiled.
nfdump
has a lot of already pre-defined output
formats such as raw, json, csv etc. One line formats
as described for option -o
can be compiled from
various elements of a flow record. As a flow record can contains man
different elements it is often useful to compile an output format for
spcific needs.
The output format is specified by -o
“fmt: string”
string contains the field tags
to be printed as well as other characters if needed. A
tag starts with a %
sign
followed by the field name. tags are separated by
spaces from other tags. Characters or other strings, not starting with a
%
sign are copied literally to the output.
Example:
-o
“fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”
This is the definition of the predined format
line.
It adds the elements tstart
duration protocol source IP address/port followed by the literal
characters -> and destination ip address/port packets,
bytes, flows counter. Depending on the task, different output formats
are required to see the required fields of a flow record. You can either
extend a predefined format or specifiy a new one at the command line.
Example: Extend the predefined format long
with the the IP address of the sending router
-o
“fmt:%long %ra”
Predefined formats can be extended by simply add their name with a
%
sign somewhere in the format string. As described
under the output option -o
nfdump
has already many formats
predefined. Most of the time, these format are good enough. Sometimes you
may need different formats, which can be compiled as described above. In
order to prevent adding the same often used output format each time you run
nfdump
a new output format may be define in the
config file nfdump.conf The file nfdump.conf.dist
contains the definition of the already hard coded formats. These may be
uncommented and changed according to the specific needs. New formats may be
added using the following sytax:
fmt.newname = “fmt:%ts %td %pr
%sap -> %dap %pkt %byt %fl”
with newname any new or existing definition of output formats. Existing formats are overwriten with the new definition.
The following list contains all tags, which are available to compile the output format:
%<format>
%cnt
%nfv
%ts
%tfs
%tsr
%te
%ter
%tr
%trr
%td
%pr
%exp
%eng
%lbl
%sa
%da
%sap
%dap
%gsap
%gdap
%sp
%dp
%it
%ic
%sn
%dn
%nh
%nhb
%ra
%sas
%das
%nas
%pas
%in
%out
%pkt
%ipkt
%opkt
%byt
%ibyt
%obyt
%fl
%flg
%tos
%stos
%dtos
%dir
%smk
%dmk
%fwd
%svln
%dvln
%ismc
%odmc
%idmc
%osmc
%mpls1
%mpls2
%mpls3
%mpls4
%mpls5
%mpls6
%mpls7
%mpls8
%mpls9
%mpls10
%mpls
%bps
%pps
%bpp
%sc
%dc
%sloc
%dloc
%n
%ipl
%opl
%nbid
%ja3
%sni
%nbnam
%odid
%opid
%nfc
%evt
%xevt
%sgt
%msec
%iacl
%eacl
%xsa
%xda
%xsp
%xdp
%xsap
%xdap
%uname
%nevt
%ivrf
%evrf
%nsa
%nda
%nsp
%ndp
%pbstart
%pbend
%pbstep
%pbsize
%cl
%sl
%al
nfdump
processes files created by any
previous version of nfdump 1.6.x with some limitations for versions <
1.6.17. In order to convert flow files to the new 1.7.x binary format use
the following command to read//write files:
% nfdump -r oldfile -w
newfile
Print a statistic about the top 20 IP adresses, once sorted by flows and once by bytes
% nfdump -r flowfile -s
ip/flows/bytes -n 20
Print two statistics, one about the source IP and one about the destination IP address limited to flow with either source or destination port 443
% nfdump -r flowfile -s srcip/bytes
-s dstip/bytes -n 20 'port 443'
Print a statistic about the IP pairs, which exchanged most traffic.
% nfdump -r flowfile -s record/bytes
-A srcip,dstip
Print all flows in raw format with a HTTP header in the payload even if flow is not on port 80.
% nfdump -r flowfile -o raw
“payload regex 'GET|POST'”
Print a statistic about all ja3 md5 sums for those flows, which a valid ja3 can be calculated
% nfdump -r flowfile -s ja5 -n 0
'payload ja3 defined'
Aggregate all flows and write the result back to a binary file, sorted by the start time
% nfdump -r flowfile -a -Otstart -w
newfile
nfdump
returns 0 on success and 255 if
processing failed.
https://www.iana.org/assignments/ipfix/ipfix.xhtml
https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html
No software without bugs! Please report any bugs back to me.
June 19, 2024 | Debian |