NetFlow Software

NFDUMP and NfSen
NFDUMP is a set of tools to capture/record, dump,
filter, and replay NetFlow (v5/v7/9) data. Can filter flows according
to multiple user-defined profiles. NfSen is a Graphical
Web-based front-end for the NFDUMP tools. Plots aggregate statistics
over time, supports filtering and drilling down up to the individual
flow level.
CoMo
Traffic monitoring toolkit from Intel Research. Supports both
continuous real-time processing and retrospective processing.
Supports Netflow and many other traffic capture sources.
YAF – Yet
Another Flow sensor
YAF snoops packets from pcap dump files or live capture,
and produces bidirectional flows. These flows can be sent to
IPFIX collectors, or be stored in
an IPFIX-derived file format.
VERMONT (VERsatile MONitoring
Toolkit)
A reference implementation of the IPFIX and PSAMP protocols
developed as part of the HISTORY project at the
German universities of Erlangen and Tübingen, and of the European
DIADEM Firewall
project.
libipfix
A C library that implements the IPFIX protocol.
libfixbuf
Aims to be a compliant implementation of the IPFIX protocol message format, from
which fully compliant IPFIX Collecting Processes and IPFIX Exporting
Processes may be built. In addition of the IPFIX Protocol, libfixbuf
supports efficient persistent storage of IPFIX data using the method
outlined in draft-trammell-ipfix-file-NN.
NetSA Aggregated Flow (NAF)
toolchain
Tools for creating and analyzing timeslice-organized
bidirectional flow files in the IPFIX-inspired NAF
format.
FlowScan
A Perl-based system to analyze and report on flows collected by
flow-tools, lfapd or cflowd, by Dave Plonka. Sample output graphs are
available too, as well as Majordomo-driven mailing
lists
for announcements and general discussion (archive).
It is currently built on Cflow.pm.
User-contributed tools based on FlowScan include:

CarrierIn
from Stanislav Sinyagin
which claims to be more suitable for larger ISP/Carriers
CUFlow
from Matt Selsky and Johan M. Andersen at Columbia University
which is an alternative graphing tool "designed to combine
the features of CampusIO and SubNetIO". Robert S. Galloway has
contributed a nice howto-style
document
describing how it can be used.
FlowMonitor
from Johan M. Andersen at Columbia University
monitors individual users’ network usage against a bandwidth
usage policy.
JKFlow
by Jurgen Kobierczynski
A new reporting module which is highly configurable using an
XML configuration file.
FlowScan+
An extension to FlowScan developed by KISTI/KAIST. Adds
servlet-based visualization and support for queries for top
user, AS, port, protocol, etc. This is supposed to be available
under http://flowscan.kreonet2.net/,
but that site doesn’t seem to be responsive.
flow-tools
Similar to cflowd but implemented
as a set of smaller tools, with the addition of compression of the
recorded data, thus capable of recording many more flows in a given
amount of disk space. See paper
about its application for Intrusion Detection. There is also a mailing
list
for the package.

There is a short presentation called Ohio
Gigapop Traffic Measurements
that shows some examples on how
flow-tools can be used.

The package is widely used, and there are quite a few user
contributions, such as

FlowViewer
Web-interface to flow-tools. Consists
of three tools: FlowViewer provides the user with web access
to many of the textual and statistical flow-tools reports.
FlowGrapher provides a web page with a graph of the selected
flow data. These web pages can be saved. FlowTracker
(introduced in FlowViewer 3.0, released in July 2006) allows the user
to maintain this information long-term by creating four MRTG-like
graphs. Filtered flow data is collected every five minutes and the
graphs are updated. FlowTracker requires Tobi Oetiker’s RRDtool package.
Screenshots are available.
flow-extract
which can be used to filter flow-tools-recorded flows through
user-specified tests
a set of "Inter.netPH
contribs"
by Horatio B. Bogbindero
some patches and a Python
module
by Robin Sommer.
flow-pairs
A script that extracts lists of the highest bandwidth
consumers by host and by port. Installed at
UCB
. Seems to have similar uses as the older MATHE system.
jflow
A set of Java classes for collecting and analyzing NetFlow data.
Supports Netflow versions 5 and 6, multithreaded implementation to
facilitate real-time traffic accounting and analysis.
Autofocus
A traffic analysis and visualization tool that describes the
traffic mix of a link through textual reports and time series plots.
The underlying research is documented in a SIGCOMM 2003 paper,
Automatically Inferring Patterns of Resource Consumption in
Network Traffic
, C. Estan, S. Savage, G. Varghese (PDF
paper, PPT
slides).
Wisconsin Netpy
Netpy is a network traffic analysis and visualization package
developed at University of Wisconsin-Madison. This application is
intended for the use of network administrators and it can help
understand usage trends in your network as well as support interactive
analysis of specific network events of interest. Netpy is distributed
under GPL and a BDS-like license. Netpy stores NetFlow records in a
local database after applying some sampling to reduce the size of the
data. The analysis engine supports interactive analyses on this data
where the user chooses the time interval of interest, the filtering
rules to apply to the traffic and the type of analysis. The netpy
console allows the user to manage the database, and perform analyses
interactively or through scripts. The graphical user interface
visualizes the results of the analyses accessing the database locally
or remotely through a netpy server that is also part of the
package.
Stager
Stager is a system for aggregation and presentation of network
statistics from the flow-tools package. Includes PostgreSQL storage
of aggregated statistics, as well as a Web frontend. A public demo is available.
nfstat
Developed to analyze (sampled) Netflow data from the Internet2
Abilene backbone. This is used to generate the Internet2 NetFlow Weekly
Reports
, which contain interesting statistics not easily found
elsewhere, such as distribution of bulk flow throughput. There are
two mailing lists for announcements
and for user
discussions
, respectively.
CAIDA cflowd
Rather complex system with distributed log servers. Released in
1998, this was the first open-source software system to work on
NetFlow data, but doesn’t seem to be maintained anymore. CAIDA have
prepared a nice FAQ
which contains interesting information both on Cflowd and on NetFlow
in general. CAIDA has announced that they no longer support cflowd,
and recommend that people move to flow-tools instead.
Aflow
Small Netflow monitoring tool developed by ARIN, available under
GPL. Features include easy configuration, maintenance of and graph
generation from RRDtool files,
pf/tcpdump-style filter rules. There is a mailing list for
announcements and discussion.
ASFLOW (already missing in
action?)
Tool to analyze traffic to "would-be" BGP neighbors. Presented by
Richard Steenbergen and Nathan Patrick at NANOG 35, October
2005. There is supposed to be both an easy-to-use Perl version and a
high-performance (but somewhat complex) C version.
Fluxoscope
Software used for charging, monitoring, and traffic analysis at
SWITCH. Includes its own NetFlow v5 accounting receiver which
aggregates traffic into multidimensional matrices
(AS/site/application). Most of the software is written in Common
Lisp.
UDP
Samplicator
A small program that receives UDP datagrams and redistributes
them to a set of receivers. Useful to distribute NetFlow accounting
streams to multiple post-processing programs. Is able to distribute
only a specified percentage of all packets to each receiver. Note
that recent versions added the possibility of “spoofing” the
original sender’s IP address.
Anonymization
Application Programming Interface (AAPI)/AnonTool
An open-source implementation of Anonymization API. Includes a
set of ready-to-use applications for anonymization of Netflow (v5 and
v9), as well as PCAP traces.
CANINE
"A NetFlows Conversion/Anonymization Tool for Format
Interoperability and Secure Sharing". Converts NetFlow data between
various formats including NetFlow v5 and v7, NFDUMP, CiscoNCSA and ArgusNCSA, and is able to
apply various methods of anonymization based on user configuration.
See also the FlowCon 2005 paper by
K. Luo, Y. Li, A. Slagell, and W. Yurick.
Panoptis
An open-source project started in 2001 by Costas Kotsokalis of
GRNET. Uses NetFlow accounting data to detect (Distributed) Denial of
Service attacks. Status as of November 2006: Supports NetFlow v1, v5
and v8 (router-aggregated) (with v8 untested for its biggest
part). The system supports proof-of-concept attack trace-back using a
mesh of detectors. Updates have been introduced so that the project
compiles on newer systems.
Flamingo
Real-time 3D traffic visualization system developed at Merit. This client/server system
based on Netflow and OpenGL plots traffic patterns by IP address, AS,
or port numbers, and allows interactive exploration of this data.
Sample graphics and a paper are available from the Website.
MHTG
(Multi Host Traffic Grapher)
Uses NetFlow to generate per-host graphs of traffic for a campus
network. Nice user interface implemented as a Java applet which
allows interaction with traffic plots. The software consists of a C++
program to process NetFlow data, a Mysql backend, and Perl frontend
and the Java grapher.
Matt’s Quick & Dirty CFLOWD tutorial and scripts…
Postprocessing scripts for cflowd data by Matthew Petach
flow2rrd.pl
Converts a cisco NetFlow stream into set of RRDtool files, based
on set of IP netmasks.
By Alex Pilosov.
bmpcount
A library of bitmap counting algorithms that count the number of
active flows in a network traffic trace. To be able to use it, you
should be familiar with the paper that describes the algorithms it
implements: _Bitmap algorithms for counting active flows on high speed
links_, C. Estan, G. Varghese, M. Fisk, Internet Measurement
Conference 2003 (PDF
paper, PPT
slides)
Slate
An application that converts LFAP data into NetFlow records – see
http://www.nmops.org/.
Ntop
This well-known libpcap-based network usage monitor has been
extended to produce NetFlow v5 accounting data. It also supports
sFlow.
SiLK
SiLK, the System for Internet-Level Knowledge, is a collection of
netflow tools developed by the CERT/NetSA (Network Situational
Awareness) Team to facilitate security analysis in large networks.
The toolset includes programs such as rwfilter,
rwcount, rwuniq. There are plans to develop this
further into an "Analyst’s Desktop", described in a FloCon’05 paper,
R: A Proposed Analysis and Visualization Environment for Network
Security Data
, J. McNutt (PDF).
(Ed.: Should this be "RAVE: A Proposed…"?)
The idea is to base this on the R statistical programming
language (see www.r-project.org), which
supports exploratory data analysis well.
Java Netflow
Collect-Analyzer
Collects Netflow v1/v5/v7/v8/v9 packets from Cisco/Juniper
routers or nProbe. It can store both raw data or analyzed contents to
a database using JDBC.
UPFrame
This UDP/Netflow Processing Framework is a system for
real-time processing of UDP packet streams such as Netflow export
data. It features a general infrastructure for dynamically
configurable plugin modules.
nProbe
A small self-contained program that generates NetFlow accounting
data for a traffic stream sniffed off one or several interfaces.
Works under Unix and Windows environments. It can be used to build
inexpensive NetFlow probes.
fprobe (I)
Traffic probe that can generate NetFlow data. Based on the
libpcap library. Fairly small implementation in C.
fprobe (II)
Another NetFlow-generating software traffic probe.
Softflowd
Traffic probe that can generate NetFlow data. Based on libpcap.
Comes with a NetFlow collector in Perl. Both the server (probe) and
client (collector) support export/import over IPv6. Very lean (as of
June 2004) implementation in C.

The pfflowd
variant is based on OpenBSD’s PF interface.

The flowd companion
NetFlow collector includes features such as multicast, IPv6 and
NetFlow v9 support, as well as fast upfront filtering.

Argus from QoSient
This network Audit Record Generation and Utilization
System
can be used for intrusion detection and QoS
monitoring. It is also mentioned
in the reference section of these pages.
RENETCOL
(RENATER Network Collector)
GPL’ed Netflow collector with support for Netflow v9, IPv6,
Multicast, and MPLS.
Flowc
"a tool for gathering, storing and analyzing traffic accounting
for Cisco routers with NetFlow enabled switching (version 5). This
package could be used by ISP for planning, analysis and billing
procedures."
CESNET NetFlow Monitor
by Jan Nejman.
RUS-CERT tools
The CERT of the Stuttgart University computing center (RUS-CERT)
has published some tools that they use internally to analyze Netflow
data. Some of the documentation is in German.
pmacct
A set of tools to account and aggregate IP traffic. Supports
libpcap, Netflow v1/v5/v7/v8/v9, and sFlow v2/v4/v5 for both
IPv4 and IPv6 traffic.
NEye
NEye is a Netflow V5 collector. It logs incoming Netflow V5 data
to ASCII, MySQL, or SQLite databases, and it makes full use of POSIX
threads if available. It works on most major platforms (Linux,
Solaris, AIX, Irix, HP/UX, Mac OS X, Digital Unix, etc.) and older
ones too (Ultrix, Nextstep, etc.).
NetFlow2MySQL,
NetFlow2XML,
and pcNetFlow
Three products from a research project at the NARA Institute of
Science and Technology.
F.L.A.V.I.O. (see also the FreshMeat page)
A Perl-based NetFlow collector that stores flow data "into a
MySQL database and gets it back to graph daily, weekly, monthly and
yearly charts."
NetFlowMet
Starting with release 4.2, Nevil Brownlee’s NeTraMet
package includes NetFlowMet, which implements an RTFM meter
fed on Netflow accounting data.
NetFlow Accounting
software
from ABPSoft
A self-contained NetFlow processing system written in C. Writes
captured flows to file. Postprocessor breaks up this data over peers
according to a definition file.
EHNT
(Extreme Happy NetFlow Tool) by Nik Weidenbacher
Another self-contained NetFlow accounting packet processor. The
receiving process also functions as a server to which various kinds of
clients can connect. Also written in C.
Hendrik
Visage’s NetFlow tools
FTP site with various tools for NetFlow postprocessing. In
particular, you will find:

  1. a UDP duplicator (hack of samplicator to preserve the source router
    IP)
  2. a couple of hacks to cflowd for dumping the flows every %n
    seconds as well as a "flhh" to output flowdump stuff
    aggregated, ready for a
    `grep|sed "s/../update /"|rrdtool -`
netMET – Network’s
METrology
Network measurement solution for the French regional academic
networking community, developed at the C.I.R.I.L in Nancy. Includes
an HTML interface and support for accounting and security
monitoring.
MATHE
An article (in French) about a Netflow accounting and
visualization system used at EPFL.
Uses an Oracle database and Perl DBI/GD scripts to generate a nice
breakdown of external traffic to departments/institutes.
JANET Traffic Accounting Site
An impressive application of Netflow which is used for
volume-based charging for JANET’s U.S. connection.
Other statistics at JANET
were done using NeTraMet.
InMon sFlow Toolkit
Open source tools for analyzing sFlow data. Allows sFlow data to
be used with a number of open source tools, including: tcpdump, snort
and MRTG or rrdtool. Also able to convert sFlow packets to NetFlow
packets.
Net::sFlow
Perl module to parse sFlow messages. Written by Elisa Jasinska
from AMS-IX as a basis of the sFlow-based traffic analysis service for
AMS-IX members. The use of this at AMS-IX has been described in
presentations and a paper, links to which can be found in
the references section.

Commercial Applications

Watch4net APG (Automated
Performance Grapher)
APG is a reporting tool that provides performance and capacity
reports on network, servers, applications and Netflow data.
Apogee Networks
The NetCountant network usage-based billing system and
the NetScope real-time network monitoring and performance
analysis solution support NetFlow, RMON2, RADIUS, other SNMP MIBs, and
“Layer 7” application/content switches.
Arbor Networks
Peakflow DOS detects denial-of-service attacks, and
Peakflow Traffic analyzes traffic and routing history. Both
can process NetFlow accounting data. As of November 2003, Arbor is
said to support Netflow v9.
Network Signature BENTO
BENTO stands for “BGP Enabled Network Traffic Organizer” and is
a high-performance NetFlow data processor with an integrated BGP-4
implementation to facilitate traffic analysis based on complex
external routing relationships. Product offerings include a
software/support package and an “appliance” consisting of a
preconfigured rack-mount server.
Caligare Flow Inspector
and NetImonitor
Analyzes NetFlow data for network monitoring as well as attack
detection and response. Works with NetFlow data export version
1,5,6,7 and 9. NetImonitor is primarily designed for use in the
United States.
Cisco
NetFlow
FlowCollector
/Network
Data Analyzer

Similar to cflowd but productized, with a (Java-based)
GUI and possibly better possibilities of defining filters and
aggregation schemes.

Cisco NAM
(Network Analyzer Module)
This is a "NetFlow collector on a linecard" for the Catalyst
6500/7600 OSR platform.
Concord
Network Health uses NetFlow and RMON2 accounting
information “to determine application, bandwitdth and server usage.”
Crannog Software’s Netflow
Monitor
LAN and WAN bandwidth analysis based on NetFlow data. Includes a
Web interface including Java applets to display traffic graphs and to
enable drill-down. Runs on Microsoft Windows NT4/2000/XP and on Unix.
Evaluation version of NetFlow Live available.
Cyclades-nQuirer
A network traffic monitoring appliance that can generate data in
both Netflow and nTop formats.
Digiquant
IMS accounting and billing system based on
Oracle 9i under Unix.
Gadgets Software &
Professional Services Ltd.
Network
Intelligence
traffic measurement and visualisation software
for GNU/Linux and Windows (client only) platforms. Free trial
available. Includes 3D visualization using OpenGL.

The author also wrote bbnfc, a
“bare-bones Netflow collector tool” that simply receives and
displayes Netflow v5 packets.

Hewlett-Packard
The Smart Internet Billing Solution usage management
system and well as OpenView Performance Insight for Networks
(OVPI) use NetFlow accounting data as possible input.
Infosim
StableNet
– Performance Management Engine
StableNet PME provides End-to-End (E2E) Service Level Management
(SLM) by monitoring and reporting on the systems, networks and
applications. StableNet supports the following flow technologies out
of the box: Netflow, cFlow, sFlow, Netstream.
InfoVista Corporation
InfoVista Service Level Management (SLM) and conformance
solution.
InMon Traffic
Sentinel
is a commercial, web-based application running on Linux that
provides real-time and historical analysis of flow information from
NetFlow, sFlow, LFAP or HP Extended RMON sources. Web queries provide
easy access to historical traffic matrices. Real-time top talker
charts identify sources of congestion. Includes network-wide
threshold and alert features as well as anomaly detection.
IsarFlow from IsarNet
IsarFlow is a traffic analysis tool for accounting, capacity
planning, QoS monitoring, and application distribution within Citrix
sessions based on Netflow.
Ixia
IxTraffic integrates NetFlow accounting data with
topology information from a live BGP-4 feed to allow analysis of
inter-domain traffic patterns.
Lancope StealthWatch
Flow-based Network Behavior Analysis appliance with advanced user
identity tracking. Can handle Netflow and sFlow data, or capture
packets from mirrored ports.
LoriotPro
A network monitoring ("supervision" in franglais) system that
includes a Netflow
plugin
. Stores flow data in a MySQL database.
ManageEngine
NetFlow Analyzer
Netflow-based bandwidth monitoring tool from AdventNet. Supports
location of bottlenecks and allows drilling down to find traffic that
is causing them. Thirty-day evaluation license available free of
charge. Versions for Windows and Linux (x86).
Mazu Networks
Mazu
Profiler
analyzes and models enterprise network traffic. It
provides visibility into network behavior, protects against worms and
other malware, and supports auditing and policy enforcement. It
supports Netflow v1/5/7/9 as well as other data collection mechanisms.
Micromuse
Cisco Info Center USM “acquires, analyzes, displays and
exports Internet usage data.” Note that Micromuse was integrated
into IBM under the "IBM Tivoli Netcool" brand.
NARUS
OSS Mediation solutions. They also do anomaly
detection.
Nazca.Billing
Integrated billing software for "Telephony, Internet and
Networks". Contains interfaces to many accounting systems including
NetFlow.
NetQoS
ReporterAnalyzer
Scalable solution for network capacity planning, troubleshooting,
and traffic analysis, including traffic visualization capabilities.
NetUp
Products
UTM is a billing
system for ISPs. It can use Netflow (v5) and several other accounting
methods. It supports a rich variety of charging and payment
schemes.

NDSAD Traffic
Collector
is an open-source (GPL’ed) tool that captures packets
and generates a Netflow (v5) accounting stream.

NetUsage from Apoapsis (formerly
called WANBUS)
The NetUsage suite strives to provide visibility of network
traffic, producing meaningful reports not only for network
professionals, but for IT management, business managers and accounts
departments. Supports network traffic monitoring, capacity planning,
business justification and cost control.
SolarWinds Orion NetFlow Traffic Analyzer
Windows-based commercial system that stores NetFlow data,
generates various types of charts, and provides "drill-down"
capabilities.
PRTG Traffic Manager
Windows-based bandwidth management software from Paessler. Uses SNMP, Netflow, and
packet capture for monitoring and classifying bandwidth usage.
Besides the commercial license, there is also a (restricted)
"freeware" license.
QRadar from Q1 Labs
The system can use Netflow data, but also includes its own
payload-aware flow collector which produces bi-directional flow
information in a format called QFlow. Includes anomaly
detection.
Plixer Scrutinizer NetFlow Analyzer
NetFlow-based Enterprise-level traffic analysis tool with
GUI-based reporting (topN hosts/applications etc.) and
zoom/drill-down. Uses MySQL
back-end. Free (as in
free beer) edition
available.
I-ABA and M-NTM from Tek Yazilim
Windows-based software to analyze NetFlow (and Cisco IP
Accounting) statistics. I-ABA specifically analyzes AS-to-AS traffic
streams. Trial versions can be downloaded.
Quallaby
Has a Netflow Application Pack for its PROVISO system
for network performance monitoring and service assurance. Quallaby
was acquired by Micromuse, which was itself acquired by IBM. The
Netflow Application Pack is maintained in the 4.4.1 release and
supports Netflow versions up to v8.
NetScout
nGenius Performance Manager “is a complete solution for
proactive monitoring, troubleshooting, capacity planning, and Voice
over IP (VoIP) monitoring”.
Portal Software
Infranet real-time customer management and billing
software.
RODOPI
Billing software for ISPs.
XACCT
Commercial vendor of accounting and billing solutions with the
ability to process (among others) Netflow accounting data

Did you enjoy this post? Why not leave a comment below and continue the conversation, or subscribe to my feed and get articles like this delivered automatically to your feed reader.

Comments

There’s another free NBA/NBAD product that uses Netflows:

http://www.akmalabs.com/flowmatrix.php

Here is a NetFlow Probe: P5 DataFlow Probe
http://www.p5nettech.com

Leave a comment

(required)

(required)


http://www.tanasi.it/kceojv.php