Flow collectors

Flow collectors are external hosts that can collect flow data for tracking and reporting purposes. Bonders can export the flow data to these flow collectors using industry-standard protocols.

Note

Flow collection is only available on bonders running 6.2 or later.

Note

Flow collectors can use nDPI to enable some extra protocol identification but this requires some extra configuration so ask your support contact if you want this enabled

Protocols

The following flow collection protocols are available.

IPFIX

IPFIX (IP Flow Information Export) is a standard protocol for flow export defined in RFC7011. It is supported by many modern devices and collectors, and is the recommended protocol for most situations.

Many collectors will listen for IPFIX data on port 2055 by default.

Netflow (v5 and v9)

Netflow is a protocol for flow export designed by Cisco. It is available on most Cisco products and some products from other vendors. It has been superceded by the formally standardized IPFIX protocol, which is heavily based on Netflow v9, but Netflow is still in common use.

Many collectors will listen for Netflow data on port 2055 by default.

sFlow

sFlow is an alternative to the IPFIX/Netflow protocols that operates by random sampling of traffic instead of tracking individual flows. As such, it is less accurate, but scales to higher speed networks and provides a more real-time view of traffic.

Collectors will listen for sFlow data on its official port, 6363.

Managing flow collectors

Each flow collector may be used on one or more bonders. When a collector’s settings are changed, those settings are automatically applied to all bonds configured to use the collector.

To see the list of flow collectors, select the Flow Collectors menu item in the Policies menu at the top of the administration interface. The collectors for the current space, as well as child spaces, are listed in the Flow Collectors tab, while collectors for parent spaces that are available to the current space, are listed in the Inherited Flow Collectors tab. Inherited collectors are read-only and may not be edited.

Adding a flow collector

To add a flow collector, click Add Flow Collector.

add-flow-collector

See the Flow collector parameters section below for a description of the flow collector parameters.

Editing a flow collector

To edit a flow collector, click the flow collector in the list.

See the Flow collector parameters section below for a description of the flow collector parameters.

Deleting a flow collector

To delete a flow collector, click the flow collector in the list and click the Delete button near the top of the page.

delete-flow-collector

Flow collector parameters

Name
A descriptive name for the flow collector.
IP
The IP address to send flow data to.
Port
The port to send flow data to. This is typically 2055 for IPFIX and Netflow, or 6363 for sFlow.
Type
The protocol type. See the Protocols section above for descriptions of the individual protocols.
Source IP policy
The policy to determine the IP address in packets sent to the collector. For IPFIX and Netflow, this determines the source address of the packets sent. For sFlow, this determines the agentIP field in the messages. If Connected IP is selected, the first Connected IP on the bond will be used. If Management VPN IP is selected, the management VPN IP will be used. If Tunnel is selected, the tunnel IP address will be used, which will be NAT’ed to the aggregator’s IP address for IPFIX and Netflow packets. If Connected IP is selected and there are no connected IPs defined, the tunnel IP address is used.
Space
The space for the collector. The collector will be available to bonds in this space, as well as child spaces, if the child spaces have the Use flow collectors from parent space parameter enabled.
IP precedence
If set, packets sent to the collectors will contain the given IP precedence value.
Direction
Defines traffic direction. Can be in or out.
Timeouts

Allows to tune a set of timeouts to be applied over collected packets. The value is expected in the following form: name=value:name2=value2:.... The set of supported timeouts, in seconds, and their default values are listed below:

  • tcp (generic tcp flow life) 3600
  • tcp.rst (TCP RST flow life) 120
  • tcp.fin (TCP FIN flow life) 300
  • udp (UDP flow life) 300
  • icmp (ICMP flow life) 300
  • general (generic flow life) 3600
  • maxlife (maximum flow life) 604800
  • expint (expiry interval) 60

expint is a passive timeout, ie. by default if a flow does not make any traffic in 60 secs it gets evicted from the cache. tcp, tcp.rst, tcp.fin, udp, icmp and general are active timeouts, ie. by default a tcp flow is evicted after 3600 secs even if still active, that is, it’s still making traffic.

This parameter only applies to IPFIX and NetFlow collectors.

Hop limit
Value of TTL for the newly generated NetFlow datagrams. This parameter only applies to IPFIX and NetFlow collectors.
Maximum flows
Maximum number of flows that can be tracked simultaneously. This parameter only applies to IPFIX and NetFlow collectors.
Engine
Allows to define Engine ID and Engine Type fields. It applies only to NetFlow v5/v9 and IPFIX. In NetFlow v9/IPFIX, the supplied value fills last two bytes of SourceID field. Expects two non-negative numbers, up to 255 each and separated by the : symbol. It also allows a collector to distinguish between distinct probe instances running on the same device, which is also important for letting NetFlow v9/IPFIX templates work correctly: template IDs will only get automatically selected inside single instances.
Set AgentSubId
If enabled, sets the value of agentSubId field inside the sFlow datagram header to the bond ID. This parameter only applies to sFlow collectors.
Interface speed
Statically associates an interface speed (in bits/second) for an interface whose traffic is exported to an sFlow collector.

Considerations for NAT

If a bond is using private addresses via NAT, it is important to note that some collectors may be unable to properly distinguish flow data in certain cases.

Duplicate address ranges

If multiple bonds are using the same address ranges, some collectors will aggregate the data for duplicate IP addresses, making it impossible to distinguish which bond the traffic passed through. Some of those collectors have ways to distinguish this traffic using, for example, the source IP address of the flow collection traffic, or via multiple collectors defined on separate ports.

Source IP of flow collection traffic

Some collectors use the address the flow collection traffic arrived from to determine which device the traffic passed though. If this traffic passed through NAT, the address seen by the collector will match the NAT’ed IP, not the private IP.

The Source IP policy parameter on the collector setup defines what IP the bonder will use when it sends the traffic data to the collector in order to help the collector distinguish between bonds. For IPFIX and NetFlow, the source IP of the packets will match the IP for the policy.

If the bond is using a CPE NAT IP, the collector will see the CPE NAT IP as the source address. If the bond is in a private WAN space with a NAT-via-PWAN gateway, the collector will see the gateway IP as the source address.

Counteracting NAT effects

In the case where a CPE NAT IP is used, the collector should simply be able to differentiate the traffic by tracking the CPE NAT IP.

Private WAN is not as simple, however, since multiple bonds share a single external address. One way to counteract the effect of NAT would be to pass traffic at the gateway without NAT, using an external gateway. That gateway can be set up to perform NAT when communicating with external networks, but not when sending traffic to the flow collector, preserving the source IP address.

Another option is to use the management VPN address. This involves placing a central collector in the management VPN network, terminated at the management server. This will require some planning and manual setup. Please contact support if you wish to pursue this option.