Provisioning private WAN routers

The steps to provision a private WAN (PWAN) router are nearly identical to the steps for provisioning an aggregator. As such, this document describes the PWAN router provisioning process in terms of the differences between aggregators and PWAN routers.

Please refer to the Provisioning aggregators document for complete aggregator provisioning steps, which should be used for PWAN routers except as noted below.

Virtualization considerations

Virtualization considerations for aggregators also apply to PWAN routers.

Note

Some virtualization systems like VMWare can make dedicated gateway via PWAN router difficult to set up with VLANs. This is due to how VMWare’s virtual networking is unable to handle tagged and untagged traffic on a single interface.

Private WAN router installation procedure

1. Provision the PWAN router operating system and install SD-WAN

Same as aggregator procedure.

Private WAN routers must use Linux kernel 3.16 or greater. This is the default for nodes imaged with the custom ISO.

2. Install private WAN software and update network settings

The steps for this installation are different than the steps for the aggregator installation.

The PWAN router network needs to be configured for its access to the partner core network. There are two ways to connect to this network:

  1. Use the same interface for traffic to/from aggregators and space VLAN traffic to/from the partner core network. This is the default and uses an internal bridge device to route traffic between the physical network interface and the private networking engine.

  2. Use a separate interface for traffic to/from aggregators and space VLAN traffic. To use this mode, you need to specify the interface on the PWAN router’s page in the web application. For example, it could be eth1.

Run the following command to configure the core networking:

/usr/sbin/pwan-configure-networking

The command offers to update the /etc/network/interfaces file with a bridge configuration necessary for private WAN using a single network interface for all traffic. It parses the interfaces file and suggests adding an Ethernet interface (usually eth0) to a new bridge called ext, using the Ethernet interface settings on the bridge. For example, if the aggregator had a static IP 203.0.113.1/24 on eth0, the script would offer to use 203.0.113.1/24 on the ext bridge and use eth0 as the only initial member of the bridge.

You will need to accept the prompt from the installer to update /etc/network/interfaces manually, or update that file yourself with an ext bridge configuration if you want to use a different interface for the bridge. If you want to use a separate interface for space VLAN traffic, you will also need to add a section to bring up that interface on boot, using the instructions later in this section.

For more information, see Private WAN routers.

Single network interface bridge

If you update the bridge configuration manually, it should be similar to this:

auto ext
iface ext inet static
    bridge_ports eth0
    address 203.0.113.1/24
    gateway 203.0.113.254
    # dns-* options are implemented by the resolvconf package, if installed
    dns-nameservers 8.8.8.8 8.8.4.4
    dns-search example.com

Note that the exact indentation of the parameters isn’t important (there has to be some white space for each parameter, but the exact amount and mix of tabs and spaces doesn’t matter), and that the network, broadcast, dns-nameservers, and dns-search parameters are optional. The auto ext line is mandatory.

To configure an IPv6 address as well, append something like this to the previous configuration:

iface ext inet6 static
        bridge_ports eth0
        address 2001:db8::c0ca:1eaf/64
        gateway 2001:db8::1ead:ed:beef

Note

An IPv4 address configuration is always required.

To use a VLAN interface as the default interface, simply use:

bridge_ports eth0.<VLAN-ID>

For example, if default traffic should use VLAN 3, use:

bridge_ports eth0.3

After updating the network settings, reboot:

reboot

Separate network interfaces

To use separate network interfaces for traffic between the router and aggregators, and between the router and the partner core network, update the file /etc/network/interfaces manually, adding a block such as:

auto eth1
iface eth1 inet manual
    pre-up ifconfig $IFACE up
    post-down ifconfig $IFACE down

Replace eth1 with the name of the interface to use for VLAN traffic. This name must also be provided in the VLAN trunk interface field on the PWAN router’s page in the management server.

The configuration here does not need to include any IP addresses, because the raw interface isn’t actually used for anything—only VLANs on that interface are used, and IP information for those interfaces are provided in the management application for each space that uses the PWAN router.

After updating the network settings, reboot:

reboot

3. Reconfigure network settings

Same as aggregator procedure, except that changes must be made to the ext bridge device if present, not to an Ethernet device.

4. If using VMware, install VMware tools

Same as aggregator procedure.

Depending on the type of virtual switch used in VMware, you may need to enable promiscuous mode on the VLAN trunk port. This is because each space present on a PWAN router has its own virtual network interface with its own MAC address, so Ethernet traffic from the PWAN router uses a number of different source MACs, and some virtual switches don’t learn MAC addresses in the same way that hardware network switches do. Without promiscuous mode, response packets from the partner core network never make it back to the PWAN router.

For a more detailed explanation of why promiscuous mode is necessary, see http://www.virtuallyghetto.com/2013/11/why-is-promiscuous-mode-forged.html. The issue with multiple MAC addresses from a PWAN router is similar to the case with a nested ESXi environment discussed in that article.

6. Configure dynamic routing

See bird implementation for private WAN routers.

7. Enable filesystem monitoring if necessary

High availability works differently in PWAN routers than in aggregators, but setting up the filesystem monitor to disable the PWAN router if its disk becomes read-only may still make high availability more reliable in some failure modes.