====================================================
AN-006 High-availability bonds using dynamic routing
====================================================

This document outlines how to set up bonds in a High-Availability (HA)
configuration using dynamic routing. The bonds will be configured with private
connected IPs in order to establish a dynamic routing protocol with customer
equipment on premise.

|ha-diagram|

Rather than defining the connected IPs and routes on the bond as in a
traditional bonding setup, they will be configured on the customer equipment and
dynamically routed from the customer's network, through the bonds to the
aggregators, and into the datacenter network.  While either OSPF or BGP may be
used between the customer equipment and the bonder, and between the aggregators
and the datacenter network, this document will use OSPF.

As an example, this document will assume the customer network is 203.0.113.0/30.

Prerequisites
^^^^^^^^^^^^^

#. All involved nodes (bonds and aggregators) must be running 6.5 or later.
#. A router or switch behind the bonders, capable of OSPF/BGP.
#. Source address verification must be disabled on each bond (this is the
   default for new bonds).
#. Dynamic routing configured between the aggregators and datacenter network.

.. note::
   Dynamic routing on the aggregator using quagga is not supported. You must
   configure configure `dynamic routing in bonding
   <../dynamic-routing/configuring-dynamic-routing-in-bonding.html>`_ instead.

Setup
^^^^^

Bonds
-----

Create two bonds: one **master** and one **backup**. Each bond can have its own
aggregator, bond settings, and legs, although in practice those settings will
be the same. The bonders can share the same Internet lines for each of their
legs.

Connected IPs
-------------

Create a private network used to communicate between the bonders and the
router/switch. For the purposes of this document, 192.168.66.0/29 is chosen. IPs
are assigned from this network as follows:

#. 192.168.66.1/29: *on-premise router/switch*
#. 192.168.66.2/29: *master bond*
#. 192.168.66.3/29: *backup bond*

Other required information
--------------------------

You will need to take note of the following information from the bond details
page of your HA bond pair:

- The master bond's tunnel subnet, found on the bond details page (e.g.,
  172.30.0.0/31). The first IP is the bonder's tunnel IP (e.g., 172.30.0.0), and
  the second IP is the aggregator's tunnel IP (e.g., 172.30.0.1).
- The backup bond's tunnel subnet, found in the same manner (e.g., 172.30.0.2/31)

Filtering
---------

It's very important that route filtering is configured on the aggregators to
prevent customer-defined routes from unexpectedly being propagated into the
datacentre network.

.. warning::
   Without restrictive filtering, nefarious routes could be injected into your
   core network from the customer premises!

Aggregator filter
~~~~~~~~~~~~~~~~~

The following filter should be defined on the aggregator, which will only import
the expected 203.0.113.0/30 route and reject anything else.

Import filter
+++++++++++++

- **Name**: HA bond 01 import filter
- *Add a rule*:

  + **Description**: Import 203.0.113.0/30
  + *Add a match*:

    - **Attribute**: Network
    - *Add a value*:

      + **Value 1**: 203.0.113.0/30

  + *Add an action*:

    - **Action**: Accept

- *Add a rule*:

  + **Description**: Reject everything else
  + *Add an action*:

    - **Action**: Reject

Bond filter
~~~~~~~~~~~

As the aggregator filter is restrictive on what it imports from the bonds,
filtering is not strictly required on the bonders. However, it is recommended to
filter routes being imported from the customer device to avoid any unexpected
routes from entering the bond's routing tables.

Bond protocols
--------------

Starting with the master bond, add the following protocols.

.. note::
   The customer router/switch must be configured to prefer routes via the master
   bonder. In OSPF, this is achieved by adding a higher cost to the OSPF
   interface connected to the backup bonder. You may also use the same interface
   and instead use separate OSPF instance IDs.

OSPF
~~~~

- **Name**: Customer OSPF
- **Protocol**: OSPF
- *Add an area*:

  + **Area ID**: 0.0.0.0
  + *Add an interface*:

    - *Add an interface link*:

      + **Link 1**: :abbr:`eth0 (The connected IP interface)`

BGP
~~~

- **Name**: BGP to aggregator
- **Protocol**: BGP
- **IPv4 import**: None
- **IPv4 export**: All
- **IPv6 import**: None
- **IPv6 export**: None
- **Local**:

  + **IP**: :abbr:`172.30.0.0 (The bonder tunnel IP)`
  + **AS number**: :abbr:`65536 (choose the same AS for local/remote)`

- **Neighbor**:

  + **IP**: :abbr:`172.30.0.1 (The aggregator tunnel IP)`
  + **AS number**: :abbr:`65536 (choose the same AS for local/remote)`

- **IPv4**:

  + **Self Next Hop**: Yes

- **General**:

  + **Direct**: Yes
  + **Hold time**: 10
  + **Startup hold time**: 10
  + **Connect delay**: 2
  + **Connect retry**: 2
  + **Error wait time min**: 5
  + **Error wait time length**: 30

Next, configure these same protocols on the backup bond, but with the
following important change, using the backup bond's tunnel subnet (172.30.0.2/31
in this example):

- **Local**:

  + **IP**: :abbr:`172.30.0.2 (The backup bond's tunnel IP)`

- **Neighbor**:

  + **IP**: :abbr:`172.30.0.3 (The backup bond's aggregator tunnel IP)`

Aggregator protocol
-------------------

The next step is to configure the converse of the BGP protocols on the
aggregators. Note that only one protocol is required on each aggregator as the
tunnel subnet is used for the IP range.

- **Name**: BGP to master bond
- **Protocol**: BGP
- **IPv4 import**: Filter
- **Filter**: HA bond 01 import filter
- **IPv4 export**: None
- **IPv6 import**: None
- **IPv6 export**: None
- **Local**:

  + **AS number**: 65536

- **Neighbor**:

  + **IP range**: 172.30.0.0/16
  + **AS number**: 65536

- **General**:

  + **Direct**: Yes
  + **Hold time**: 10
  + **Startup hold time**: 10
  + **Connect delay**: 2
  + **Connect retry**: 2
  + **Error wait time min**: 5
  + **Error wait time length**: 30

At this point, the routing should be working and the core should have a route
for the 203.0.113.0/30 network.

Security
^^^^^^^^

As configured above, the aggregator will only accept BGP connections from the
172.30.0.0/16 tunnel network, but any bonds hosted on the aggregator could
configure a BGP protocol and connect. For added security, you can configure MD5
authentication and TTL security on the bond and aggregator BGP protocols.

Testing
^^^^^^^

This document is meant to outline how HA could be achieved with a pair of bonds
and the dynamic routing functionality introduced in 6.5, but may not account for
every failure case. It is strongly recommended that any HA implementations are
extensively tested in a lab environment prior to deploying them to production
sites.

.. |ha-diagram| image:: /attachments/application-notes/an-006-bonder-ha-diagram.png
   :scale: 50%
