Add support for l3vpn with NB driver

Creates VRF/VXLAN per VNI, exposed through FRR with kernel routes
Vlan interfaces are added to the appropriate VNI, configured per bgpvpn
config options on the logical switch.

Related-bug: #2051105
Change-Id: I097c4629922d787827aba7761164f4004ed1305a

doc/source/bgp_supportability_matrix.rst

@@ -53,8 +53,8 @@ OVS-DPDK and hardware offload (HWOL) is supported.
 | L2VNI           | Extends the L2 segment on a given VNI.              | No need to expose it, automatic with the | Ingress: vxlan + bridge device           |  N/A                     | No                    | No            |
 |                 |                                                     | FRR configuration and the wiring.        | Egress: nothing                          |                          |                       |               |
 +-----------------+-----------------------------------------------------+------------------------------------------+------------------------------------------+--------------------------+-----------------------+---------------+
-| VRF             | Expose IPs on a given VRF (vni id).                 | Add IPs to dummy NIC associated to the   | Ingress: vxlan + bridge device           |  Yes                     | No                    | No            |
+| VRF             | Expose IPs on the routing table of a given VRF      | Add routes to the routing table of the   | Ingress: vxlan + bridge device           |  Yes                     | No                    | Yes           |
-|                 |                                                     | VRF device (lo_VNI_ID).                  | Egress: flow to redirect to VRF device   |  (Not implemented)       |                       |               |
+|                 | (vni id), creating L3VNI EVPN functionality.        | corresponding VRF (vrf-VNI_ID).          | Egress: flow to redirect to VRF device   |                          |                       |               |
 +-----------------+-----------------------------------------------------+------------------------------------------+------------------------------------------+--------------------------+-----------------------+---------------+
 | Dynamic         | Mix of the previous. Depending on annotations it    | Mix of the previous three.               | Ingress: mix of all the above            |  Depends on the method   | No                    | No            |
 |                 | exposes IPs differently and on different VNIs.      |                                          | Egress: mix of all the above             |  used                    |                       |               |

doc/source/contributor/drivers/nb_bgp_mode_design.rst

@@ -120,6 +120,9 @@ for now you can select:
 - ``underlay``: using kernel routing (what we describe in this document), same
   as supported by the driver at :ref:`bgp_driver`.
 
+- ``vrf``: using kernel routing, similar to the evpn driver, but with some
+  changes, as outlined in :ref:`evpn_wiring`.
+
 - ``ovn``: using an extra OVN cluster per node to perform the routing at
   OVN/OVS level instead of kernel, enabling datapath acceleration
   (Hardware Offloading and OVS-DPDK). More information about this mechanism
@@ -136,8 +139,9 @@ networking accordingly.
   .. note::
 
      Linux Kernel Networking is used when the default ``exposing_method``
-     (``underlay``) is used. If ``ovn`` is used instead, OVN routing is
+     (``underlay``) or (``vrf``) is used. If ``ovn`` is used instead, OVN
-     used instead of Kernel. For more details on this see :ref:`ovn_routing`.
+     routing is used instead of Kernel. For more details on this see
+     :ref:`ovn_routing`.
 
 The following events are watched and handled by the BGP watcher:
 
@@ -294,6 +298,9 @@ To accomplish the network configuration and advertisement, the driver ensures:
 
 .. include:: ../bgp_advertising.rst
 
+.. _evpn_wiring:
+.. include:: ../evpn_advertising.rst
+
 
 Traffic flow from tenant networks
 +++++++++++++++++++++++++++++++++
@@ -378,6 +385,7 @@ OVN load balancers:
 
 .. include:: ../agent_deployment.rst
 
+.. _NB_BGP_driver_limitations:
 
 Limitations
 -----------
@@ -390,23 +398,25 @@ The following limitations apply:
 - There is no API to decide what to expose, all VMs/LBs on providers or with
   floating IPs associated with them are exposed. For the VMs in the tenant
   networks, use the flag ``address_scopes`` to filter which subnets to expose,
-  which also prefents having overlapping IPs.
+  which also prevents having overlapping IPs.
 
 - In the currently implemented exposing methods (``underlay`` and
   ``ovn``) there is no support for overlapping CIDRs, so this must be
   avoided, e.g., by using address scopes and subnet pools.
 
-- For the default exposing method (``underlay``) the network traffic is steered
+- For the default exposing method (``underlay``) but also with the ``vrf``
-  by kernel routing (ip routes and rules), therefore OVS-DPDK, where the kernel
+  exposing method the network traffic is steered by kernel routing (ip
-  space is skipped, is not supported. With the ``ovn`` exposing method
+  routes and rules), therefore OVS-DPDK, where the kernel space is skipped,
-  the routing is done at ovn level, so this limitation does not exists.
+  is not supported.
-  More details in :ref:`ovn_routing`.
+  With the ``ovn`` exposing method the routing is done at ovn level, so this
+  limitation does not exists. More details in :ref:`ovn_routing`.
 
-- For the default exposing method (``underlay``) the network traffic is steered
+- For the default exposing method (``underlay``) but also with the ``vrf``
-  by kernel routing (ip routes and rules), therefore SRIOV, where the hypervisor
+  exposing method the network traffic is steered by kernel routing (ip
-  is skipped, is not supported.  With the ``ovn`` exposing method
+  routes and rules), therefore SRIOV, where the hypervisor is skipped, is
-  the routing is done at ovn level, so this limitation does not exists.
+  not supported.
-  More details in :ref:`ovn_routing`.
+  With the ``ovn`` exposing method the routing is done at ovn level, so this
+  limitation does not exists. More details in :ref:`ovn_routing`.
 
 - In OpenStack with OVN networking the N/S traffic to the ovn-octavia VIPs on
   the provider or the FIPs associated with the VIPs on tenant networks needs to

doc/source/contributor/evpn_advertising.rst

@@ -0,0 +1,200 @@
+EVPN Advertisement (expose method ``vrf``)
+++++++++++++++++++++++++++++++++++++++++++
+
+When using expose method ``vrf``, the OVN BGP Agent is in charge of triggering
+FRR (IP routing protocol suite for Linux which includes protocol daemons for
+BGP, OSPF, RIP, among others) to advertise/withdraw directly connected and
+kernel routes via BGP.
+
+To do that, when the agent starts, it will search for all provider networks
+and configure them.
+
+In order to expose a provider network, each provider network must match these
+criteria:
+
+- The provider network can be matched to the bridge mappings as defined in the
+  running OpenVSwitch instance (e.g. ``ovn-bridge-mappings="physnet1:br-ex"``)
+
+- The provider network has been configured by an admin with at least a ``vni``,
+  and the vpn type has been configured too with value ``l3``.
+
+  For example (when using the OVN tools):
+
+  .. code-block:: bash
+
+    $ ovn-nbctl set logical-switch neutron-cd5d6fa7-3ed7-452b-8ce9-1490e2d377c8 external_ids:"neutron_bgpvpn\:type"=l3
+    $ ovn-nbctl set logical-switch neutron-cd5d6fa7-3ed7-452b-8ce9-1490e2d377c8 external_ids:"neutron_bgpvpn\:vni"=100
+    $ ovn-nbctl list logical-switch | less
+    ...
+    external_ids        : {.. "neutron_bgpvpn:type"=l3, "neutron_bgpvpn:vni"="1001" ..}
+    name                : neutron-cd5d6fa7-3ed7-452b-8ce9-1490e2d377c8
+    ...
+
+  It is also possible to configure this using the Neutron BGP VPN API.
+
+Initialization sequence per VRF
+'''''''''''''''''''''''''''''''
+
+Once the networks have been initialized, the driver waits for the first ip to be
+exposed, before actually exposing the VRF on the host.
+
+Once a VRF is exposed on the host, the following will be done (per VRF):
+
+1. Create EVPN related devices
+
+   - Create VRF device, using the VNI number as name suffix: vrf-1001
+
+     .. code-block:: bash
+
+       $ ip link add vrf-1001 type vrf table 1001
+
+   - Create the VXLAN device, using the VNI number as the vxlan id, as well as
+     for the name suffix: vxlan-1001
+
+     .. code-block:: bash
+
+       $ ip link add vxlan-1001 type vxlan id 1001 dstport 4789 local LOOPBACK_IP nolearning
+
+   - Create the Bridge device, where the vxlan device is connected, and
+     associate it to the created vrf, also using the VNI number as name suffix:
+     br-1001
+
+     .. code-block:: bash
+
+       $ ip link add name br-1001 type bridge stp_state 0
+       $ ip link set br-1001 master vrf-1001
+       $ ip link set vxlan-1001 master br-1001
+
+2. Reconfigure local FRR instance (``frr.conf``) to ensure the new VRF is
+   exposed. To do that it uses ``vtysh shell``. It connects to the existing
+   FRR socket (--vty_socket option) and executes the next commands, passing
+   them through a file (-c FILE_NAME option):
+
+   .. code-block:: jinja
+
+     vrf {{ vrf_name }}
+         vni {{ vni }}
+     exit-vrf
+     router bgp {{ bgp_as }} vrf {{ vrf_name }}
+       bgp router-id {{ bgp_router_id }}
+       address-family ipv4 unicast
+       redistribute connected
+       redistribute kernel
+       exit-address-family
+
+       address-family ipv6 unicast
+         redistribute connected
+         redistribute kernel
+       exit-address-family
+       address-family l2vpn evpn
+         advertise ipv4 unicast
+         advertise ipv6 unicast
+         rd {{ local_ip }}:{{ vni }}
+       exit-address-family
+
+3. Connect EVPN to OVN overlay so that traffic can be redirected from the node
+   to the OVN virtual networking. It needs to connect the VRF to the OVS
+   provider bridge:
+
+   - Create a veth device, that will be used for routing between the vrf and
+     OVN, using the uuid of the localnet port in the logical-switch-port table
+     and connect it to ovs (in this example the uuid of the localnet port is
+     ``12345678-1234-1234-1234-123456789012``, and the first 11 chars will
+     be used in the interface name):
+
+     .. code-block:: bash
+
+       $ ip link add name vrf12345678-12 type veth peer name ovs12345678-12
+       $ ovs-vsctl add-port br-ex ovs12345678-12
+       $ ip link set up dev ovs12345678-12
+
+   - For EVPN l3 mode (only supported mode currently), it will attach the vrf
+     side to the vrf:
+
+     .. code-block:: bash
+
+       $ ip link set vrf12345678-12 master vrf-1001
+       $ ip link set up dev vrf12345678-12
+
+     And it will add routing IPs on the veth interface, so the kernel is able
+     to do L3 routing within the VRF. By default it will add a 169.254.x.x
+     address based on the VNI/VLAN.
+
+     If possible it will use the dhcp options to determine if it can use an
+     actually configured router ip address, in addition to the 169.254.x.x
+     address:
+
+     .. code-block:: bash
+
+       $ ip address add 10.0.0.1/32 dev vrf12345678-12  # router option from dhcp opts
+       $ ip address add 169.254.0.123/32 dev vrf12345678-12  # generated 169.254.x.x address for vlan 123
+       $ ip -6 address add fd53:d91e:400:7f17::7b/128 dev vrf12345678-12  # generated ipv6 address for vlan 123
+
+4. Add needed OVS flows into the OVS provider bridge (e.g., br-ex) to redirect
+   the traffic back from OVN to the proper VRF, based on the subnet CIDR and
+   the router gateway port MAC address.
+
+   .. code-block:: bash
+
+      $ ovs-ofctl add-flow br-ex cookie=0x3e7,priority=900,ip,in_port=<OVN_PATCH_PORT_ID>,actions=mod_dl_dst:VETH|VLAN_MAC,NORMAL
+
+5. If ``CONF.anycast_evpn_gateway_mode`` is enabled, it will make sure that the
+   mac address on the vrf12345678-12 interface is equal on all nodes, using the
+   VLAN id and VNI id as an offset while generating a MAC address.
+
+   .. code-block:: bash
+
+     $ ip link set address 02:00:03:e7:00:7b dev vrf12345678-12  # generated mac for vni 1001 and vlan 123
+
+     # Replace link local address and update to generated vlan mac (used for ipv6 router advertisements)
+     $ ip -6 address del <some fe80::/10 address> dev vrf12345678-12
+     $ ip -6 address add fe80::200:3e7:65/64 dev vrf12345678-12
+
+6. If IPv6 subnets are defined (checked in dhcp opts once again), then configure
+   FRR to handle neighbor discovery (and do router advertisements for us)
+
+   .. code-block:: jinja
+
+     interface {{ vrf_intf }}
+      {% if is_dhcpv6 %}
+      ipv6 nd managed-config-flag
+      {% endif %}
+      {% for server in dns_servers %}
+      ipv6 nd rdnss {{ server }}
+      {% endfor %}
+      ipv6 nd prefix {{ prefix }}
+      no ipv6 nd suppress-ra
+     exit
+
+7. Then, finally, add the routes to expose to the VRF, since we use full
+   kernel routing in this VRF, we also expose the MAC address that belongs
+   to this route, so we do not rely on ARP proxies in OVN.
+
+   .. code-block:: bash
+
+     $ ip route add 10.0.0.5/32 dev vrf12345678-12
+     $ ip route show table 1001 | grep veth
+     local 10.0.0.1 dev vrf12345678-12 proto kernel scope host src 10.0.0.1
+     10.0.0.5 dev vrf12345678-12 scope link
+     local 169.254.0.123 dev vrf12345678-12 proto kernel scope host src 169.254.0.123
+
+     $ ip neigh add 10.0.0.5 dev vrf12345678-12 lladdr fa:16:3e:7d:50:ad nud permanent
+     $ ip neigh show vrf vrf-100 | grep veth
+     10.0.0.5 dev vrf12345678-12 lladdr fa:16:3e:7d:50:ad PERMANENT
+     fe80::f816:3eff:fe7d:50ad dev vrf12345678-12 lladdr fa:16:3e:7d:50:ad STALE
+
+
+.. note::
+
+  The VRF is not associated to one OpenStack tenant, but can be mixed with
+  other provider networks too. When using VLAN provider networks, one can
+  connect multiple networks to the same VNI, effectively placing them in the
+  same VRF, routed and handled through kernel and FRR.
+
+.. note::
+  As we also want to be able to expose VM connected to tenant networks
+  (when ``expose_tenant_networks`` or ``expose_ipv6_gua_tenant_networks``
+  configuration options are enabled), there is a need to expose the Neutron
+  router gateway port (cr-lrp on OVN) so that the traffic to VMs in tenant
+  networks is injected into OVN overlay through the node that is hosting
+  that port.

doc/source/examples/index.rst

@@ -0,0 +1,8 @@
+===================
+Example deployments
+===================
+
+ .. toctree::
+    :maxdepth: 2
+
+    nb_evpn_vrf

doc/source/examples/nb_evpn_vrf.rst

@@ -0,0 +1,201 @@
+========================================
+EVPN L3VNI using OVN NB database
+========================================
+
+One example deployment, is to run the OVN BGP Agent and to expose provider
+networks and tenant networks with pure layer 3 functionality.
+
+Traffic flow
+~~~~~~~~~~~~
+
+The next figure shows the N/S traffic flow through the VRF to the VM,
+including information the routes in the VRF routing table.
+
+.. image:: ../../images/evpn-flow-l3vpn.svg
+   :alt: EVPN l3vpn diagram
+   :align: center
+   :width: 100%
+
+
+On Host-2, the IPs of both the external router gateway port (GW, ``172.16.2.12``),
+as well as the subnet it exposes (``192.168.0.0/24``) gets added to the
+routing table of the vrf (``vrf-1001``). Also any instance that is attached
+directly on a provider network is added to the routing table.
+
+Then FRR is utilized to expose this vrf through BGP/EVPN. Routes from other nodes
+are imported and added as an VXLAN route, pointing to the bridge (``br-1001``).
+
+This allows the external route to reach the internal VM, possibly routed
+through the host that is hosting the router gateway port.
+
+Configuration
+~~~~~~~~~~~~~
+
+frr-bgpd.conf
+--------------------
+
+A typical FRR BGP configuration would look like this (for example for host-1):
+
+.. code-block:: ini
+
+  router bgp 64531
+    bgp router-id 10.100.100.1
+    bgp default l2vpn-evpn
+
+    neighbor rr-peers peer-group
+    neighbor rr-peers remote-as internal
+    neighbor rr-peers bfd
+
+    neighbor upstream-peers peer-group
+    neighbor upstream-peers remote-as internal
+    neighbor upstream-peers bfd
+
+    ! Upstream routers (these will most likely expose the default outbound route)
+    neighbor 10.100.250.3 peer-group upstream-peers
+    neighbor 10.100.250.4 peer-group upstream-peers
+
+    ! Route reflector peers (used for distributing routes between compute nodes)
+    neighbor 10.100.50.66 peer-group rr-peers
+    neighbor 10.100.51.66 peer-group rr-peers
+    neighbor 10.100.52.66 peer-group rr-peers
+
+    address-family l2vpn evpn
+      neighbor rr-peers soft-reconfiguration inbound
+      neighbor upstream-peers soft-reconfiguration inbound
+      advertise-all-vni
+    exit-address-family
+  exit
+
+.. note::
+  In our best practice we use FRR instances on central nodes to act as route
+  reflector. How to scale your BGP network and what practices you might use
+  is beyond the perview of this example document.
+
+
+ovn-bgp-agent.conf
+------------------
+To run OVN BGP Agent with NB driver and EVPN L3 mode, the following configuration is recommended:
+
+.. code-block:: ini
+
+  [DEFAULT]
+  # Time (seconds) between re-sync actions.
+  reconcile_interval = 600
+
+  # Time (seconds) between re-sync actions to ensure frr configuration is correct.
+  # NOTE: This function does not do anything in our setup, so this high interval is fine.
+  frr_reconcile_interval = 86400
+
+  # Expose VM IPs on tenant networks.
+  expose_tenant_networks = True
+
+  # The NB driver is capable of advertising the tenant networks either per
+  # host or per subnet. So either per /32 or /128 or per subnet like /24.
+  # Choose "host" as value for this option to advertise per host or choose
+  # "subnet" to announce per subnet prefix.
+  advertisement_method_tenant_networks = subnet
+
+  # Require SNAT on the router port to be disabled before exposing the tenant
+  # networks. Otherwise the exposed tenant networks will be reachable from the
+  # outside, but the connections set up from within the tenant vm will always
+  # be SNAT-ed by the router, thus be the router ip. When SNAT is disabled,
+  # OVN will do pure routing without SNAT
+  require_snat_disabled_for_tenant_networks = True
+
+  # Expose only VM IPv6 IPs on tenant networks if they are GUA.
+  # expose_ipv6_gua_tenant_networks = False
+
+  # Driver to be used.
+  driver = 'nb_ovn_bgp_driver'
+
+  # The connection string for the native OVSDB backend.
+  ovsdb_connection = tcp:127.0.0.1:6640
+
+  # Timeout in seconds for the OVSDB connection transaction.
+  # ovsdb_connection_timeout = 180
+
+  # AS number to be used by the Agent when running in BGP mode.
+  bgp_AS = < CONFIGURE YOUR AS HERE >
+
+  # Router ID to be used by the Agent when running in BGP mode.
+  bgp_router_id = < CONFIGURE YOUR ROUTER ID/IP HERE >
+
+  # IP address of local EVPN VXLAN (tunnel) endpoint.
+  evpn_local_ip = < CONFIGURE YOUR HOST'S EVPN VXLAN IP HERE>
+
+  # If enabled, all routes are removed from the VRF table at startup.
+  clear_vrf_routes_on_startup = False
+
+  # Allows to filter on the address scope (optional, comma separated list of uuids)
+  # address_scopes = 11111111-1111-1111-1111-111111111111,22222222-2222-2222-2222-222222222222
+
+  # The exposing mechanism to be used.
+  exposing_method = 'vrf'
+
+  # When using exposing_method vrf and l3 mode on networks, then one can create
+  # anycast mac addresses, basically using the same mac address on all nodes for
+  # use with routing.
+  anycast_evpn_gateway_mode = True
+
+  [ovn]
+  # The connection string for the OVN_Northbound OVSDB.
+  # Use tcp:IP:PORT for TCP connection.
+  # Use unix:FILE for unix domain socket connection.
+  ovn_nb_connection = < CONNECTION STRING TO NB OVN DB>
+
+Configure provider networks
+---------------------------
+
+This section assumes, that you've already configured OVN, and applied the correct bridge-mappings
+on the hosts themselves, see `Neutron documentation regarding provider networks. <https://docs.openstack.org/neutron/latest/admin/ovn/refarch/provider-networks.html>`_
+
+First, create your provider network through neutron
+
+.. code-block:: bash
+
+  openstack network create my_network \
+     --provider-network-type vlan \
+     --provider-physical-network physnet1 \
+     --provider-segment 123 \
+     --mtu 1500 \
+     --external \
+     --default
+
+Then, configure your provider networks through either Neutron BGPVPN API or
+with ovn commandline:
+
+.. code-block:: bash
+
+  ovn-nbctl set logical-switch < UUID > external_ids:"neutron_bgpvpn\:type"="l3"
+  ovn-nbctl set logical-switch < UUID > external_ids:"neutron_bgpvpn\:vni"="1001"  # or any other number
+
+Now use this network to attach routers on it (so update router:external on
+the provider network) or share your network among tenants (shared = True)
+
+And create some routers, or add some instances on the provider network, so a
+host will start exposing the networks and/or ips.
+
+Current known limitations
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+- Only one Flat provider network can be exposed per vni. Recommendation is
+  to use VLAN provider networks.
+
+- Do not use the same VLAN id twice in the same VNI. A provider network with
+  type flat is considered vlan 0.
+
+- It is not possible to have a tenant network (which is routed through a
+  gateway) in separate VRF's, make sure to use address scopes and subnet pools
+  to prevent ip overlaps, if you are planning to expose tenant networks.
+
+- Provider networks of type ``flat`` is supported, but is limited (because of
+  how ``flat`` networks operate) to one provider network per bridge mapping.
+  It is recommended to use provider networks of type ``vlan``. That way it is
+  also easier to create multiple provider networks, without having to create
+  new bridgemappings for every provider network.
+
+  Every provider network can be assigned a separate VNI, so IP overlap is not
+  an issue between provider networks, as long as separate VNI's are used for
+  those provider networks.
+
+See other known limitations at the NB BGP driver :ref:`NB_BGP_driver_limitations`

doc/source/index.rst

@@ -14,6 +14,7 @@ Contents:
 
    readme
    contributor/index
+   examples/index
    bgp_supportability_matrix
 
 Indices and tables

ovn_bgp_agent/constants.py

@@ -56,6 +56,9 @@ FRR_SOCKET_PATH = "/run/frr/"
 IP_VERSION_6 = 6
 IP_VERSION_4 = 4
 
+# initial mac address to generate anycast addresses for (vni and vlan will
+# be added to this value)
+MAC_LLADDR_OFFSET = "02:00:00:00:00:00"
 ARP_IPV4_PREFIX = "169.254."
 NDP_IPV6_PREFIX = "fd53:d91e:400:7f17::"
 
@@ -64,8 +67,20 @@ IPV4_OCTET_RANGE = 256
 BGP_MODE = 'BGP'
 EVPN_MODE = 'EVPN'
 
+# NOTE(mnederlof, ltomasbo): for lack of better variables we are using the
+# neutron_bgpvpn namespace for now. If in the future another API endpoint
+# is created, we should adapt or maybe move them to another location that
+# makes sense at that time.
 OVN_EVPN_VNI_EXT_ID_KEY = 'neutron_bgpvpn:vni'
 OVN_EVPN_AS_EXT_ID_KEY = 'neutron_bgpvpn:as'
+OVN_EVPN_TYPE_EXT_ID_KEY = 'neutron_bgpvpn:type'
+OVN_EVPN_ROUTE_TARGETS_EXT_ID_KEY = 'neutron_bgpvpn:rt'
+OVN_EVPN_ROUTE_DISTINGUISHERS_EXT_ID_KEY = 'neutron_bgpvpn:rd'
+OVN_EVPN_IMPORT_TARGETS_EXT_ID_KEY = 'neutron_bgpvpn:it'
+OVN_EVPN_EXPORT_TARGETS_EXT_ID_KEY = 'neutron_bgpvpn:et'
+
+OVN_EVPN_TYPE_L2 = 'l2'
+OVN_EVPN_TYPE_L3 = 'l3'
 OVN_EVPN_VRF_PREFIX = "vrf-"
 OVN_EVPN_BRIDGE_PREFIX = "br-"
 OVN_EVPN_VXLAN_PREFIX = "vxlan-"
@@ -77,8 +92,20 @@ OVN_INTEGRATION_BRIDGE = 'br-int'
 OVN_LRP_PORT_NAME_PREFIX = 'lrp-'
 OVN_CRLRP_PORT_NAME_PREFIX = 'cr-lrp-'
 
+# the new prefix will get the first 11 chars of the localnet port prefixed,
+# neutron-tap style
+OVN_EVPN_VETH_VRF_UUID_PREFIX = "vrf"
+OVN_EVPN_VETH_OVS_UUID_PREFIX = "ovs"
+
 OVS_PATCH_PROVNET_PORT_PREFIX = 'patch-provnet-'
 
+EVPN_EXT_ID_MAPPING = {
+    'route_targets': OVN_EVPN_ROUTE_TARGETS_EXT_ID_KEY,
+    'route_distinguishers': OVN_EVPN_ROUTE_DISTINGUISHERS_EXT_ID_KEY,
+    'export_targets': OVN_EVPN_EXPORT_TARGETS_EXT_ID_KEY,
+    'import_targets': OVN_EVPN_IMPORT_TARGETS_EXT_ID_KEY,
+}
+
 LINK_UP = "up"
 LINK_DOWN = "down"
 
@@ -116,6 +143,7 @@ POLICY_ACTION_TYPES = (POLICY_ACTION_REROUTE)
 LR_POLICY_PRIORITY_MAX = 32767
 
 ROUTE_DISCARD = 'discard'
+ROUTE_TYPE_UNICAST = 1
 
 # Family constants
 AF_INET = socket.AF_INET
@@ -125,3 +153,5 @@ AF_INET6 = socket.AF_INET6
 ROUTING_TABLES_FILE = '/etc/iproute2/rt_tables'
 ROUTING_TABLE_MIN = 1
 ROUTING_TABLE_MAX = 252
+
+VLAN_ID_UNTAGGED = 0

ovn_bgp_agent/drivers/openstack/nb_ovn_bgp_driver.py

@@ -38,7 +38,7 @@ LOG = logging.getLogger(__name__)
 # logging.basicConfig(level=logging.DEBUG)
 
 OVN_TABLES = ['Logical_Switch_Port', 'NAT', 'Logical_Switch', 'Logical_Router',
-              'Logical_Router_Port', 'Load_Balancer']
+              'Logical_Router_Port', 'Load_Balancer', 'DHCP_Options']
 LOCAL_CLUSTER_OVN_TABLES = ['Logical_Switch', 'Logical_Switch_Port',
                             'Logical_Router', 'Logical_Router_Port',
                             'Logical_Router_Policy',
@@ -148,11 +148,18 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
                   watcher.LogicalSwitchPortProviderDeleteEvent(self),
                   watcher.LogicalSwitchPortFIPCreateEvent(self),
                   watcher.LogicalSwitchPortFIPDeleteEvent(self),
-                  watcher.LocalnetCreateDeleteEvent(self),
                   watcher.OVNLBCreateEvent(self),
                   watcher.OVNLBDeleteEvent(self),
                   watcher.OVNPFCreateEvent(self),
                   watcher.OVNPFDeleteEvent(self)}
+
+        if CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+            # For vrf we require more information on the logical_switch
+            # before performing a sync.
+            events.add(watcher.LogicalSwitchUpdateEvent(self))
+        else:
+            events.add(watcher.LocalnetCreateDeleteEvent(self))
+
         if self._expose_tenant_networks:
             events.update({watcher.ChassisRedirectCreateEvent(self),
                            watcher.ChassisRedirectDeleteEvent(self),
@@ -279,6 +286,8 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
                     self._exposed_ips.setdefault(logical_switch, {}).update(
                         {ip: {'bridge_device': bridge_device,
                               'bridge_vlan': bridge_vlan}})
+            else:
+                return False
         except Exception as e:
             LOG.exception("Unexpected exception while wiring provider port: "
                           "%s", e)
@@ -622,6 +631,21 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
     def withdraw_remote_ip(self, ips, ips_info):
         self._withdraw_remote_ip(ips, ips_info)
 
+    def _get_exposed_ip(self, exposed_ip):
+        for ls, ip_info in self._exposed_ips.items():
+            if exposed_ip in ip_info:
+                return ls, ip_info[exposed_ip]
+
+    def _get_router_port_info_for_ls(self, ls):
+        # LOG.debug('Searching router port info for ls %s', ls)
+        lrps = list(self.ovn_local_lrps.get(ls, []))
+        if not lrps:
+            LOG.debug('Could not find router gateway port for ls %s', ls)
+            return
+
+        _, cr_lrp_info = self._get_exposed_ip(lrps[0])
+        return cr_lrp_info
+
     def _expose_remote_ip(self, ips, ips_info):
         if (CONF.advertisement_method_tenant_networks ==
                 constants.ADVERTISEMENT_METHOD_SUBNET):
@@ -638,10 +662,23 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
 
         LOG.debug("Adding BGP route for tenant IP(s) %s on chassis %s",
                   ips_to_expose, self.chassis)
-        bgp_utils.announce_ips(ips_to_expose)
+        if CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+            cr_lrp_info = self._get_router_port_info_for_ls(
+                ips_info['logical_switch'])
+            if not cr_lrp_info:
+                LOG.debug('Unable to export routed ips %s: could not find '
+                          'router gateway port for ls %s',
+                          ips, ips_info['logical_switch'])
+                return
+
+            # Add the bridge_vlan, bridge_device and via to the ips_info
+            ips_info.update(cr_lrp_info)
+
+        bgp_utils.announce_ips(ips_to_expose, ips_info=ips_info)
         for ip in ips_to_expose:
             self._exposed_ips.setdefault(
-                ips_info['logical_switch'], {}).update({ip: {}})
+                ips_info['logical_switch'], {}).setdefault(ip, {})
+
         LOG.debug("Added BGP route for tenant IP(s) %s on chassis %s",
                   ips_to_expose, self.chassis)
 
@@ -660,12 +697,22 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
 
         LOG.debug("Deleting BGP route for tenant IP(s) %s on chassis %s",
                   ips_to_withdraw, self.chassis)
-        bgp_utils.withdraw_ips(ips_to_withdraw)
+        if CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+            cr_lrp_info = self._get_router_port_info_for_ls(
+                ips_info['logical_switch'])
+            if not cr_lrp_info:
+                LOG.debug('Unable to withdraw routed ips %s: could not find '
+                          'router gateway port for ls %s',
+                          ips, ips_info['logical_switch'])
+                return
+
+            # Add the bridge_vlan, bridge_device and via to the ips_info
+            ips_info.update(cr_lrp_info)
+
+        bgp_utils.withdraw_ips(ips_to_withdraw, ips_info=ips_info)
         for ip in ips_to_withdraw:
-            if self._exposed_ips.get(
+            self._exposed_ips.get(ips_info['logical_switch'], {}).pop(ip, None)
-                    ips_info['logical_switch'], {}).get(ip):
+
-                self._exposed_ips[
-                    ips_info['logical_switch']].pop(ip)
         LOG.debug("Deleted BGP route for tenant IP(s) %s on chassis %s",
                   ips_to_withdraw, self.chassis)
 
@@ -805,10 +852,18 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
         if (CONF.advertisement_method_tenant_networks ==
                 constants.ADVERTISEMENT_METHOD_SUBNET):
             # Fix ips to be the network address, instead of the lrp address
-            # so the cleanup will not remove them, since they match what's
+            # so we can advertise the entire subnet. This way a cleanup of
-            # in the kernel
+            # EVPN would not remove the incorrect entry as well.
             ips = driver_utils.get_prefixes_from_ips(ips)
 
+        elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+            # For evpn, we work with routes and since we are not exposing per
+            # subnet, we need to remove the subnetmask part, so the ips are
+            # exposed as a /32 or /128. Otherwise the cleanup would remove
+            # the exposed route again, since 10.0.0.1/24 (a router ip) would
+            # not match the kernel route of 10.0.0.0/24.
+            ips = [ip.split('/')[0] for ip in ips]
+
         ips_to_process = []
         for ip in ips:
             if not CONF.expose_tenant_networks:
@@ -845,10 +900,11 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
                     self._exposed_ips.setdefault(logical_switch, {}).update(
                         {ip: {
                             'bridge_device': cr_lrp_info.get('bridge_device'),
-                            'bridge_vlan': cr_lrp_info.get('bridge_vlan')}})
+                            'bridge_vlan': cr_lrp_info.get('bridge_vlan'),
+                            'via': cr_lrp_info.get('ips')}})
 
                     self.ovn_local_lrps.setdefault(
-                        subnet_info['network'], []).append(ip)
+                        subnet_info['network'], set()).add(ip)
                 else:
                     error_msg = ("Something happen while exposing the subnet"
                                  "and they have not been properly exposed")
@@ -875,10 +931,15 @@ class NBOVNBGPDriver(driver_api.AgentDriverBase):
         if (CONF.advertisement_method_tenant_networks ==
                 constants.ADVERTISEMENT_METHOD_SUBNET):
             # Fix ips to be the network address, instead of the lrp address
-            # so the cleanup will not remove them, since they match what's
+            # so we can withdraw the corrent subnet entry.
-            # in the kernel
             ips = driver_utils.get_prefixes_from_ips(ips)
 
+        elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+            # For evpn, we work with routes and since we are not exposing per
+            # subnet, we need to remove the subnetmask part, so the ips are
+            # withdrawn as a /32 or /128.
+            ips = [ip.split('/')[0] for ip in ips]
+
         ips_to_process = []
         for ip in ips:
             if (not CONF.expose_tenant_networks and

ovn_bgp_agent/drivers/openstack/utils/bgp.py

@@ -16,6 +16,8 @@ from oslo_config import cfg
 from oslo_log import log as logging
 
 from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import driver_utils
+from ovn_bgp_agent.drivers.openstack.utils import evpn
 from ovn_bgp_agent.drivers.openstack.utils import frr
 from ovn_bgp_agent.utils import linux_net
 
@@ -24,11 +26,44 @@ CONF = cfg.CONF
 LOG = logging.getLogger(__name__)
 
 
-def announce_ips(port_ips):
+def announce_ips(port_ips, ips_info=None):
+    if CONF.exposing_method in [constants.EXPOSE_METHOD_VRF]:
+        if ips_info is None or ips_info.get('bridge_device', None) is None:
+            return
+
+        # Lookup the EVPN vlan dev
+        vlan_dev = evpn.lookup_vlan(ips_info['bridge_device'],
+                                    ips_info['bridge_vlan'])
+
+        # Split the via ip's per ip version
+        via = driver_utils.ips_per_version(ips_info.get('via', []))
+
+        for ip in port_ips:
+            # Add route for each ip in routing table.
+            if via:
+                ver = linux_net.get_ip_version(ip)
+                vlan_dev.add_route(None, ip, None, via=via.get(ver))
+            else:
+                vlan_dev.add_route(None, ip, ips_info['mac'])
+        return
+
     linux_net.add_ips_to_dev(CONF.bgp_nic, port_ips)
 
 
-def withdraw_ips(port_ips):
+def withdraw_ips(port_ips, ips_info=None):
+    if CONF.exposing_method in [constants.EXPOSE_METHOD_VRF]:
+        if ips_info is None or ips_info.get('bridge_device', None) is None:
+            return
+
+        # Lookup the EVPN vlan dev
+        vlan_dev = evpn.lookup_vlan(ips_info['bridge_device'],
+                                    ips_info['bridge_vlan'])
+
+        for ip in port_ips:
+            # Add route for each ip in routing table.
+            vlan_dev.del_route(None, ip)
+        return
+
     linux_net.del_ips_from_dev(CONF.bgp_nic, port_ips)
 
 

ovn_bgp_agent/drivers/openstack/utils/driver_utils.py

@@ -85,6 +85,28 @@ def is_pf_lb(lb):
     return check_name_prefix(lb, constants.OVN_LB_PF_NAME_PREFIX)
 
 
+def ips_per_version(ips: 'list[str]') -> 'dict[int, str]':
+    '''Separate list of ips into ip versions.
+
+    For example, this list ['10.0.0.1/32', 'fe80::1/128'] will be converted
+    to dictionary {
+        4: '10.0.0.1',
+        6: 'fe80::1',
+    }
+
+    If there are more than 1 ip for the same ip version, it will overwrite
+    the previous ip for that ip version.
+    '''
+    ip_list = {constants.IP_VERSION_4: None,
+               constants.IP_VERSION_6: None}
+
+    for ip in ips:
+        ver = linux_net.get_ip_version(ip)
+        ip_list[ver] = ipaddress.ip_address(ip.split('/')[0]).compressed
+
+    return ip_list
+
+
 def get_prefixes_from_ips(ips: 'list[str]') -> 'list[str]':
     '''Return the network address for any given ip (with mask)
 
@@ -106,3 +128,30 @@ def remove_port_from_ip(ip_address):
     if ip_address[last_colon_index + 1:].isdigit():
         return ip_address[:last_colon_index]
     return ip_address
+
+
+def get_port_vlan(port):
+    '''Will return the tag of the given logical switch port row as string
+
+    If the vlan tag is not configured, it will return the value of
+    constants.VLAN_ID_UNTAGGED
+    '''
+    if port.tag:
+        return str(port.tag[0])
+    return str(constants.VLAN_ID_UNTAGGED)
+
+
+def get_port_vrf_settings(port):
+    """Create a comparable object with the settings of the vrf
+
+    Returns None if the settings are not found in the port, otherwise it
+    will return a string value. Either an empty string or a concatenation
+    of the type and vni, e.g. l3::1001
+    """
+    if hasattr(port, 'external_ids'):
+        try:
+            return "%s::%s" % (
+                port.external_ids[constants.OVN_EVPN_TYPE_EXT_ID_KEY],
+                port.external_ids[constants.OVN_EVPN_VNI_EXT_ID_KEY])
+        except (AttributeError, KeyError):
+            return ''

ovn_bgp_agent/drivers/openstack/utils/evpn.py

@@ -0,0 +1,541 @@
+# Copyright 2024 team.blue/nl
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import collections
+import netaddr
+
+from oslo_config import cfg
+from oslo_log import log as logging
+
+from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import driver_utils
+from ovn_bgp_agent.drivers.openstack.utils import frr
+from ovn_bgp_agent.drivers.openstack.utils import ovs
+from ovn_bgp_agent import exceptions
+from ovn_bgp_agent.utils import linux_net
+
+CONF = cfg.CONF
+LOG = logging.getLogger(__name__)
+
+# dictionary to hold all evpn bridge classes
+local_bridges: 'dict[str, EvpnBridge]' = {}
+
+# dictionary to hold all vlandev mappings, based on port uuid
+local_vlandevs: 'dict[str, VlanDev]' = {}
+
+
+class EvpnBridge:
+    def __init__(self, ovs_bridge: str, vni: int, evpn_opts: dict,
+                 mode=constants.OVN_EVPN_TYPE_L3, ovs_flows: dict = None):
+
+        if not CONF.evpn_local_ip:
+            LOG.error("EVPN device must have an IP associated for the "
+                      "VXLAN local ip")
+            raise exceptions.ConfOptionRequired(option='evpn_local_ip')
+
+        self.ovs_bridge = ovs_bridge
+        self.vni = vni
+        self.mode = mode
+        self.ovs_flows = ovs_flows or {}
+
+        self.vrf_name = '%s%s' % (constants.OVN_EVPN_VRF_PREFIX, vni)
+        self.bridge_name = '%s%s' % (constants.OVN_EVPN_BRIDGE_PREFIX, vni)
+        self.vxlan_name = '%s%s' % (constants.OVN_EVPN_VXLAN_PREFIX, vni)
+        self.local_ip = CONF.evpn_local_ip
+        self.evpn_opts = dict(
+            vni=self.vni,
+            local_ip=self.local_ip,
+            vrf_name=self.vrf_name,
+            redistribute=['connected', 'kernel'],
+        )
+        self.evpn_opts.update(evpn_opts)
+
+        self.vlans: dict[str, 'VlanDev'] = {}
+        self._setup_done = False
+
+    def setup(self):
+        if self._setup_done:
+            return
+
+        LOG.debug('Creating bridge %s', self.bridge_name)
+        linux_net.ensure_bridge(self.bridge_name)
+
+        LOG.debug('Creating vxlan interface %s for vni %s with local ip %s',
+                  self.vxlan_name, self.vni, self.local_ip)
+        linux_net.ensure_vxlan(self.vxlan_name, self.vni, self.local_ip,
+                               CONF.evpn_udp_dstport)
+
+        LOG.debug('Connect vxlan interface %s to bridge %s',
+                  self.vxlan_name, self.bridge_name)
+        linux_net.set_master_for_device(self.vxlan_name, self.bridge_name)
+
+        LOG.debug('Disable learning for vxlan device %s', self.vxlan_name)
+        linux_net.disable_learning_vxlan_intf(self.vxlan_name)
+
+        LOG.debug('Configure FRR VRF (add)')
+        frr.vrf_reconfigure(self.evpn_opts, 'add-vrf')
+
+        if self.mode == constants.OVN_EVPN_TYPE_L3:
+            LOG.debug('Create L3 EVPN devices')
+            linux_net.ensure_vrf(self.vrf_name, self.vni)
+
+            LOG.debug('Attach bridge %s to vrf %s',
+                      self.bridge_name, self.vrf_name)
+            linux_net.set_master_for_device(self.bridge_name, self.vrf_name)
+
+        self._setup_done = True
+
+    def _eval_disconnect(self):
+        if not self._setup_done:
+            return
+
+        connected = [1 for v in self.vlans.values() if v._setup_done]
+        if len(connected) == 0:
+            # All vlan interfaces are unprovisioned, proceed with disconnect
+            return self.disconnect()
+
+        LOG.debug('No disconnect needed, there are still %s vlans active',
+                  len(connected))
+
+    def disconnect(self):
+        LOG.info('Disconnecting evpn bridge %s',
+                 self.vrf_name)
+
+        for devname in [self.bridge_name, self.vxlan_name, self.vrf_name]:
+            LOG.info('Delete device %s', devname)
+            linux_net.delete_device(devname)
+
+        # We need to do the frr reconfigure after deleting all devices.
+        # otherwise, frr will throw an error that it can only delete
+        # inactive vrf's
+        LOG.debug('Configure FRR VRF (del)')
+        frr.vrf_reconfigure(self.evpn_opts, action="del-vrf")
+
+        self._setup_done = False
+
+    def connect_vlan(self, port):
+        vlan_tag = driver_utils.get_port_vlan(port)
+        if vlan_tag not in self.vlans:
+            self.vlans[vlan_tag] = VlanDev(self, port)
+
+        return self.vlans[vlan_tag]
+
+    def get_vlan(self, vlan: 'int|str|None') -> 'VlanDev':
+        if vlan is None:
+            vlan = constants.VLAN_ID_UNTAGGED
+
+        return self.vlans[str(vlan)]
+
+
+class VlanDev:
+    def __init__(self, bridge: 'EvpnBridge', port):
+        self.bridge = bridge
+        self.port = port  # localnet port
+
+        uuid = str(port.uuid)[0:11]
+        self.veth_vrf = constants.OVN_EVPN_VETH_VRF_UUID_PREFIX + uuid
+        self.veth_ovs = constants.OVN_EVPN_VETH_OVS_UUID_PREFIX + uuid
+
+        if uuid in local_vlandevs:
+            # It already exists, but probably in another vni
+            local_vlandevs[uuid].teardown()
+
+        local_vlandevs[uuid] = self
+
+        self.vlan_tag = driver_utils.get_port_vlan(port)
+
+        # Will be filled by the @property with the mac address of
+        # the vrf-vlan interface
+        self._lladdr = None
+
+        # boolean to indicate if setup is required for this interface
+        self._setup_done = False
+        self._veth_created = False
+
+        # list of custom addresses to use during setup, before adding the
+        # 169.254.x.x address, making another possibly public ip the
+        # primary ip in traceroutes
+        self._custom_ips = set()
+
+        # list with tuple of (task, args, kwargs) we should execute once the
+        # setup has completed. For example to run frr config for
+        # ipv6 neighbor discovery, or to add ip's to the interface.
+        self._post_setup_tasks = []
+
+        self._agent_routing_tables_routes = collections.defaultdict(list)
+        self._route_table_routes = {}
+
+    def _set_agent_cache(self, routing_tables_routes):
+        if routing_tables_routes is not None:
+            self._agent_routing_tables_routes = (
+                routing_tables_routes[self.veth_vrf])
+
+    @property
+    def lladdr(self):
+        if not self._lladdr:
+            if not self._veth_created:
+                self.setup()
+            self._lladdr = linux_net.get_interface_address(self.veth_vrf)
+        return self._lladdr
+
+    def setup(self):
+        if self._setup_done:
+            return
+
+        # Run the setup of the bridge.
+        self.bridge.setup()
+
+        LOG.debug('Create VLAN veth interface %s <-> %s',
+                  self.veth_vrf, self.veth_ovs)
+        linux_net.ensure_veth(self.veth_vrf, self.veth_ovs)
+        self._veth_created = True
+
+        # Connect the veth_ovs to ovs
+        ovs_vlan_tag = self.vlan_tag
+        if self.vlan_tag == '0':
+            ovs_vlan_tag = None
+
+        ovs.add_device_to_ovs_bridge(self.veth_ovs,
+                                     self.bridge.ovs_bridge,
+                                     vlan_tag=ovs_vlan_tag)
+
+        # Connect veth to bridge for L2
+        if self.bridge.mode == constants.OVN_EVPN_TYPE_L2:
+            linux_net.set_master_for_device(self.veth_vrf,
+                                            self.bridge.bridge_name)
+            self._setup_done = True
+            return
+
+        # Connect veth to vrf for L3
+        # Create vrf interface, connect bridge and veth_vrf to it.
+        LOG.debug('Configure L3 for EVPN devices')
+        linux_net.set_master_for_device(self.veth_vrf, self.bridge.vrf_name)
+
+        if self._custom_ips:
+            linux_net.add_ips_to_dev(self.veth_vrf, ips=list(self._custom_ips))
+
+        # Add 169.254.x.x address to veth_vrf for ipv4 and ipv6
+        linux_net.ensure_arp_ndp_enabled_for_bridge(
+            self.veth_vrf, offset=int(self.vlan_tag), vlan_tag=self.vlan_tag
+        )
+
+        # Configure mac on the veth interface to be the same on all hosts
+        offset = _offset_for_vni_and_vlan(self.bridge.vni, self.vlan_tag)
+        linux_net.ensure_anycast_mac_for_interface(
+            self.veth_vrf, offset=offset
+        )
+
+        # Make sure ipv4 and ipv6 forwarding is enabled
+        linux_net.enable_routing_for_interfaces(self.veth_vrf,
+                                                self.bridge.bridge_name)
+
+        # As long as we use 169.254.x.x addresses, we require proxy arp to be
+        # there for initial router discovery
+        linux_net.enable_proxy_arp(self.veth_vrf)
+        linux_net.enable_proxy_ndp(self.veth_vrf)
+
+        ovs_ok = self._setup_ovs()
+        if ovs_ok is False:
+            LOG.error('Unable to setup ovs, a retry will pick it up.')
+            return
+
+        # Any post-setup tasks to run.
+        for method, a, kw in self._post_setup_tasks:
+            method(*a, **kw)
+
+        self._setup_done = True
+
+    def _setup_ovs(self):
+        try:
+            in_port = ovs.get_ovs_patch_port_ofport(self.port.name)
+            LOG.debug('ovs in-port: %s', in_port)
+        except Exception:
+            return False
+
+        ovs_flows = self.bridge.ovs_flows
+        ovs_bridge = self.bridge.ovs_bridge
+
+        pmm = ovs_flows[ovs_bridge].setdefault('port-mac-mapping', {})
+        pmm[in_port] = self.lladdr
+
+        ovs.ensure_mac_tweak_flows(ovs_bridge,
+                                   self.lladdr,
+                                   [in_port],
+                                   constants.OVS_RULE_COOKIE)
+
+        ovs.remove_extra_ovs_flows(ovs_flows, ovs_bridge,
+                                   constants.OVS_RULE_COOKIE)
+
+    def _eval_disconnect(self):
+        if not self._setup_done:
+            return
+
+        if len(self._agent_routing_tables_routes) == 0:
+            return self.disconnect()
+
+        LOG.debug('No disconnect needed, there are still %s announcements',
+                  len(self._agent_routing_tables_routes))
+
+    def disconnect(self):
+        LOG.info('Disconnecting vlan interface %s.%s',
+                 self.bridge.vrf_name, self.vlan_tag)
+
+        LOG.info('Remove device %s from ovs bridge %s', self.veth_ovs,
+                 self.bridge.ovs_bridge)
+        ovs.del_device_from_ovs_bridge(self.veth_ovs,
+                                       self.bridge.ovs_bridge)
+
+        LOG.info('Delete device %s', self.veth_vrf)
+        linux_net.delete_device(self.veth_vrf)
+
+        self._veth_created = False
+        self._setup_done = False
+        self.bridge._eval_disconnect()
+
+    def teardown(self):
+        LOG.info('Running teardown for vlandev %s (vni change)', self.veth_vrf)
+        self.disconnect()
+        del self.bridge.vlans[self.vlan_tag]
+
+    def _run(self, method, *a, **kw):
+        # Run the method if setup is done, otherwise, run them when setup
+        # is called
+        if not self._setup_done:
+            self._post_setup_tasks.append([method, a, kw])
+        else:
+            method(*a, **kw)
+
+    def process_dhcp_opts(self, dhcp_opts):
+        '''Add IP's or router advertisements from configured dhcp options
+
+        For networks that have DHCP enabled (and have lsp with dhcp options),
+        we can add the IP of the gateway on our vlan interface. Then OVN is
+        able to discover the IP.
+
+        Also if IPv6 is configured, we should enable router advertisements
+        through FRR (since it has it built-in anyway), so vm's can then
+        receive the router information from the 'provider' side.
+        '''
+
+        for opt in dhcp_opts:
+            ver = netaddr.IPNetwork(opt.cidr).version
+
+            if opt.options.get('router', False):
+                LOG.debug('Adding IPv%s gateway ip: %s',
+                          ver, opt.options['router'])
+                self.add_ips([opt.options['router']])
+
+            if ver == 6 and opt.cidr:
+                LOG.debug('Configure ipv6nd for %s and opts %s',
+                          opt.cidr, opt.options)
+                self.configure_nd(opt.cidr, opts=opt.options)
+
+    def configure_nd(self, cidr, opts):
+        self._run(frr.nd_reconfigure, self.veth_vrf, cidr, opts)
+
+    def add_ips(self, ips: list):
+        self._custom_ips.update(ips)
+        self._run(linux_net.add_ips_to_dev, self.veth_vrf, ips=ips)
+
+    def add_route(self, routing_tables_routes: 'dict | None', ip: str,
+                  mac: 'str | None', via: 'str | None' = None):
+        '''Will add route to the routing table for this vlan_dev
+
+        Please make sure pass along the routing_tables_routes dictionary at
+        least the first time a route is added (for example when exposing the
+        lrp or lsp). Then with a expose_remote_ip we can re-use the reference
+        from the agent set earlier.
+        '''
+        self.setup()  # setup the bridge and vlan, if not already done.
+        self._set_agent_cache(routing_tables_routes)
+
+        if self.bridge.mode != constants.OVN_EVPN_TYPE_L3:
+            return
+
+        mask = None
+        if '/' in ip:
+            ip, mask = ip.split('/')
+
+        self._agent_routing_tables_routes.append({
+            'ip': ip, 'mask': mask, 'mac': mac, 'via': via,
+        })
+        LOG.debug('Add route %s/%s via %s dev %s table %s',
+                  ip, mask, via, self.veth_vrf, self.bridge.vni)
+        linux_net.add_ip_route(self._route_table_routes, ip, self.bridge.vni,
+                               self.veth_vrf, mask=mask, via=via)
+
+        # When a floating ip is passed along, it is a set of mac
+        # addresses, so ensure we are always processing a list.
+        for lladdr in _ensure_list(mac):
+            LOG.debug('Add neigh %s -> %s dev %s', ip, mac, self.veth_vrf)
+            linux_net.add_ip_nei(ip, lladdr, self.veth_vrf)
+
+    def del_route(self, routing_tables_routes: 'dict | None', ip: str,
+                  lladdr: 'str | None' = None):
+        '''Will remove the route from the routing table for this vlan_dev
+
+        Please make sure pass along the routing_tables_routes dictionary at
+        least the first time a route is added (for example when exposing the
+        lrp or lsp). Then with a withdraw_remote_ip we can re-use the reference
+        from the agent set earlier.
+
+        lladdr is optional, as it will be fetched from the internal
+        route table dictionary
+        '''
+
+        if self.bridge.mode != constants.OVN_EVPN_TYPE_L3:
+            return
+        self._set_agent_cache(routing_tables_routes)
+
+        # When a floating ip is passed along, it is a set of mac
+        # addresses, so ensure we are always processing a list.
+
+        mask = None
+        if '/' in ip:
+            ip, mask = ip.split('/')
+
+        route = _find_route_info(self._agent_routing_tables_routes, ip)
+
+        # Remove route from vrf
+        linux_net.del_ip_route(self._route_table_routes, ip, self.bridge.vni,
+                               self.veth_vrf, mask=mask or route['mask'],
+                               via=route['via'])
+
+        # Remove any neighbor information for route.
+        for mac in _ensure_list(lladdr or route['mac']):
+            linux_net.del_ip_nei(ip, mac, self.veth_vrf)
+
+        if route in self._agent_routing_tables_routes:
+            self._agent_routing_tables_routes.remove(route)
+
+        self._eval_disconnect()
+
+    def cleanup_excessive_routes(self, routing_tables_routes: dict):
+        if not self._setup_done:
+            return
+
+        self._set_agent_cache(routing_tables_routes)
+
+        # Get all routes on host for our vrf and our veth_vrf
+        intf_idx = linux_net.get_interface_index(self.veth_vrf)
+        current_routes = dict([
+            (r.get_attr('RTA_DST'), r)
+            for r in linux_net._get_table_routes(self.bridge.vni)
+            if r.get_attr('RTA_OIF') == intf_idx and
+            r['type'] == constants.ROUTE_TYPE_UNICAST and
+            r.get_attr('RTA_DST') not in ('fe80::') and
+            not r.get_attr('RTA_DST').startswith(
+                constants.NDP_IPV6_PREFIX)
+        ])
+
+        # Create set with prefixes currently on host
+        prefixes = {r.get_attr('RTA_DST') for r in current_routes.values()}
+
+        # Create set with prefixes we maintain
+        exposed_prefixes = {r['ip']
+                            for r in self._agent_routing_tables_routes}
+
+        if len(prefixes - exposed_prefixes) == 0:
+            LOG.debug('No excessive routes to remove.')
+
+        for ip in prefixes - exposed_prefixes:
+            LOG.info('Remove excessive route %s', ip)
+            kernel_route = current_routes[ip]
+            route = _find_route_info(self._agent_routing_tables_routes, ip)
+            if ((route['mask'] and
+                    int(route['mask']) != kernel_route['dst_len']) or
+                    route['via'] != kernel_route.get_attr('RTA_GATEWAY')):
+                self._agent_routing_tables_routes.append({
+                    'ip': ip, 'mask': kernel_route['dst_len'], 'mac': None,
+                    'via': kernel_route.get_attr('RTA_GATEWAY'),
+                })
+
+            self.del_route(routing_tables_routes, ip)
+
+
+def _ensure_list(var):
+    if var is None:
+        return []
+
+    if not isinstance(var, (list, tuple, set)):
+        var = [var]
+
+    return var
+
+
+def _find_route_info(routes: 'list[dict]', ip: str):
+    for r in routes:
+        if r['ip'] == ip:
+            return r
+
+    return {'ip': ip, 'mask': None, 'mac': None, 'via': None}
+
+
+def _offset_for_vni_and_vlan(vni: int, vlan: str):
+    '''Generate a offset (in numeric system), based on the vni and vlan
+
+    vni has range 1-16777214 (6 bytes)
+    vlan has range 0-4094 (3 bytes)
+
+    It will transform the vni to a 6 digit hex, append the 3 digit vlan hex
+    and transform it back to a integer.
+    '''
+    if vni > 16777214:
+        LOG.warning('Configured vni value %d is too big (range 1-16777214)',
+                    vni)
+        vni = vni % 0xffffff  # reset vni, to prevent overflow.
+
+    if int(vlan) > 4094:
+        LOG.warning('Configured vlan value %d is too big (range 0-4094)',
+                    int(vlan))
+        vlan = int(vlan) % 0xfff  # reset vlan, to prevent overflow.
+
+    return int(''.join([
+        ('%x' % vni).zfill(6),
+        ('%x' % int(vlan)).zfill(4),
+    ]), 16)
+
+
+def setup(ovs_bridge, vni, evpn_opts, mode=constants.OVN_EVPN_TYPE_L3,
+          ovs_flows={}) -> EvpnBridge:
+    # This method will either create the EvpnBridge or return the one that
+    # already exists for the current vni.
+
+    vni = int(vni)  # make sure the vni is a int, for lookup purposes
+
+    if local_bridges.get(vni, None) is None:
+        local_bridges[vni] = EvpnBridge(ovs_bridge, vni, evpn_opts,
+                                        mode=mode, ovs_flows=ovs_flows)
+    else:
+        local_bridges[vni].ovs_flows = ovs_flows
+
+    return local_bridges[vni]
+
+
+def lookup(ovs_bridge: str, vlan: str) -> EvpnBridge:
+    if vlan is None:
+        vlan = constants.VLAN_ID_UNTAGGED
+
+    for br in local_bridges.values():
+        if br.ovs_bridge == ovs_bridge:
+            if str(vlan) in br.vlans:
+                return br
+
+    raise KeyError('Could not locate EVPN for bridge %s and/or vlan %s' % (
+                   ovs_bridge, vlan))
+
+
+def lookup_vlan(ovs_bridge: str, vlan: str) -> VlanDev:
+    bridge = lookup(ovs_bridge, vlan)
+    return bridge.get_vlan(vlan)

ovn_bgp_agent/drivers/openstack/utils/frr.py

@@ -16,15 +16,31 @@ import json
 import tempfile
 
 from jinja2 import Template
+from oslo_config import cfg
 from oslo_log import log as logging
 
 from ovn_bgp_agent import constants
 import ovn_bgp_agent.privileged.vtysh
 
+CONF = cfg.CONF
+
 LOG = logging.getLogger(__name__)
 
 DEFAULT_REDISTRIBUTE = {'connected'}
 
+CONFIGURE_ND_TEMPLATE = '''
+interface {{ intf }}
+{% if is_dhcpv6 %}
+ ipv6 nd managed-config-flag
+{% endif %}
+{% for server in dns_servers %}
+ ipv6 nd rdnss {{ server }}
+{% endfor %}
+ ipv6 nd prefix {{ prefix }}
+ no ipv6 nd suppress-ra
+exit
+'''
+
 ADD_VRF_TEMPLATE = '''
 vrf {{ vrf_name }}
   vni {{ vni }}
@@ -44,12 +60,28 @@ router bgp {{ bgp_as }} vrf {{ vrf_name }}
   address-family l2vpn evpn
     advertise ipv4 unicast
     advertise ipv6 unicast
+{% if route_distinguishers|length > 0 %}
+    rd {{ route_distinguishers[0] }}
+{% else %}
+    rd {{ local_ip }}:{{ vni }}
+{% endif %}
+{% for route_target in route_targets %}
+    route-target import {{ route_target }}
+    route-target export {{ route_target }}
+{% endfor %}
+{% for route_target in export_targets %}
+    route-target export {{ route_target }}
+{% endfor %}
+{% for route_target in import_targets %}
+    route-target import {{ route_target }}
+{% endfor %}
   exit-address-family
 
 '''
 
 DEL_VRF_TEMPLATE = '''
 no vrf {{ vrf_name }}
+no interface veth-{{ vrf_name }}
 no router bgp {{ bgp_as }} vrf {{ vrf_name }}
 
 '''
@@ -118,6 +150,33 @@ def set_default_redistribute(redist_opts):
     DEFAULT_REDISTRIBUTE.update(redist_opts)
 
 
+def nd_reconfigure(interface, prefix, opts):
+    LOG.info('FRR IPv6 ND reconfiguration (intf %s, prefix %s)', interface,
+             prefix)
+    nd_template = Template(CONFIGURE_ND_TEMPLATE)
+
+    # Need to define what setting is for SLAAC
+    if (not opts.get('dhcpv6_stateless', False) or
+            opts.get('dhcpv6_stateless', '') not in ('true', True)):
+        prefix += ' no-autoconfig'
+
+    # Parse dns servers from dhcp options.
+    dns_servers = []
+    if opts.get('dns_server'):
+        dns_servers = [s.strip() for s in opts['dns_server'][1:-1].split(',')]
+
+    is_dhcpv6 = True  # Need a better way to define this one.
+
+    nd_config = nd_template.render(
+        intf=interface,
+        prefix=prefix,
+        dns_servers=dns_servers,
+        is_dhcpv6=is_dhcpv6,
+    )
+
+    _run_vtysh_config_with_tempfile(nd_config)
+
+
 def vrf_leak(vrf, bgp_as, bgp_router_id=None, template=LEAK_VRF_TEMPLATE):
     LOG.info("Add VRF leak for VRF %s on router bgp %s", vrf, bgp_as)
     if not bgp_router_id:
@@ -136,21 +195,29 @@ def vrf_leak(vrf, bgp_as, bgp_router_id=None, template=LEAK_VRF_TEMPLATE):
 def vrf_reconfigure(evpn_info, action):
     LOG.info("FRR reconfiguration (action = %s) for evpn: %s",
              action, evpn_info)
-    if action == "add-vrf":
+
-        vrf_template = Template(ADD_VRF_TEMPLATE)
+    # If we have more actions, we can define them in this list.
-        vrf_config = vrf_template.render(
+    vrf_templates = {
-            vrf_name="{}{}".format(constants.OVN_EVPN_VRF_PREFIX,
+        'add-vrf': ADD_VRF_TEMPLATE,
-                                   evpn_info['vni']),
+        'del-vrf': DEL_VRF_TEMPLATE,
-            bgp_as=evpn_info['bgp_as'],
+    }
-            redistribute=DEFAULT_REDISTRIBUTE,
+    if action not in vrf_templates:
-            vni=evpn_info['vni'])
-    elif action == "del-vrf":
-        vrf_template = Template(DEL_VRF_TEMPLATE)
-        vrf_config = vrf_template.render(
-            vrf_name="{}{}".format(constants.OVN_EVPN_VRF_PREFIX,
-                                   evpn_info['vni']),
-            bgp_as=evpn_info['bgp_as'])
-    else:
         LOG.error("Unknown FRR reconfiguration action: %s", action)
         return
+
+    # Set default opts, so all params are available for the templates
+    # Then update them with evpn_info
+    opts = dict(route_targets=[], route_distinguishers=[], export_targets=[],
+                import_targets=[], local_ip=CONF.evpn_local_ip,
+                redistribute=DEFAULT_REDISTRIBUTE,
+                bgp_as=CONF.bgp_AS, vrf_name='', vni=0)
+    opts.update(evpn_info)
+
+    if not opts['vrf_name']:
+        opts['vrf_name'] = "{}{}".format(constants.OVN_EVPN_VRF_PREFIX,
+                                         evpn_info['vni'])
+
+    vrf_template = Template(vrf_templates.get(action))
+    vrf_config = vrf_template.render(**opts)
+
     _run_vtysh_config_with_tempfile(vrf_config)

ovn_bgp_agent/drivers/openstack/utils/ovn.py

@@ -391,14 +391,36 @@ class OvsdbNbOvnIdl(nb_impl_idl.OvnNbApiIdlImpl, Backend):
 
     def get_network_vlan_tag_by_network_name(self, network_name):
         tags = []
-        cmd = self.db_find_rows('Logical_Switch_Port', ('type', '=',
+        for row in self.get_localnet_ports_by_network_name(network_name):
-                                constants.OVN_LOCALNET_VIF_PORT_TYPE))
+            if row.tag:
-        for row in cmd.execute(check_error=True):
-            if (row.tag and row.options and
-                    row.options.get('network_name') == network_name):
                 tags.append(row.tag[0])
         return tags
 
+    def get_localnet_ports_by_network_name(self, network_name):
+        conditions = (
+            ('options', '=', {'network_name': network_name}),
+            ('type', '=', constants.OVN_LOCALNET_VIF_PORT_TYPE),
+        )
+        cmd = self.db_find_rows('Logical_Switch_Port', *conditions)
+        for row in cmd.execute(check_error=True):
+            yield row
+
+    def get_bgpvpn_networks_for_ports(self, ports,
+                                      vpn_type=constants.OVN_EVPN_TYPE_L3):
+        cmd = self.db_find_rows(
+            'Logical_Switch',
+            ('external_ids', '=', {constants.OVN_EVPN_TYPE_EXT_ID_KEY:
+                                   vpn_type}))
+        networks = []
+        for row in cmd.execute(check_error=True):
+            if not row.ports:
+                continue
+
+            localnet_ports = [p for p in row.ports if p in ports]
+            if localnet_ports:
+                networks.append(row)
+        return networks
+
     def ls_has_virtual_ports(self, logical_switch):
         ls = self.lookup('Logical_Switch', logical_switch)
         for port in ls.ports:

ovn_bgp_agent/drivers/openstack/utils/ovs.py

@@ -124,12 +124,19 @@ def ensure_mac_tweak_flows(bridge, mac, ports, cookie):
 def remove_extra_ovs_flows(ovs_flows, bridge, cookie):
     expected_flows = []
     for port in ovs_flows[bridge].get('in_port'):
-        flow = ("=900,ip,in_port={} actions=mod_dl_dst:{},NORMAL".format(
+        pmm = ovs_flows[bridge].get('port-mac-mapping', {})
-            port, ovs_flows[bridge]['mac']))
+        lladdr = pmm.get(port, ovs_flows[bridge]['mac'])
-        expected_flows.append(flow)
+
-        flow_v6 = ("=900,ipv6,in_port={} actions=mod_dl_dst:{},NORMAL".format(
+        for flow in [
-            port, ovs_flows[bridge]['mac']))
+            # Add ipv4 flow in 'normal' and OpenFlow13 format
-        expected_flows.append(flow_v6)
+            "=900,ip,in_port={} actions=mod_dl_dst:{},NORMAL",
+            "=900,ip,in_port={} actions=set_field:{}->eth_dst,NORMAL",
+
+            # Add ipv6 flow in 'normal' and OpenFlow13 format
+            "=900,ipv6,in_port={} actions=mod_dl_dst:{},NORMAL",
+            "=900,ipv6,in_port={} actions=set_field:{}->eth_dst,NORMAL",
+        ]:
+            expected_flows.append(flow.format(port, lladdr))
 
     cookie_id = "cookie={}/-1".format(cookie)
     current_flows = get_bridge_flows(bridge, cookie_id)

ovn_bgp_agent/drivers/openstack/utils/wire.py

@@ -12,10 +12,15 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+import ast
+
 from oslo_config import cfg
 from oslo_log import log as logging
 
 from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import driver_utils
+from ovn_bgp_agent.drivers.openstack.utils import evpn
+from ovn_bgp_agent.drivers.openstack.utils import ovn
 from ovn_bgp_agent.drivers.openstack.utils import ovs
 from ovn_bgp_agent import exceptions as agent_exc
 from ovn_bgp_agent.utils import helpers
@@ -30,9 +35,13 @@ def ensure_base_wiring_config(idl, ovs_idl, ovn_idl=None, routing_tables={}):
     if CONF.exposing_method == constants.EXPOSE_METHOD_UNDERLAY:
         return _ensure_base_wiring_config_underlay(idl, ovs_idl,
                                                    routing_tables)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:  # Type 5 evpn
+        return _ensure_base_wiring_config_evpn(idl, ovs_idl)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         return _ensure_base_wiring_config_ovn(ovs_idl, ovn_idl)
 
+    raise agent_exc.UnsupportedWiringConfig(method=CONF.exposing_method)
+
 
 def _ensure_base_wiring_config_underlay(idl, ovs_idl, routing_tables):
     # Get bridge mappings: xxxx:br-ex,yyyy:br-ex2
@@ -68,6 +77,105 @@ def _ensure_base_wiring_config_underlay(idl, ovs_idl, routing_tables):
     return ovn_bridge_mappings, flows_info
 
 
+def _ensure_base_wiring_config_evpn(idl: 'ovn.OvsdbNbOvnIdl|ovn.OvsdbSbOvnIdl',
+                                    ovs_idl: 'ovs.OvsIdl',
+                                    mode=constants.OVN_EVPN_TYPE_L3):
+    # This method will create the bridge mappings and make sure the
+    # vrf bridge and vlans are provisioned
+
+    # Get bridge mappings: xxxx:br-ex,yyyy:br-ex2
+    bridge_mappings = ovs_idl.get_ovn_bridge_mappings()
+
+    ovn_bridge_mappings = {}
+    flows_info = {}  # dictionary to use for mappings with vrf's
+    for bridge_mapping in bridge_mappings:
+        try:
+            # for example: physnet1, br-ex
+            network, bridge = bridge_mapping.split(":", 1)
+        except ValueError:
+            LOG.debug('Invalid bridge mapping: %s', bridge_mapping)
+            continue
+
+        ovn_bridge_mappings[network] = bridge
+        LOG.debug('Setup EVPN base wiring for network %s on bridge %s',
+                  network, bridge)
+
+        # Make sure the bridge exists
+        ovs_idl.idl_ovs.add_br(bridge).execute(check_error=True)
+
+        if bridge not in flows_info:
+            flows_info[bridge] = {
+                'mac': linux_net.get_interface_address(bridge),
+                'in_port': ovs.get_ovs_patch_ports_info(bridge),
+                'evpn': {}
+            }
+
+        # Find all provider networks, and create the vrf's
+        localnet_ports = list(idl.get_localnet_ports_by_network_name(network))
+        if not localnet_ports:
+            LOG.debug('No localnet ports found for network %s', network)
+            continue
+
+        provnets = idl.get_bgpvpn_networks_for_ports(localnet_ports,
+                                                     vpn_type=mode)
+        if not provnets:
+            LOG.debug('No provider networks found for %s %s',
+                      constants.OVN_EVPN_TYPE_EXT_ID_KEY, mode)
+            continue
+
+        for ls in provnets:
+            LOG.info('Network %s (settings: %s)', ls.name, ls.external_ids)
+
+            if constants.OVN_EVPN_VNI_EXT_ID_KEY not in ls.external_ids:
+                LOG.warning('Skipped, VNI required for EVPN VRF setup')
+                continue
+
+            evpn_opts = {}
+            for opt, ext_id_key in constants.EVPN_EXT_ID_MAPPING.items():
+                evpn_opts[opt] = ast.literal_eval(
+                    ls.external_ids.get(ext_id_key, '[]')
+                )
+
+            # Create or return the EVPN bridge
+            evpn_bridge = evpn.setup(
+                ovs_bridge=bridge,
+                vni=ls.external_ids[constants.OVN_EVPN_VNI_EXT_ID_KEY],
+                evpn_opts=evpn_opts,
+                mode=mode,
+                ovs_flows=flows_info,
+            )
+
+            # Connect all VLAN interfaces to this VRF and gather dhcp
+            # options to be configured for l3 mode.
+            evpn_dev, dhcp_opts = _ensure_evpn_vlan_dev(ls, localnet_ports,
+                                                        evpn_bridge,
+                                                        flows_info, bridge)
+
+            if dhcp_opts and evpn_dev:
+                evpn_dev.process_dhcp_opts(dhcp_opts)
+
+    return ovn_bridge_mappings, flows_info
+
+
+def _ensure_evpn_vlan_dev(ls, localnet_ports, evpn_bridge, flows_info, bridge):
+    evpn_dev = None
+    dhcp_opts = set()
+    for port in ls.ports:
+        if port not in localnet_ports:
+            if port.dhcpv4_options:
+                dhcp_opts.update(port.dhcpv4_options)
+            if port.dhcpv6_options:
+                dhcp_opts.update(port.dhcpv6_options)
+            continue
+
+        LOG.info('VLAN tag %s', driver_utils.get_port_vlan(port))
+        evpn_dev = evpn_bridge.connect_vlan(port)
+
+        flows_info[bridge]['evpn'][str(evpn_dev.vlan_tag)] = evpn_bridge
+
+    return evpn_dev, dhcp_opts
+
+
 def _ensure_base_wiring_config_ovn(ovs_idl, ovn_idl):
     """Base configuration for extra OVN cluster instead of kernel networking
 
@@ -376,6 +484,8 @@ def cleanup_wiring(idl, bridge_mappings, ovs_flows, exposed_ips,
         return _cleanup_wiring_underlay(idl, bridge_mappings, ovs_flows,
                                         exposed_ips, routing_tables,
                                         routing_tables_routes)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+        return _cleanup_wiring_evpn(ovs_flows, routing_tables_routes)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         # TODO(ltomasbo): clean up old policies, routes and proxy_arps cidrs
         return True
@@ -432,6 +542,17 @@ def delete_vlan_devices_leftovers(idl, bridge_mappings):
                 linux_net.delete_vlan_device_for_network(ovs_device, vlan)
 
 
+def _cleanup_wiring_evpn(ovs_flows, routing_tables_routes):
+    for flow_conf in ovs_flows.values():
+        for vlan, evpn_bridge in flow_conf.get('evpn', {}).items():
+            LOG.debug('Running cleanup for vrf %s vlan %s',
+                      evpn_bridge.vrf_name, vlan)
+            evpn_bridge.get_vlan(vlan).cleanup_excessive_routes(
+                routing_tables_routes
+            )
+    return True
+
+
 def wire_provider_port(routing_tables_routes, ovs_flows, port_ips,
                        bridge_device, bridge_vlan, localnet, routing_table,
                        proxy_cidrs, lladdr=None, mac=None, ovn_idl=None):
@@ -441,6 +562,12 @@ def wire_provider_port(routing_tables_routes, ovs_flows, port_ips,
                                             bridge_vlan, localnet,
                                             routing_table, proxy_cidrs,
                                             lladdr=lladdr)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+        return _wire_provider_port_evpn(routing_tables_routes, ovs_flows,
+                                        port_ips, bridge_device,
+                                        bridge_vlan, localnet,
+                                        proxy_cidrs,
+                                        mac=mac)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         # We need to add a static mac binding due to proxy-arp issue in
         # core ovn that would reply on the incomming traffic from the LR,
@@ -456,6 +583,10 @@ def unwire_provider_port(routing_tables_routes, port_ips, bridge_device,
                                               bridge_device, bridge_vlan,
                                               routing_table, proxy_cidrs,
                                               lladdr=lladdr)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+        return _unwire_provider_port_evpn(routing_tables_routes, port_ips,
+                                          bridge_device, bridge_vlan,
+                                          lladdr)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         # We need to remove thestatic mac binding added due to proxy-arp issue
         # in core ovn that would reply on the incomming traffic from the LR,
@@ -512,6 +643,27 @@ def _wire_provider_port_underlay(routing_tables_routes, ovs_flows, port_ips,
     return True
 
 
+def _wire_provider_port_evpn(routing_tables_routes, ovs_flows, port_ips,
+                             bridge_device, bridge_vlan, localnet,
+                             proxy_cidrs, mac=None, via=None):
+    try:
+        evpn_dev = evpn.lookup_vlan(bridge_device, bridge_vlan)
+    except KeyError:
+        msg = ('EVPN has not been setup for bridge %s with vlan device %s. '
+               'Either the network has not been configured, or something '
+               'went wrong in the base wiring method.')
+        LOG.warning(msg, bridge_device, bridge_vlan)
+        return
+
+    via = via or {}  # make sure it is at least a empty dictionary
+
+    for ip in port_ips:
+        ver = linux_net.get_ip_version(ip)
+        evpn_dev.add_route(routing_tables_routes, ip, mac, via=via.get(ver))
+
+    return True
+
+
 def _wire_provider_port_ovn(ovn_idl, port_ips, mac):
     cmds = []
     port = "{}-openstack".format(constants.OVN_CLUSTER_ROUTER)
@@ -561,6 +713,24 @@ def _unwire_provider_port_underlay(routing_tables_routes, port_ips,
     return True
 
 
+def _unwire_provider_port_evpn(routing_tables_routes, port_ips,
+                               bridge_device, bridge_vlan, lladdr):
+    # locate the evpn_dev, based on bridge and vlan
+    try:
+        evpn_dev = evpn.lookup_vlan(bridge_device, bridge_vlan)
+    except KeyError:
+        msg = ('EVPN has not been setup for bridge %s with vlan device %s. '
+               'Either the network has not been configured, or something '
+               'went wrong in the base wiring method.')
+        LOG.warning(msg, bridge_device, bridge_vlan)
+        return
+
+    for ip in port_ips:
+        evpn_dev.del_route(routing_tables_routes, ip, lladdr)
+
+    return True
+
+
 def _unwire_provider_port_ovn(ovn_idl, port_ips):
     cmds = []
     port = "{}-openstack".format(constants.OVN_CLUSTER_ROUTER)
@@ -579,6 +749,9 @@ def wire_lrp_port(routing_tables_routes, ip, bridge_device, bridge_vlan,
         return _wire_lrp_port_underlay(routing_tables_routes, ip,
                                        bridge_device, bridge_vlan,
                                        routing_tables, cr_lrp_ips)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+        return _wire_lrp_port_evpn(routing_tables_routes, ip, bridge_device,
+                                   bridge_vlan, cr_lrp_ips)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         # TODO(ltomasbo): Add flow on br-ex(-X)
         # ovs-ofctl add-flow br-ex
@@ -622,12 +795,35 @@ def _wire_lrp_port_underlay(routing_tables_routes, ip, bridge_device,
     return True
 
 
+def _wire_lrp_port_evpn(routing_tables_routes, ip, bridge_device,
+                        bridge_vlan, cr_lrp_ips):
+
+    # Generate the via addresses
+    via = driver_utils.ips_per_version(cr_lrp_ips)
+
+    try:
+        evpn_dev = evpn.lookup_vlan(bridge_device, bridge_vlan)
+    except KeyError:
+        msg = ('EVPN has not been setup for bridge %s with vlan device %s. '
+               'Either the network has not been configured, or something '
+               'went wrong in the base wiring method.')
+        LOG.warning(msg, bridge_device, bridge_vlan)
+        return
+
+    ver = linux_net.get_ip_version(ip)
+    evpn_dev.add_route(routing_tables_routes, ip, None, via=via.get(ver))
+    return True
+
+
 def unwire_lrp_port(routing_tables_routes, ip, bridge_device, bridge_vlan,
                     routing_tables, cr_lrp_ips):
     if CONF.exposing_method == constants.EXPOSE_METHOD_UNDERLAY:
         return _unwire_lrp_port_underlay(routing_tables_routes, ip,
                                          bridge_device, bridge_vlan,
                                          routing_tables, cr_lrp_ips)
+    elif CONF.exposing_method == constants.EXPOSE_METHOD_VRF:
+        return _unwire_lrp_port_evpn(routing_tables_routes, ip,
+                                     bridge_device, bridge_vlan)
     elif CONF.exposing_method == constants.EXPOSE_METHOD_OVN:
         # TODO(ltomasbo): Remove flow(s) and router route
         return
@@ -660,3 +856,20 @@ def _unwire_lrp_port_underlay(routing_tables_routes, ip, bridge_device,
                 via=cr_lrp_ip)
     LOG.debug("Deleted IP Routes for network %s", ip)
     return True
+
+
+def _unwire_lrp_port_evpn(routing_tables_routes, ip, bridge_device,
+                          bridge_vlan):
+    # locate the evpn_dev, based on bridge and vlan
+    try:
+        evpn_dev = evpn.lookup_vlan(bridge_device, bridge_vlan)
+    except KeyError:
+        msg = ('EVPN has not been setup for bridge %s with vlan device %s. '
+               'Either the network has not been configured, or something '
+               'went wrong in the base wiring method.')
+        LOG.warning(msg, bridge_device, bridge_vlan)
+        return
+
+    evpn_dev.del_route(routing_tables_routes, ip)
+
+    return True

ovn_bgp_agent/drivers/openstack/watchers/base_watcher.py

@@ -85,6 +85,15 @@ class OVNLBEvent(Event):
                                    constants.OVN_LB_VIP_FIP_EXT_ID_KEY)
 
 
+class LogicalSwitchChassisEvent(Event):
+    def __init__(self, bgp_agent, events):
+        self.agent = bgp_agent
+        table = 'Logical_Switch'
+        super(LogicalSwitchChassisEvent, self).__init__(
+            events, table, None)
+        self.event_name = self.__class__.__name__
+
+
 class LSPChassisEvent(Event):
     def __init__(self, bgp_agent, events):
         self.agent = bgp_agent

ovn_bgp_agent/drivers/openstack/watchers/nb_bgp_watcher.py

@@ -311,6 +311,43 @@ class LogicalSwitchPortFIPDeleteEvent(base_watcher.LSPChassisEvent):
             self.agent.withdraw_fip(fip, row)
 
 
+class LogicalSwitchUpdateEvent(base_watcher.LogicalSwitchChassisEvent):
+    '''Event to trigger on logical switch vrf config updates'''
+    def __init__(self, bgp_agent):
+        events = (self.ROW_UPDATE, self.ROW_DELETE)
+        super(LogicalSwitchUpdateEvent, self).__init__(
+            bgp_agent, events)
+
+    def match_fn(self, event, row, old):
+        '''Match updates for vrf configuration
+
+        Will trigger whenever external_ids[neutron_bgpvpn:vni] and
+        external_ids[neutron_bgpvpn:type] have been set and either one has
+        been updated
+        '''
+
+        settings = driver_utils.get_port_vrf_settings(row)
+        if settings and event == self.ROW_DELETE:
+            # Always run sync method if we are deleting this network (and it
+            # had settings applied)
+            return True
+
+        old_settings = driver_utils.get_port_vrf_settings(old)
+        if old_settings is None:
+            # it was not provided in old, so do not process this update
+            return False
+
+        return settings != old_settings
+
+    def _run(self, event, row, old):
+        with _SYNC_STATE_LOCK.read_lock():
+            # NOTE(mnederlof): For now it makes sense to run the sync method
+            # as this is triggered with a configured interval anyway and it
+            # will add/remove the triggered logical switch.
+            # It might make sense in the future to optimize this behaviour.
+            self.agent.sync()
+
+
 class LocalnetCreateDeleteEvent(base_watcher.LSPChassisEvent):
     def __init__(self, bgp_agent):
         events = (self.ROW_CREATE, self.ROW_DELETE,)

ovn_bgp_agent/exceptions.py

@@ -33,6 +33,24 @@ class OVNBGPAgentException(Exception):
         return self.msg
 
 
+class ConfOptionRequired(OVNBGPAgentException):
+    """Configuration option is required for this driver to function properly
+
+    :param option: The configuration option required
+    """
+
+    message = _("Required driver conf option missing: CONF.%(option)s")
+
+
+class UnsupportedWiringConfig(OVNBGPAgentException):
+    """Wiring config called with unknown/unsupported exposing method
+
+    :param method: The (unsupported) exposing method
+    """
+
+    message = _("No wiring support for exposing_method %(method)s.")
+
+
 class InvalidPortIP(OVNBGPAgentException):
     """OVN Port has Invalid IP.
 

ovn_bgp_agent/privileged/linux_net.py

@@ -518,6 +518,15 @@ def _run_iproute_neigh(command, device, **kwargs):
                   command, device)
 
 
+def _run_iproute_brport(command, ifname, **kwargs):
+    try:
+        with iproute.IPRoute() as ip:
+            idx = _get_link_id(ifname)
+            return ip.brport(command, index=idx, **kwargs)
+    except netlink_exceptions.NetlinkError as e:
+        _translate_ip_device_exception(e, ifname)
+
+
 @ovn_bgp_agent.privileged.default.entrypoint
 def create_interface(ifname, kind, **kwargs):
     ifname = ifname[:15]
@@ -545,12 +554,18 @@ def set_link_attribute(ifname, **kwargs):
 
 
 @ovn_bgp_agent.privileged.default.entrypoint
-def add_ip_address(ip_address, ifname):
+def set_brport_attribute(ifname, **kwargs):
+    _run_iproute_brport("set", ifname, **kwargs)
+
+
+@ovn_bgp_agent.privileged.default.entrypoint
+def add_ip_address(ip_address, ifname, prefixlen=None, **kwargs):
     ifname = ifname[:15]
     net = netaddr.IPNetwork(ip_address)
     ip_version = l_net.get_ip_version(ip_address)
     address = str(net.ip)
-    prefixlen = 32 if ip_version == 4 else 128
+    if not prefixlen:
+        prefixlen = 32 if ip_version == 4 else 128
     family = common_utils.IP_VERSION_FAMILY_MAP[ip_version]
     _run_iproute_addr('add',
                       ifname,
@@ -560,18 +575,20 @@ def add_ip_address(ip_address, ifname):
 
 
 @ovn_bgp_agent.privileged.default.entrypoint
-def delete_ip_address(ip_address, ifname):
+def delete_ip_address(ip_address, ifname, prefixlen=None, **kwargs):
     ifname = ifname[:15]
     net = netaddr.IPNetwork(ip_address)
     ip_version = l_net.get_ip_version(ip_address)
     address = str(net.ip)
-    prefixlen = 32 if ip_version == 4 else 128
+    if not prefixlen:
+        prefixlen = 32 if ip_version == 4 else 128
     family = common_utils.IP_VERSION_FAMILY_MAP[ip_version]
     _run_iproute_addr("delete",
                       ifname,
                       address=address,
                       mask=prefixlen,
-                      family=family)
+                      family=family,
+                      **kwargs)
 
 
 @ovn_bgp_agent.privileged.default.entrypoint

ovn_bgp_agent/tests/unit/drivers/openstack/test_nb_ovn_bgp_driver.py

@@ -833,7 +833,32 @@ class TestNBOVNBGPDriver(test_base.TestCase):
         }
         self.nb_bgp_driver.expose_remote_ip(ips, ips_info)
 
-        m_announce_ips.assert_called_once_with(ips)
+        m_announce_ips.assert_called_once_with(ips, ips_info=ips_info)
+
+    @mock.patch.object(bgp_utils, 'announce_ips')
+    def test_expose_remote_ip_vrf(self, m_announce_ips):
+        CONF.set_override('exposing_method', constants.EXPOSE_METHOD_VRF)
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        mock__get_router_port_info_for_ls = mock.patch.object(
+            self.nb_bgp_driver, '_get_router_port_info_for_ls'
+        ).start()
+
+        mock__get_router_port_info_for_ls.return_value = {
+            'bridge_device': self.bridge, 'bridge_vlan': None,
+            'via': self.router1_info['ips'],
+        }
+
+        ips = [self.ipv4, self.ipv6]
+        ips_info = {
+            'mac': 'fake-mac',
+            'cidrs': [],
+            'type': constants.OVN_VM_VIF_PORT_TYPE,
+            'logical_switch': 'test-ls'
+        }
+        self.nb_bgp_driver.expose_remote_ip(ips, ips_info)
+
+        m_announce_ips.assert_called_once_with(ips, ips_info=ips_info)
 
     @mock.patch.object(driver_utils, 'is_ipv6_gua')
     @mock.patch.object(bgp_utils, 'announce_ips')
@@ -852,7 +877,32 @@ class TestNBOVNBGPDriver(test_base.TestCase):
         m_gua.side_effect = [False, True]
         self.nb_bgp_driver.expose_remote_ip(ips, ips_info)
 
-        m_announce_ips.assert_called_once_with([self.ipv6])
+        m_announce_ips.assert_called_once_with([self.ipv6], ips_info=ips_info)
+
+    def test__get_exposed_ip(self):
+        self.nb_bgp_driver._exposed_ips = {
+            'provider-ls': {'vip': {'bridge_device': self.bridge,
+                                    'bridge_vlan': None}}}
+
+        ls, info = self.nb_bgp_driver._get_exposed_ip('vip')
+        self.assertEqual('provider-ls', ls)
+        self.assertDictEqual({'bridge_device': self.bridge,
+                              'bridge_vlan': None}, info)
+
+    def test__get_router_port_info_for_ls(self):
+        ls = 'provider-ls'
+        self.nb_bgp_driver._exposed_ips = {
+            ls: {'vip': {'bridge_device': self.bridge, 'bridge_vlan': None}}
+        }
+
+        tenant_ls = 'tenant_ls'
+        self.nb_bgp_driver.ovn_local_lrps[tenant_ls] = {'vip'}
+
+        info = self.nb_bgp_driver._get_router_port_info_for_ls(tenant_ls)
+        self.assertDictEqual({'bridge_device': self.bridge,
+                              'bridge_vlan': None}, info)
+        self.assertIsNone(self.nb_bgp_driver._get_router_port_info_for_ls(
+            'other_ls'))
 
     @mock.patch.object(bgp_utils, 'withdraw_ips')
     def test_withdraw_remote_ip(self, m_withdraw_ips):
@@ -865,7 +915,32 @@ class TestNBOVNBGPDriver(test_base.TestCase):
         }
         self.nb_bgp_driver.withdraw_remote_ip(ips, ips_info)
 
-        m_withdraw_ips.assert_called_once_with(ips)
+        m_withdraw_ips.assert_called_once_with(ips, ips_info=ips_info)
+
+    @mock.patch.object(bgp_utils, 'withdraw_ips')
+    def test_withdraw_remote_ip_vrf(self, m_withdraw_ips):
+        CONF.set_override('exposing_method', constants.EXPOSE_METHOD_VRF)
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        mock__get_router_port_info_for_ls = mock.patch.object(
+            self.nb_bgp_driver, '_get_router_port_info_for_ls'
+        ).start()
+
+        mock__get_router_port_info_for_ls.return_value = {
+            'bridge_device': self.bridge, 'bridge_vlan': None,
+            'via': self.router1_info['ips'],
+        }
+
+        ips = [self.ipv4, self.ipv6]
+        ips_info = {
+            'mac': 'fake-mac',
+            'cidrs': [],
+            'type': constants.OVN_VM_VIF_PORT_TYPE,
+            'logical_switch': 'test-ls'
+        }
+        self.nb_bgp_driver.withdraw_remote_ip(ips, ips_info)
+
+        m_withdraw_ips.assert_called_once_with(ips, ips_info=ips_info)
 
     @mock.patch.object(driver_utils, 'is_ipv6_gua')
     @mock.patch.object(bgp_utils, 'withdraw_ips')
@@ -884,7 +959,7 @@ class TestNBOVNBGPDriver(test_base.TestCase):
         m_gua.side_effect = [False, True]
         self.nb_bgp_driver.withdraw_remote_ip(ips, ips_info)
 
-        m_withdraw_ips.assert_called_once_with([self.ipv6])
+        m_withdraw_ips.assert_called_once_with([self.ipv6], ips_info=ips_info)
 
     def test_expose_subnet(self):
         ips = ['10.0.0.1/24']

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_bgp.py

@@ -0,0 +1,122 @@
+# Copyright 2024 team.blue/nl
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from oslo_config import cfg
+from unittest import mock
+
+from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import bgp as bgp_utils
+from ovn_bgp_agent.tests import base as test_base
+
+
+CONF = cfg.CONF
+
+
+class TestEVPN(test_base.TestCase):
+
+    def setUp(self):
+        super(TestEVPN, self).setUp()
+
+        self.mock_frr = mock.patch.object(bgp_utils, 'frr').start()
+        self.mock_linux_net = mock.patch.object(bgp_utils, 'linux_net').start()
+
+    def _set_exposing_method(self, exposing_method):
+        CONF.set_override('exposing_method', exposing_method)
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+    def _test_announce_ips(self, exposing_method):
+        ips = ['10.10.10.1', '10.20.10.1']
+        self._set_exposing_method(exposing_method)
+
+        bgp_utils.announce_ips(list(ips))
+
+        if exposing_method in [constants.EXPOSE_METHOD_VRF]:
+            self.mock_linux_net.add_ips_to_dev.assert_not_called()
+        else:
+            self.mock_linux_net.add_ips_to_dev.assert_called_once_with(
+                CONF.bgp_nic, ips)
+
+    def test_announce_ips_underlay(self):
+        self._test_announce_ips('underlay')
+
+    def test_announce_ips_dynamic(self):
+        self._test_announce_ips('dynamic')
+
+    def test_announce_ips_ovn(self):
+        self._test_announce_ips('ovn')
+
+    def test_announce_ips_vrf(self):
+        self._test_announce_ips('vrf')
+
+    def test_announce_ips_l2vni(self):
+        self._test_announce_ips('l2vni')
+
+    def _test_withdraw_ips(self, exposing_method):
+        ips = ['10.10.10.1', '10.20.10.1']
+        self._set_exposing_method(exposing_method)
+
+        bgp_utils.withdraw_ips(list(ips))
+
+        if exposing_method in [constants.EXPOSE_METHOD_VRF]:
+            self.mock_linux_net.del_ips_from_dev.assert_not_called()
+        else:
+            self.mock_linux_net.del_ips_from_dev.assert_called_once_with(
+                CONF.bgp_nic, ips)
+
+    def test_withdraw_ips_underlay(self):
+        self._test_withdraw_ips('underlay')
+
+    def test_withdraw_ips_dynamic(self):
+        self._test_withdraw_ips('dynamic')
+
+    def test_withdraw_ips_ovn(self):
+        self._test_withdraw_ips('ovn')
+
+    def test_withdraw_ips_vrf(self):
+        self._test_withdraw_ips('vrf')
+
+    def test_withdraw_ips_l2vni(self):
+        self._test_withdraw_ips('l2vni')
+
+    def _test_ensure_base_bgp_configuration(self, exposing_method):
+        self._set_exposing_method(exposing_method)
+
+        bgp_utils.ensure_base_bgp_configuration()
+
+        if exposing_method not in [constants.EXPOSE_METHOD_UNDERLAY,
+                                   constants.EXPOSE_METHOD_DYNAMIC,
+                                   constants.EXPOSE_METHOD_OVN]:
+            self.mock_frr.vrf_leak.assert_not_called()
+            self.mock_linux_net.ensure_vrf.assert_not_called()
+            self.mock_linux_net.ensure_ovn_device.assert_not_called()
+        else:
+            self.mock_frr.vrf_leak.assert_called_once()
+            self.mock_linux_net.ensure_vrf.assert_called_once()
+            self.mock_linux_net.ensure_ovn_device.assert_called_once()
+
+    def test_ensure_base_bgp_configuration_underlay(self):
+        self._test_ensure_base_bgp_configuration('underlay')
+
+    def test_ensure_base_bgp_configuration_dynamic(self):
+        self._test_ensure_base_bgp_configuration('dynamic')
+
+    def test_ensure_base_bgp_configuration_ovn(self):
+        self._test_ensure_base_bgp_configuration('ovn')
+
+    def test_ensure_base_bgp_configuration_vrf(self):
+        self._test_ensure_base_bgp_configuration('vrf')
+
+    def test_ensure_base_bgp_configuration_l2vni(self):
+        self._test_ensure_base_bgp_configuration('l2vni')

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_driver_utils.py

@@ -121,6 +121,34 @@ class TestDriverUtils(test_base.TestCase):
         lb = utils.create_row(name='lb-someuuid')
         self.assertFalse(driver_utils.is_pf_lb(lb))
 
+    def test_ips_per_version(self):
+        ips = ['10.0.0.1/32', 'fe80::1/128']
+        self.assertDictEqual(driver_utils.ips_per_version(ips), {
+            constants.IP_VERSION_4: '10.0.0.1',
+            constants.IP_VERSION_6: 'fe80::1',
+        })
+
+        # More than 1 ip per version and without prefix
+        ips = ['10.0.0.1/32', '10.0.0.254', 'fe80::1', 'fc00::1/128']
+        self.assertDictEqual(driver_utils.ips_per_version(ips), {
+            constants.IP_VERSION_4: '10.0.0.254',
+            constants.IP_VERSION_6: 'fc00::1',
+        })
+
+        # Only IPv4
+        ips = ['10.0.0.1/32']
+        self.assertDictEqual(driver_utils.ips_per_version(ips), {
+            constants.IP_VERSION_4: '10.0.0.1',
+            constants.IP_VERSION_6: None,
+        })
+
+        # Only IPv6
+        ips = ['fc00::1/128']
+        self.assertDictEqual(driver_utils.ips_per_version(ips), {
+            constants.IP_VERSION_4: None,
+            constants.IP_VERSION_6: 'fc00::1',
+        })
+
     def test_get_prefixes_from_ips(self):
         # IPv4
         ips = ['192.168.0.1/24', '192.168.0.244/28', '172.13.37.59/27']
@@ -137,3 +165,37 @@ class TestDriverUtils(test_base.TestCase):
         ips = ['172.13.37.59/27', 'ff00::13:37/112']
         self.assertListEqual(driver_utils.get_prefixes_from_ips(ips),
                              ['172.13.37.32/27', 'ff00::13:0/112'])
+
+    def test_get_port_vlan_untagged(self):
+        port = utils.create_row(tag=[])
+        self.assertEqual('0', driver_utils.get_port_vlan(port))
+
+    def test_get_port_vlan_tagged(self):
+        port = utils.create_row(tag=[100])
+        self.assertEqual('100', driver_utils.get_port_vlan(port))
+
+        port = utils.create_row(tag=[100, 123])
+        self.assertEqual('100', driver_utils.get_port_vlan(port))
+
+    def test_get_port_vrf_settings(self):
+        row = utils.create_row(external_ids={
+            constants.OVN_EVPN_TYPE_EXT_ID_KEY: constants.OVN_EVPN_TYPE_L3,
+            constants.OVN_EVPN_VNI_EXT_ID_KEY: 1001,
+        })
+        self.assertEqual('l3::1001', driver_utils.get_port_vrf_settings(row))
+
+    def test_get_port_vrf_settings_no_vni(self):
+        row = utils.create_row(external_ids={
+            constants.OVN_EVPN_TYPE_EXT_ID_KEY: constants.OVN_EVPN_TYPE_L3,
+        })
+        self.assertEqual('', driver_utils.get_port_vrf_settings(row))
+
+    def test_get_port_vrf_settings_no_type(self):
+        row = utils.create_row(external_ids={
+            constants.OVN_EVPN_VNI_EXT_ID_KEY: 1001,
+        })
+        self.assertEqual('', driver_utils.get_port_vrf_settings(row))
+
+    def test_get_port_vrf_settings_not_provided(self):
+        row = utils.create_row()
+        self.assertIsNone(driver_utils.get_port_vrf_settings(row))

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_evpn.py

@@ -0,0 +1,670 @@
+# Copyright 2024 team.blue/nl
+# All Rights Reserved.
+#
+#    Licensed under the Apache License, Version 2.0 (the "License"); you may
+#    not use this file except in compliance with the License. You may obtain
+#    a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+#    Unless required by applicable law or agreed to in writing, software
+#    distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
+#    WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
+#    License for the specific language governing permissions and limitations
+#    under the License.
+
+from oslo_config import cfg
+from unittest import mock
+
+from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import evpn
+from ovn_bgp_agent import exceptions
+from ovn_bgp_agent.tests import base as test_base
+from ovn_bgp_agent.tests import utils
+
+
+CONF = cfg.CONF
+
+
+class TestEVPN(test_base.TestCase):
+
+    def setUp(self):
+        super(TestEVPN, self).setUp()
+        CONF.set_override('evpn_local_ip', '127.0.0.1')
+
+        self.mock_ovs = mock.patch.object(evpn, 'ovs').start()
+        self.mock_ovs.get_ovs_patch_port_ofport.return_value = 12
+
+        self.mock_frr = mock.patch.object(evpn, 'frr').start()
+        self.mock_linux_net = mock.patch.object(evpn, 'linux_net').start()
+
+        self.fake_mac = fake_mac = 'fe:12:34:56:89:90'
+        self.mock_linux_net.get_interface_address.return_value = fake_mac
+
+        self._bridge_args = {
+            'ovs_bridge': 'br-ex',
+            'vni': 100,
+            'evpn_opts': {},
+            'mode': constants.OVN_EVPN_TYPE_L3,
+            'ovs_flows': {'br-ex': {}},
+        }
+        self.vrf_name = 'vrf-100'
+        self.vxlan_name = 'vxlan-100'
+        self.bridge_name = 'br-100'
+
+        self.veth_vrf = '_to_be_set_by__create_bridge_and_vlan'
+        self.veth_ovs = '_to_be_set_by__create_bridge_and_vlan'
+
+        # evpn.local_bridges = {}
+    def _reset_evpn_local_bridges(self):
+        evpn.local_bridges = {}
+
+    def _create_bridge(self, **override_args) -> evpn.EvpnBridge:
+        self.addCleanup(self._reset_evpn_local_bridges)
+        args = dict(**self._bridge_args)
+        args.update(**override_args)
+        return evpn.setup(**args)
+
+    def _create_bridge_and_vlan(
+        self, vlan_tag=4094, **bridge_args
+    ) -> 'tuple[object, evpn.EvpnBridge, evpn.VlanDev]':
+        port = utils.create_row(
+            name='fake-port-name',
+            tag=[vlan_tag]
+        )
+
+        uuid = str(port.uuid)[0:11]
+        self.veth_vrf = constants.OVN_EVPN_VETH_VRF_UUID_PREFIX + uuid
+        self.veth_ovs = constants.OVN_EVPN_VETH_OVS_UUID_PREFIX + uuid
+
+        evpn_bridge = self._create_bridge(**bridge_args)
+        return port, evpn_bridge, evpn_bridge.connect_vlan(port)
+
+    def test_setup_no_ip(self):
+        CONF.set_override('evpn_local_ip', None)
+        self.assertRaises(exceptions.ConfOptionRequired, self._create_bridge)
+
+    def test_setup(self):
+        evpn_bridge = self._create_bridge()
+
+        self.assertIsInstance(evpn_bridge, evpn.EvpnBridge)
+        self.assertEqual(evpn_bridge.ovs_bridge, 'br-ex')
+        self.assertEqual(evpn_bridge.vni, 100)
+        self.assertEqual(evpn_bridge.vrf_name, self.vrf_name)
+        self.assertEqual(evpn_bridge.bridge_name, self.bridge_name)
+        self.assertEqual(evpn_bridge.vxlan_name, self.vxlan_name)
+
+        other_bridge = self._create_bridge()
+        self.assertEqual(evpn_bridge, other_bridge)
+
+    def test_lookup_vlan_int(self):
+        port = utils.create_row(
+            tag=[4094]
+        )
+        evpn_bridge = self._create_bridge()
+        evpn_bridge.connect_vlan(port)
+
+        bridge = evpn.lookup(self._bridge_args['ovs_bridge'], 4094)
+        self.assertIsInstance(bridge, evpn.EvpnBridge)
+
+    def test_lookup_vlan_str(self):
+        port = utils.create_row(
+            tag=[4094]
+        )
+        evpn_bridge = self._create_bridge()
+        evpn_bridge.connect_vlan(port)
+
+        bridge = evpn.lookup(self._bridge_args['ovs_bridge'], '4094')
+        self.assertIsInstance(bridge, evpn.EvpnBridge)
+
+    def test_lookup_vlan_None(self):
+        port = utils.create_row(
+            tag=[]
+        )
+        evpn_bridge = self._create_bridge()
+        evpn_bridge.connect_vlan(port)
+
+        bridge = evpn.lookup(self._bridge_args['ovs_bridge'], None)
+        self.assertIsInstance(bridge, evpn.EvpnBridge)
+
+    def test_lookup_vlan_unknown(self):
+        self._create_bridge_and_vlan(vlan_tag=123)
+        self._create_bridge_and_vlan(vlan_tag=123, vni=123, ovs_bridge='foo')
+        self._create_bridge_and_vlan(vlan_tag=4094, vni=123, ovs_bridge='foo')
+        self.assertRaises(KeyError, evpn.lookup,
+                          self._bridge_args['ovs_bridge'], '4094')
+
+    def test_evpnbridge_setup_l3(self):
+        bridge = self._create_bridge()
+        bridge.setup()
+
+        self.assertTrue(bridge._setup_done)
+
+        # Create pointer for shorter lines.
+        linux_net = self.mock_linux_net
+
+        linux_net.ensure_bridge.assert_called_once_with(self.bridge_name)
+        linux_net.ensure_vxlan.assert_called_once_with(self.vxlan_name, 100,
+                                                       '127.0.0.1', 4789)
+        linux_net.set_master_for_device.assert_has_calls([
+            mock.call(self.vxlan_name, self.bridge_name),
+            mock.call(self.bridge_name, self.vrf_name),
+        ])
+        linux_net.disable_learning_vxlan_intf.assert_called_once_with(
+            self.vxlan_name)
+        linux_net.ensure_vrf.assert_called_once_with(self.vrf_name, 100)
+
+        frr = self.mock_frr
+        frr.vrf_reconfigure.assert_called_once_with(mock.ANY, 'add-vrf')
+
+    def test_evpnbridge_setup_l2(self):
+        bridge = self._create_bridge(mode=constants.OVN_EVPN_TYPE_L2)
+        bridge.setup()
+
+        self.assertTrue(bridge._setup_done)
+
+        # Create pointer for shorter lines.
+        linux_net = self.mock_linux_net
+
+        linux_net.ensure_bridge.assert_called_once_with(self.bridge_name)
+        linux_net.ensure_vxlan.assert_called_once_with(self.vxlan_name, 100,
+                                                       '127.0.0.1', 4789)
+        linux_net.set_master_for_device.assert_has_calls([
+            mock.call(self.vxlan_name, self.bridge_name),
+        ])
+        linux_net.disable_learning_vxlan_intf.assert_called_once_with(
+            self.vxlan_name)
+        linux_net.ensure_vrf.assert_not_called()
+
+        frr = self.mock_frr
+        frr.vrf_reconfigure.assert_called_once_with(mock.ANY, 'add-vrf')
+
+    def test_evpnbridge_setup_done(self):
+        bridge = self._create_bridge()
+        bridge._setup_done = True
+        bridge.setup()
+
+        self.mock_linux_net.ensure_bridge.assert_not_called()
+
+    def test_evpnbridge_eval_disconnect(self):
+        _, bridge, evpn_vlan = self._create_bridge_and_vlan()
+
+        bridge_disconnect = mock.patch.object(bridge, 'disconnect').start()
+
+        bridge._eval_disconnect()
+        bridge_disconnect.assert_not_called()
+
+        bridge._setup_done = True
+        evpn_vlan._setup_done = True
+        bridge._eval_disconnect()
+
+        bridge_disconnect.assert_not_called()
+
+        evpn_vlan._setup_done = False
+
+        bridge._eval_disconnect()
+        bridge_disconnect.assert_called_once()
+
+    def test_evpnbridge_disconnect(self):
+        bridge = self._create_bridge()
+        bridge._setup_done = True
+        bridge.disconnect()
+
+        calls = [mock.call('br-100'),
+                 mock.call('vxlan-100'),
+                 mock.call('vrf-100')]
+        self.mock_linux_net.delete_device.assert_has_calls(calls)
+        self.mock_frr.vrf_reconfigure.assert_called_once_with(mock.ANY,
+                                                              action='del-vrf')
+
+        self.assertFalse(bridge._setup_done)
+
+    def test_evpnbridge_connect_vlan_again(self):
+        port, bridge, evpn_vlan = self._create_bridge_and_vlan()
+
+        vlan = bridge.connect_vlan(port)
+        self.assertEqual(vlan, evpn_vlan)
+
+    def test_evpnbridge_get_vlan(self):
+        _, bridge, evpn_vlan = self._create_bridge_and_vlan()
+        self.assertEqual(bridge.get_vlan(4094), evpn_vlan)
+
+    def test_evpnbridge_vlan_lladdr_property_calls_setup(self):
+        _, bridge, evpn_vlan = self._create_bridge_and_vlan()
+
+        evpn_vlan_setup = mock.patch.object(evpn_vlan, 'setup').start()
+        self.assertEqual(evpn_vlan.lladdr, self.fake_mac)
+        evpn_vlan_setup.assert_called_once()
+
+    def test_evpnbridge_vlan_setup_l2(self):
+        vlan_tag = 4094
+        vlan_tag_str = '4094'
+        _, evpn_bridge, vlan_dev = self._create_bridge_and_vlan(vlan_tag,
+                                                                mode='l2')
+
+        evpn_setup = mock.patch.object(evpn_bridge, 'setup').start()
+
+        vlan_dev.setup()
+
+        evpn_setup.assert_called_once()
+
+        linux_net = self.mock_linux_net
+        linux_net.ensure_veth.assert_called_once_with(self.veth_vrf,
+                                                      self.veth_ovs)
+        self.mock_ovs.add_device_to_ovs_bridge(self.veth_ovs, 'br-ex',
+                                               vlan_tag=vlan_tag_str)
+        linux_net.set_master_for_device.assert_called_once_with(self.veth_vrf,
+                                                                'br-100')
+
+        linux_net.ensure_arp_ndp_enabled_for_bridge.assert_not_called()
+        linux_net.enable_routing_for_interfaces.assert_not_called()
+        linux_net.enable_proxy_arp.assert_not_called()
+        linux_net.enable_proxy_ndp.assert_not_called()
+
+        self.mock_ovs.ensure_mac_tweak_flows.assert_not_called()
+        self.mock_ovs.remove_extra_ovs_flows.assert_not_called()
+
+        self.assertTrue(vlan_dev._veth_created)
+        self.assertTrue(vlan_dev._setup_done)
+
+    def test_evpnbridge_vlan_setup_l3(self, custom_ips=[]):
+        vlan_tag = 4094
+        vlan_tag_str = '4094'
+        _, evpn_bridge, vlan_dev = self._create_bridge_and_vlan(vlan_tag)
+
+        evpn_setup = mock.patch.object(evpn_bridge, 'setup').start()
+
+        if custom_ips:
+            vlan_dev.add_ips(list(custom_ips))
+
+        vlan_dev.setup()
+
+        evpn_setup.assert_called_once()
+
+        linux_net = self.mock_linux_net
+        linux_net.ensure_veth.assert_called_once_with(self.veth_vrf,
+                                                      self.veth_ovs)
+        self.mock_ovs.add_device_to_ovs_bridge(self.veth_ovs, 'br-ex',
+                                               vlan_tag=vlan_tag_str)
+        linux_net.set_master_for_device.assert_called_once_with(self.veth_vrf,
+                                                                self.vrf_name)
+        linux_net.ensure_arp_ndp_enabled_for_bridge.assert_called_once_with(
+            self.veth_vrf, offset=vlan_tag, vlan_tag=vlan_tag_str)
+
+        linux_net.enable_routing_for_interfaces.assert_called_once_with(
+            self.veth_vrf, 'br-100')
+
+        linux_net.enable_proxy_arp.assert_called_once_with(self.veth_vrf)
+        linux_net.enable_proxy_ndp.assert_called_once_with(self.veth_vrf)
+
+        self.mock_ovs.ensure_mac_tweak_flows.assert_called_once_with(
+            'br-ex', self.fake_mac, [12], constants.OVS_RULE_COOKIE)
+        self.mock_ovs.remove_extra_ovs_flows.assert_called_once_with(
+            mock.ANY, 'br-ex', constants.OVS_RULE_COOKIE)
+
+        self.assertTrue(vlan_dev._veth_created)
+        self.assertTrue(vlan_dev._setup_done)
+
+        if custom_ips:
+            linux_net.add_ips_to_dev.assert_has_calls([
+                mock.call(self.veth_vrf, ips=custom_ips),
+            ])
+
+        linux_net.ensure_anycast_mac_for_interface.assert_called_once_with(
+            self.veth_vrf, offset=6557694)
+
+    def test_evpnbridge_vlan_setup_l3_with_custom_ips(self):
+        self.test_evpnbridge_vlan_setup_l3(custom_ips=['10.10.10.1'])
+
+    def test_evpnbridge_vlan_setup_l3_failed_ovs_call(self):
+        vlan_tag = 4094
+        _, evpn_bridge, vlan_dev = self._create_bridge_and_vlan(vlan_tag)
+
+        mock.patch.object(evpn_bridge, 'setup').start()
+
+        self.mock_ovs.get_ovs_patch_port_ofport.side_effect = KeyError
+
+        vlan_dev.setup()
+        self.assertTrue(vlan_dev._veth_created)
+        self.assertFalse(vlan_dev._setup_done)
+
+    def test_evpnbridge_vlan__eval_disconnect(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+
+        vlan_dev_disconnect = mock.patch.object(vlan_dev, 'disconnect').start()
+
+        vlan_dev._setup_done = True
+
+        vlan_dev._agent_routing_tables_routes = ['entry']
+        vlan_dev._eval_disconnect()
+        vlan_dev_disconnect.assert_not_called()
+
+        vlan_dev._agent_routing_tables_routes = []
+        vlan_dev._eval_disconnect()
+        vlan_dev_disconnect.assert_called_once()
+
+    def test_evpnbridge_vlan__eval_disconnect_not_setup_yet(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+
+        vlan_dev_disconnect = mock.patch.object(vlan_dev, 'disconnect').start()
+
+        vlan_dev._agent_routing_tables_routes = []
+        vlan_dev._eval_disconnect()
+
+        vlan_dev_disconnect.assert_not_called()
+
+    def test_evpnbridge_vlan_disconnect(self):
+        _, evpn_bridge, vlan_dev = self._create_bridge_and_vlan()
+
+        evpn_bridge__eval_disconnect = mock.patch.object(
+            evpn_bridge, '_eval_disconnect').start()
+
+        vlan_dev.disconnect()
+
+        evpn_bridge__eval_disconnect.assert_called_once()
+
+        self.mock_ovs.del_device_from_ovs_bridge.assert_called_once_with(
+            self.veth_ovs, 'br-ex')
+        self.mock_linux_net.delete_device.assert_called_once_with(
+            self.veth_vrf)
+
+        self.assertFalse(vlan_dev._veth_created)
+        self.assertFalse(vlan_dev._setup_done)
+
+    def test_evpnbridge_vlan_teardown(self):
+        _, evpn_bridge, vlan_dev = self._create_bridge_and_vlan()
+
+        vlan_dev_disconnect = mock.patch.object(
+            vlan_dev, 'disconnect').start()
+
+        vlan_dev.teardown()
+
+        vlan_dev_disconnect.assert_called_once()
+        self.assertNotIn(vlan_dev.vlan_tag, evpn_bridge.vlans)
+
+    def test_evpnbridge_vlan_process_dhcp_opts(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        dhcp_opts = [
+            utils.create_row(cidr='10.10.10.0/24',
+                             options={'router': '10.10.10.1'}),
+            utils.create_row(cidr='fe00::/64', options={}),
+        ]
+        vlan_dev._setup_done = True
+        vlan_dev.process_dhcp_opts(dhcp_opts)
+
+        ips = {'10.10.10.1'}
+        self.assertSetEqual(vlan_dev._custom_ips, ips)
+        self.mock_linux_net.add_ips_to_dev.assert_called_once_with(
+            self.veth_vrf, ips=list(ips))
+        self.mock_frr.nd_reconfigure.assert_called_once_with(
+            self.veth_vrf, 'fe00::/64', {})
+
+    def test_evpnbridge_vlan_process_dhcp_opts_multiple_subnets(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        dhcp_opts = [
+            utils.create_row(cidr='10.10.10.0/24',
+                             options={'router': '10.10.10.1'}),
+            utils.create_row(cidr='10.10.20.0/24',
+                             options={'router': '10.10.20.1'}),
+        ]
+        vlan_dev._setup_done = True
+        vlan_dev.process_dhcp_opts(dhcp_opts)
+
+        ips = {'10.10.10.1', '10.10.20.1'}
+        self.assertSetEqual(vlan_dev._custom_ips, ips)
+
+        calls = [
+            mock.call(self.veth_vrf, ips=['10.10.10.1']),
+            mock.call(self.veth_vrf, ips=['10.10.20.1']),
+        ]
+        self.mock_linux_net.add_ips_to_dev.assert_has_calls(calls)
+
+    def test_evpnbridge_vlan_add_route(self, ip='10.10.10.10'):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev._setup_done = True
+
+        routing_tables_routes = {self.veth_vrf: []}
+        addr = ip.split('/')[0]
+        mask = None if '/' not in ip else ip.split('/')[1]
+        mac = 'fe:12:34:56:89:12'
+        via = None
+
+        vlan_dev.add_route(routing_tables_routes, ip, mac, via)
+
+        self.assertDictEqual(routing_tables_routes, {self.veth_vrf: [{
+            'ip': '10.10.10.10',
+            'mask': mask,
+            'mac': 'fe:12:34:56:89:12',
+            'via': None,
+        }]})
+
+        self.mock_linux_net.add_ip_route.assert_called_once_with(
+            mock.ANY, addr, 100, self.veth_vrf, mask=mask, via=None)
+
+        self.mock_linux_net.add_ip_nei.assert_called_once_with(
+            addr, 'fe:12:34:56:89:12', self.veth_vrf)
+
+    def test_evpnbridge_vlan_add_route_with_prefix(self):
+        self.test_evpnbridge_vlan_add_route(ip='10.10.10.10/32')
+
+    def test_evpnbridge_vlan_add_route_l2(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan(mode='l2')
+        vlan_dev._setup_done = True
+
+        routing_tables_routes = {self.veth_vrf: []}
+        ip = '10.10.10.10/32'
+        mac = 'fe:12:34:56:89:12'
+        via = None
+
+        vlan_dev.add_route(routing_tables_routes, ip, mac, via)
+
+        self.mock_linux_net.add_ip_route.assert_not_called()
+
+    def test_evpnbridge_vlan_del_route(self, ip='10.10.10.10'):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev._setup_done = True
+
+        addr = ip.split('/')[0]
+        routing_tables_routes = {self.veth_vrf: [{
+            'ip': addr,
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }, {
+            'ip': '10.10.10.11',
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }]}
+        mac = 'fe:12:34:56:89:12'
+        vlan_dev__eval_disconnect = mock.patch.object(
+            vlan_dev, '_eval_disconnect').start()
+
+        vlan_dev.del_route(routing_tables_routes, ip, mac)
+
+        self.assertDictEqual(routing_tables_routes, {self.veth_vrf: [{
+            'ip': '10.10.10.11',
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }]})
+
+        self.mock_linux_net.del_ip_route.assert_called_once_with(
+            mock.ANY, addr, 100, self.veth_vrf, mask='32', via='10.10.20.10')
+
+        self.mock_linux_net.del_ip_nei.assert_called_once_with(
+            addr, 'fe:12:34:56:89:12', self.veth_vrf)
+
+        vlan_dev__eval_disconnect.assert_called()
+
+    def test_evpnbridge_vlan_del_route_with_prefix(self):
+        self.test_evpnbridge_vlan_del_route('10.10.10.10/32')
+
+    def test_evpnbridge_vlan_del_route_no_route_table(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev._setup_done = True
+
+        addr = '10.10.10.10'
+        routing_tables_routes = {
+            self.veth_vrf: [{
+                'ip': '10.10.10.11',
+                'mask': '32',
+                'mac': 'fe:12:34:56:89:12',
+                'via': '10.10.20.10',
+            }]
+        }
+        mac = 'fe:12:34:56:89:12'
+        vlan_dev__eval_disconnect = mock.patch.object(
+            vlan_dev, '_eval_disconnect').start()
+
+        vlan_dev.del_route(routing_tables_routes, addr, mac)
+
+        self.assertDictEqual(routing_tables_routes, {self.veth_vrf: [{
+            'ip': '10.10.10.11',
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }]})
+
+        self.mock_linux_net.del_ip_route.assert_called_once_with(
+            mock.ANY, addr, 100, self.veth_vrf, mask=None, via=None)
+
+        self.mock_linux_net.del_ip_nei.assert_called_once_with(
+            addr, 'fe:12:34:56:89:12', self.veth_vrf)
+
+        vlan_dev__eval_disconnect.assert_called()
+
+    def test_evpnbridge_vlan_del_route_l2(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan(mode='l2')
+        vlan_dev._setup_done = True
+
+        routing_tables_routes = {self.veth_vrf: [{
+            'ip': '10.10.10.10',
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }]}
+        ip = '10.10.10.10'
+        mac = 'fe:12:34:56:89:12'
+
+        vlan_dev.del_route(routing_tables_routes, ip, mac)
+
+        self.assertDictEqual(routing_tables_routes, {self.veth_vrf: [{
+            'ip': '10.10.10.10',
+            'mask': '32',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '10.10.20.10',
+        }]})
+
+        self.mock_linux_net.del_ip_route.assert_not_called()
+
+    def test_evpnbridge_vlan_cleanup_excessive_routes(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev._setup_done = True
+
+        intf_idx = 1337
+        self.mock_linux_net.get_interface_index.return_value = intf_idx
+
+        routes = utils.create_linux_routes([{
+            '_attrs': [
+                ('RTA_DST', '198.51.100.0'), ('RTA_OIF', intf_idx),
+                ('RTA_GATEWAY', '100.64.0.102')
+            ],
+            'dst_len': 28, 'type': 1,
+        }, {
+            'attrs': [('RTA_DST', '198.51.100.136'), ('RTA_OIF', intf_idx)],
+            'dst_len': 32, 'type': 1,
+        }, {
+            'attrs': [('RTA_DST', '198.51.100.158'), ('RTA_OIF', intf_idx)],
+            'dst_len': 32, 'type': 1,
+        }])
+        self.mock_linux_net._get_table_routes.return_value = routes
+
+        routing_tables_routes = {self.veth_vrf: [{
+            'ip': '198.51.100.0',
+            'mask': '28',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '100.64.0.102',
+        }]}
+        del_route = mock.patch.object(vlan_dev, 'del_route').start()
+
+        vlan_dev.cleanup_excessive_routes(routing_tables_routes)
+
+        calls = [
+            mock.call(mock.ANY, '198.51.100.136'),
+            mock.call(mock.ANY, '198.51.100.158'),
+        ]
+        del_route.assert_has_calls(calls, any_order=True)
+
+    def test_evpnbridge_vlan_cleanup_excessive_routes_in_sync(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev._setup_done = True
+
+        intf_idx = 1337
+        self.mock_linux_net.get_interface_index.return_value = intf_idx
+
+        routes = utils.create_linux_routes([{
+            '_attrs': [
+                ('RTA_DST', '198.51.100.0'), ('RTA_OIF', intf_idx),
+                ('RTA_GATEWAY', '100.64.0.102')
+            ],
+            'dst_len': 28, 'type': 1,
+        }])
+        self.mock_linux_net._get_table_routes.return_value = routes
+
+        routing_tables_routes = {self.veth_vrf: [{
+            'ip': '198.51.100.0',
+            'mask': '28',
+            'mac': 'fe:12:34:56:89:12',
+            'via': '100.64.0.102',
+        }]}
+        del_route = mock.patch.object(vlan_dev, 'del_route').start()
+
+        vlan_dev.cleanup_excessive_routes(routing_tables_routes)
+        del_route.assert_not_called()
+
+    def test_evpnbridge_vlan_cleanup_excessive_routes_not_setup_yet(self):
+        _, _, vlan_dev = self._create_bridge_and_vlan()
+        vlan_dev.cleanup_excessive_routes({})
+        self.mock_linux_net._get_table_routes.assert_not_called()
+
+    def test_evpn__find_route_info(self):
+        result = evpn._find_route_info([
+            {'ip': '198.51.100.136', 'mask': '32', 'mac': None, 'via': None},
+            {'ip': '198.51.100.158', 'mask': '24', 'mac': None, 'via': None},
+            {'ip': '127.0.0.1', 'mask': '8', 'mac': None, 'via': None},
+            {'ip': '198.51.100.0', 'mask': '28', 'mac': None, 'via': None},
+        ], '127.0.0.1')
+        self.assertDictEqual(result, {'ip': '127.0.0.1', 'mask': '8',
+                                      'mac': None, 'via': None})
+
+    def test_evpn__find_route_info_not_found(self):
+        result = evpn._find_route_info([], '127.0.0.1')
+        self.assertDictEqual(result, {'ip': '127.0.0.1', 'mask': None,
+                                      'mac': None, 'via': None})
+
+    def test_evpn__ensure_list(self):
+        self.assertListEqual(evpn._ensure_list(None), [])
+        self.assertListEqual(evpn._ensure_list('aa'), ['aa'])
+        self.assertListEqual(evpn._ensure_list(['aa']), ['aa'])
+        self.assertSetEqual(evpn._ensure_list({'aa'}), {'aa'})
+        self.assertTupleEqual(evpn._ensure_list(('a', 'b',)), ('a', 'b'))
+
+    def test__offset_for_vni_and_vlan(self):
+        vni = 100
+        vlan = 100
+        exp = int(('%x' % vni).zfill(6) + ('%x' % vlan).zfill(4), 16)
+        self.assertEqual(exp, evpn._offset_for_vni_and_vlan(vni, vlan))
+
+        vni = 16777214
+        vlan = 4094
+        exp = int(('%x' % vni).zfill(6) + ('%x' % vlan).zfill(4), 16)
+        self.assertEqual(exp, evpn._offset_for_vni_and_vlan(vni, vlan))
+
+        # Below value is 1 too much, so it is being modulo'd, and should be
+        # calculated to 0
+        vni = 16777215
+        vlan = 4095
+        exp = int(('%x' % vni).zfill(6) + ('%x' % vlan).zfill(4), 16)
+        self.assertNotEqual(exp, evpn._offset_for_vni_and_vlan(vni, vlan))
+        self.assertEqual(0, evpn._offset_for_vni_and_vlan(vni, vlan))

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_frr.py

@@ -93,6 +93,37 @@ class TestFrr(test_base.TestCase):
         self._test_vrf_reconfigure(add_vrf=False)
 
     def test_vrf_reconfigure_unknown_action(self):
-        frr_utils.vrf_reconfigure('fake-evpn-info', 'non-existing-action')
+        frr_utils.vrf_reconfigure({'fake': 'evpn-info'}, 'non-existing-action')
         # Assert run_vtysh_command() wasn't called
         self.assertFalse(self.mock_vtysh.run_vtysh_config.called)
+
+    @mock.patch.object(tempfile, 'NamedTemporaryFile')
+    def _test_nd_reconfigure(self, mock_tf, stateless=False):
+        interface = 'veth-vrf-100'
+        prefix = '2a04::/64'
+        opts = {
+            'dhcpv6_stateless': str(stateless),
+            'dns_server': '{2001:4860:4860::8888, 2001:4860:4860::8844}',
+        }
+
+        frr_utils.nd_reconfigure(interface, prefix, opts)
+
+        write_arg = mock_tf.return_value.write.call_args_list[0][0][0]
+
+        self.assertIn("ipv6 nd managed-config-flag", write_arg)
+        self.assertIn("ipv6 nd rdnss 2001:4860:4860::8888", write_arg)
+        self.assertIn("ipv6 nd rdnss 2001:4860:4860::8844", write_arg)
+        self.assertIn("ipv6 nd prefix 2a04::/64", write_arg)
+        if not stateless:
+            self.assertIn('ipv6 nd prefix 2a04::/64 no-autoconfig', write_arg)
+
+        self.mock_vtysh.run_vtysh_config.assert_called_once_with(
+            mock_tf.return_value.name)
+        # Assert the file was closed
+        mock_tf.return_value.close.assert_called_once_with()
+
+    def test_nd_reconfigure_statefull(self):
+        self._test_nd_reconfigure()
+
+    def test_nd_reconfigure_stateless(self):
+        self._test_nd_reconfigure(stateless=True)

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_ovn.py

@@ -66,6 +66,7 @@ class TestOvsdbNbOvnIdl(test_base.TestCase):
         self.assertEqual(tag, ret)
         self.nb_idl.db_find_rows.assert_called_once_with(
             'Logical_Switch_Port',
+            ('options', '=', {'network_name': network_name}),
             ('type', '=', constants.OVN_LOCALNET_VIF_PORT_TYPE))
 
     def test_ls_has_virtual_ports(self):

ovn_bgp_agent/tests/unit/drivers/openstack/utils/test_wire.py

@@ -18,11 +18,13 @@ from unittest import mock
 from oslo_config import cfg
 
 from ovn_bgp_agent import constants
+from ovn_bgp_agent.drivers.openstack.utils import evpn as evpn_utils
 from ovn_bgp_agent.drivers.openstack.utils import ovn as ovn_utils
 from ovn_bgp_agent.drivers.openstack.utils import ovs as ovs_utils
 from ovn_bgp_agent.drivers.openstack.utils import wire
 from ovn_bgp_agent import exceptions as agent_exc
 from ovn_bgp_agent.tests import base as test_base
+from ovn_bgp_agent.tests import utils as test_utils
 from ovn_bgp_agent.utils import linux_net
 
 CONF = cfg.CONF
@@ -37,7 +39,10 @@ class TestWire(test_base.TestCase):
         self.ovs_idl = mock.Mock()
 
         # Helper variables that are used across multiple methods
-        self.bridge_mappings = 'datacentre:br-ex'
+        self.bridge_mappings = ['datacentre:br-ex']
+
+        self.ovs_idl.get_ovn_bridge_mappings.return_value = (
+            self.bridge_mappings)
 
         # Monkey-patch parent class methods
         self.nb_idl.ls_add = mock.Mock()
@@ -63,21 +68,82 @@ class TestWire(test_base.TestCase):
                                        ovn_idl=self.nb_idl)
         mock_ovn.assert_called_once_with(self.ovs_idl, self.nb_idl)
 
-    @mock.patch.object(wire, '_ensure_base_wiring_config_underlay')
+    @mock.patch.object(wire, '_ensure_base_wiring_config_evpn')
-    @mock.patch.object(wire, '_ensure_base_wiring_config_ovn')
+    def test_ensure_base_wiring_config_evpn(self, mock_evpn):
-    def test_ensure_base_wiring_config_not_implemeneted(self, mock_ovn,
-                                                        mock_underlay):
         CONF.set_override('exposing_method', 'vrf')
         self.addCleanup(CONF.clear_override, 'exposing_method')
 
-        wire.ensure_base_wiring_config(self.sb_idl, self.ovs_idl,
+        wire.ensure_base_wiring_config(self.nb_idl, self.ovs_idl,
                                        ovn_idl=self.nb_idl)
+        mock_evpn.assert_called_once_with(self.nb_idl, self.ovs_idl)
+
+    @mock.patch.object(wire, '_ensure_base_wiring_config_underlay')
+    @mock.patch.object(wire, '_ensure_base_wiring_config_ovn')
+    def test_ensure_base_wiring_config_not_implemented(self, mock_ovn,
+                                                       mock_underlay):
+        CONF.set_override('exposing_method', 'dynamic')
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        self.assertRaises(agent_exc.UnsupportedWiringConfig,
+                          wire.ensure_base_wiring_config,
+                          self.sb_idl, self.ovs_idl, ovn_idl=self.nb_idl)
         mock_ovn.assert_not_called()
         mock_underlay.assert_not_called()
 
     def test__ensure_base_wiring_config_ovn(self):
         pass
 
+    @mock.patch.object(linux_net, 'get_interface_address')
+    @mock.patch.object(ovs_utils, 'get_ovs_patch_ports_info')
+    def test__ensure_base_wiring_config_evpn(self, m_get_ovs_patch_ports_info,
+                                             m_get_interface_address):
+        localnet_ports = [test_utils.create_row(
+            tag=[4096],
+        )]
+        ports = localnet_ports + [test_utils.create_row(
+            dhcpv4_options=[test_utils.create_row()],
+            dhcpv6_options=[],
+        )]
+        get_localnet_ports_by_network_name = mock.patch.object(
+            self.nb_idl, 'get_localnet_ports_by_network_name').start()
+        get_localnet_ports_by_network_name.return_value = localnet_ports
+
+        provnets = [test_utils.create_row(
+            name='fake-provnet',
+            external_ids={
+                constants.OVN_EVPN_VNI_EXT_ID_KEY: 100,
+            },
+            ports=ports,
+        )]
+        get_bgpvpn_networks_for_ports = mock.patch.object(
+            self.nb_idl, 'get_bgpvpn_networks_for_ports').start()
+        get_bgpvpn_networks_for_ports.return_value = provnets
+
+        CONF.set_override('evpn_local_ip', '127.0.0.1')
+        self.addCleanup(CONF.clear_override, 'evpn_local_ip')
+
+        vlan_dev = mock.MagicMock()
+
+        evpn_bridge = mock.MagicMock()
+        evpn_bridge.connect_vlan.return_value = vlan_dev
+
+        evpn_setup = mock.patch.object(evpn_utils, 'setup').start()
+        evpn_setup.return_value = evpn_bridge
+
+        wire._ensure_base_wiring_config_evpn(self.nb_idl, self.ovs_idl)
+
+        evpn_setup.assert_called_with(ovs_bridge='br-ex',
+                                      vni=100,
+                                      evpn_opts={'route_targets': [],
+                                                 'route_distinguishers': [],
+                                                 'export_targets': [],
+                                                 'import_targets': []},
+                                      mode=constants.OVN_EVPN_TYPE_L3,
+                                      ovs_flows=mock.ANY)
+
+        evpn_bridge.connect_vlan.assert_called_with(ports[0])
+        vlan_dev.process_dhcp_opts.assert_called()
+
     def test__ensure_ovn_router(self):
         wire._ensure_ovn_router(self.nb_idl)
         self.nb_idl.lr_add.assert_called_once_with(
@@ -272,9 +338,34 @@ class TestWire(test_base.TestCase):
                                   routing_tables_routes)
         self.assertTrue(ret)
 
+    def test_cleanup_wiring_evpn(self):
+        CONF.set_override('exposing_method', 'vrf')
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        vlan_dev = mock.MagicMock()
+        evpn_bridge = mock.MagicMock()
+        evpn_bridge.get_vlan.return_value = vlan_dev
+        ovs_flows = {
+            'foo': {
+                'evpn': {
+                    '4096': evpn_bridge,
+                }
+            }
+        }
+        exposed_ips = {}
+        routing_tables = {}
+        routing_tables_routes = {}
+        ret = wire.cleanup_wiring(self.sb_idl, self.bridge_mappings, ovs_flows,
+                                  exposed_ips, routing_tables,
+                                  routing_tables_routes)
+
+        evpn_bridge.get_vlan.assert_called_with('4096')
+        vlan_dev.cleanup_excessive_routes.assert_called()
+        self.assertTrue(ret)
+
     @mock.patch.object(wire, '_cleanup_wiring_underlay')
     def test_cleanup_wiring_not_implemeneted(self, mock_underlay):
-        CONF.set_override('exposing_method', 'vrf')
+        CONF.set_override('exposing_method', 'dynamic')
         self.addCleanup(CONF.clear_override, 'exposing_method')
 
         ovs_flows = {}
@@ -325,10 +416,68 @@ class TestWire(test_base.TestCase):
                                 ovn_idl=self.nb_idl)
         mock_ovn.assert_called_once_with(self.nb_idl, port_ips, mac)
 
+    def test_wire_provider_port_evpn(self):
+        CONF.set_override('exposing_method', 'vrf')
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        routing_tables_routes = {}
+        ovs_flows = {}
+        port_ips = ['10.10.10.1']
+        bridge_device = 'fake-bridge'
+        bridge_vlan = '101'
+        localnet = 'fake-localnet'
+        routing_table = 5
+        proxy_cidrs = []
+        mac = 'fake-mac'
+
+        vlan_dev = mock.MagicMock()
+
+        evpn_bridge = mock.MagicMock()
+        evpn_bridge.get_vlan.return_value = vlan_dev
+
+        evpn_lookup = mock.patch.object(evpn_utils, 'lookup').start()
+        evpn_lookup.return_value = evpn_bridge
+
+        ret = wire.wire_provider_port(routing_tables_routes, ovs_flows,
+                                      port_ips, bridge_device, bridge_vlan,
+                                      localnet, routing_table, proxy_cidrs,
+                                      mac=mac, ovn_idl=self.nb_idl)
+        self.assertTrue(ret)
+
+        evpn_lookup.assert_called_once_with(bridge_device, bridge_vlan)
+        evpn_bridge.get_vlan.assert_called_once_with(bridge_vlan)
+        vlan_dev.add_route.assert_called_with(routing_tables_routes,
+                                              port_ips[0], mac, via=None)
+
+    def test_wire_provider_port_evpn_unconfigured(self):
+        CONF.set_override('exposing_method', 'vrf')
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        routing_tables_routes = {}
+        ovs_flows = {}
+        port_ips = ['10.10.10.1']
+        bridge_device = 'fake-bridge'
+        bridge_vlan = '101'
+        localnet = 'fake-localnet'
+        routing_table = 5
+        proxy_cidrs = []
+        mac = 'fake-mac'
+
+        evpn_lookup = mock.patch.object(evpn_utils, 'lookup').start()
+        evpn_lookup.side_effect = KeyError
+
+        ret = wire.wire_provider_port(routing_tables_routes, ovs_flows,
+                                      port_ips, bridge_device, bridge_vlan,
+                                      localnet, routing_table, proxy_cidrs,
+                                      mac=mac, ovn_idl=self.nb_idl)
+        self.assertIsNone(ret)
+
     @mock.patch.object(wire, '_wire_provider_port_underlay')
     @mock.patch.object(wire, '_wire_provider_port_ovn')
-    def test_wire_provider_port_not_implemented(self, mock_ovn, mock_underlay):
+    @mock.patch.object(wire, '_wire_provider_port_evpn')
-        CONF.set_override('exposing_method', 'vrf')
+    def test_wire_provider_port_not_implemented(self, mock_evpn, mock_ovn,
+                                                mock_underlay):
+        CONF.set_override('exposing_method', 'dynamic')
         self.addCleanup(CONF.clear_override, 'exposing_method')
 
         routing_tables_routes = {}
@@ -344,6 +493,7 @@ class TestWire(test_base.TestCase):
                                 bridge_device, bridge_vlan, localnet,
                                 routing_table, proxy_cidrs)
 
+        mock_evpn.assert_not_called()
         mock_ovn.assert_not_called()
         mock_underlay.assert_not_called()
 
@@ -410,10 +560,8 @@ class TestWire(test_base.TestCase):
                                   proxy_cidrs, ovn_idl=self.nb_idl)
         mock_ovn.assert_called_once_with(self.nb_idl, port_ips)
 
-    @mock.patch.object(wire, '_unwire_provider_port_underlay')
+    @mock.patch.object(wire, '_unwire_provider_port_evpn')
-    @mock.patch.object(wire, '_unwire_provider_port_ovn')
+    def test_unwire_provider_port_evpn(self, mock_evpn):
-    def test_unwire_provider_port_not_implemented(self, mock_ovn,
-                                                  mock_underlay):
         CONF.set_override('exposing_method', 'vrf')
         self.addCleanup(CONF.clear_override, 'exposing_method')
 
@@ -424,12 +572,72 @@ class TestWire(test_base.TestCase):
         routing_table = 5
         proxy_cidrs = []
 
+        wire.unwire_provider_port(routing_tables_routes, port_ips,
+                                  bridge_device, bridge_vlan, routing_table,
+                                  proxy_cidrs, lladdr='boo')
+        mock_evpn.assert_called_once_with(routing_tables_routes, port_ips,
+                                          bridge_device, bridge_vlan, 'boo')
+
+    @mock.patch.object(wire, '_unwire_provider_port_underlay')
+    @mock.patch.object(wire, '_unwire_provider_port_ovn')
+    @mock.patch.object(wire, '_unwire_provider_port_evpn')
+    def test_unwire_provider_port_not_implemented(self, mock_evpn, mock_ovn,
+                                                  mock_underlay):
+        CONF.set_override('exposing_method', 'dynamic')
+        self.addCleanup(CONF.clear_override, 'exposing_method')
+
+        routing_tables_routes = {}
+        port_ips = []
+        bridge_device = 'fake-bridge'
+        bridge_vlan = '101'
+        routing_table = 5
+        proxy_cidrs = []
+
         wire.unwire_provider_port(routing_tables_routes, port_ips,
                                   bridge_device, bridge_vlan, routing_table,
                                   proxy_cidrs)
+        mock_evpn.assert_not_called()
         mock_ovn.assert_not_called()
         mock_underlay.assert_not_called()
 
+    def test__unwire_provider_port_evpn(self):
+        routing_tables_routes = {}
+        port_ips = ['10.10.10.1']
+        bridge_device = 'fake-bridge'
+        bridge_vlan = '101'
+        lladdr = 'boo'
+
+        vlan_dev = mock.MagicMock()
+
+        evpn_bridge = mock.MagicMock()
+        evpn_bridge.get_vlan.return_value = vlan_dev
+
+        evpn_lookup = mock.patch.object(evpn_utils, 'lookup').start()
+        evpn_lookup.return_value = evpn_bridge
+
+        ret = wire._unwire_provider_port_evpn(routing_tables_routes, port_ips,
+                                              bridge_device, bridge_vlan,
+                                              lladdr)
+        self.assertTrue(ret)
+
+        vlan_dev.del_route.assert_called_with(routing_tables_routes,
+                                              port_ips[0], lladdr)
+
+    def test__unwire_provider_port_evpn_unconfigured(self):
+        routing_tables_routes = {}
+        port_ips = ['10.10.10.1']
+        bridge_device = 'fake-bridge'
+        bridge_vlan = '101'
+        lladdr = 'boo'
+
+        evpn_lookup = mock.patch.object(evpn_utils, 'lookup').start()
+        evpn_lookup.side_effect = KeyError
+
+        ret = wire._unwire_provider_port_evpn(routing_tables_routes, port_ips,
+                                              bridge_device, bridge_vlan,
+                                              lladdr)
+        self.assertIsNone(ret)
+
     @mock.patch.object(wire, '_execute_commands')
     def test__unwire_provider_port_ovn(self, m_cmds):
         port_ips = ['1.1.1.1']

ovn_bgp_agent/tests/unit/drivers/openstack/watchers/test_base_watcher.py

@@ -103,6 +103,11 @@ class TestOVNLBEvent(test_base.TestCase):
                          ['192.168.1.50', '172.24.4.5'])
 
 
+class FakeLogicalSwitchChassisEvent(base_watcher.LogicalSwitchChassisEvent):
+    def run(self):
+        pass
+
+
 class FakeLSPChassisEvent(base_watcher.LSPChassisEvent):
     def run(self):
         pass

ovn_bgp_agent/tests/unit/drivers/openstack/watchers/test_nb_bgp_watcher.py

@@ -579,6 +579,60 @@ class TestLogicalSwitchPortFIPDeleteEvent(test_base.TestCase):
         self.agent.withdraw_fip.assert_not_called()
 
 
+class TestLogicalSwitchUpdateEvent(test_base.TestCase):
+
+    def setUp(self):
+        super(TestLogicalSwitchUpdateEvent, self).setUp()
+        self.agent = mock.Mock()
+        self.event = nb_bgp_watcher.LogicalSwitchUpdateEvent(
+            self.agent)
+        self.row = utils.create_row(external_ids={
+            constants.OVN_EVPN_TYPE_EXT_ID_KEY: constants.OVN_EVPN_TYPE_L3,
+            constants.OVN_EVPN_VNI_EXT_ID_KEY: 1001,
+        })
+
+    def test_match_fn(self):
+        # ls had no conf in external_ids, does have it now.
+        old = utils.create_row(external_ids={})
+        self.assertTrue(self.event.match_fn(None, self.row, old))
+
+    def test_match_fn_changed_vni(self):
+        old = utils.create_row(external_ids={
+            constants.OVN_EVPN_TYPE_EXT_ID_KEY: constants.OVN_EVPN_TYPE_L3,
+            constants.OVN_EVPN_VNI_EXT_ID_KEY: 1002,
+        })
+        self.assertTrue(self.event.match_fn(None, self.row, old))
+
+    def test_match_fn_deleted_ls(self):
+        self.assertTrue(self.event.match_fn(self.event.ROW_DELETE, self.row,
+                                            None))
+
+    def test_match_fn_no_match(self):
+        # no vrf update in old, should return False
+        old = utils.create_row()
+        self.assertFalse(self.event.match_fn(None, self.row, old))
+
+    def test_match_fn_no_match_same_vni(self):
+        # same update in old, should return False
+        self.assertFalse(self.event.match_fn(None, self.row, self.row))
+
+    def test_match_fn_no_match_incomplete_row(self):
+        # incomplete configuration, should return False
+        row = utils.create_row(external_ids={
+            constants.OVN_EVPN_TYPE_EXT_ID_KEY: constants.OVN_EVPN_TYPE_L3,
+        })
+        self.assertFalse(self.event.match_fn(None, row, None))
+
+        row = utils.create_row(external_ids={
+            constants.OVN_EVPN_VNI_EXT_ID_KEY: 1001,
+        })
+        self.assertFalse(self.event.match_fn(None, row, None))
+
+    def test_run(self):
+        self.event.run(None, None, None)
+        self.agent.sync.assert_called_once()
+
+
 class TestLocalnetCreateDeleteEvent(test_base.TestCase):
 
     def setUp(self):

ovn_bgp_agent/tests/utils.py

@@ -14,6 +14,7 @@
 #    under the License.
 
 import eventlet
+import uuid
 
 
 class WaitTimeout(Exception):
@@ -21,7 +22,22 @@ class WaitTimeout(Exception):
 
 
 def create_row(**kwargs):
-    return type('FakeRow', (object,), kwargs)
+    row = type('FakeRow', (object,), kwargs)
+    if not hasattr(row, 'uuid'):
+        row.uuid = uuid.uuid4()
+    return row
+
+
+class FakeLinuxRoute(dict):
+    def get_attr(self, key, default=None):
+        for k, v in self.get('attrs', []):
+            if k == key:
+                return v
+        return default
+
+
+def create_linux_routes(route_info) -> 'list[FakeLinuxRoute]':
+    return [FakeLinuxRoute(r) for r in route_info]
 
 
 def wait_until_true(predicate, timeout=60, sleep=1, exception=None):

ovn_bgp_agent/utils/linux_net.py

@@ -156,6 +156,65 @@ def ensure_arp_ndp_enabled_for_bridge(bridge, offset, vlan_tag=None):
     enable_proxy_ndp(bridge)
 
 
+def ensure_anycast_mac_for_interface(intf, offset):
+    # Make pointer to module, to shorten amount of chars to call module.
+    priv = ovn_bgp_agent.privileged.linux_net
+
+    # The intf acting as L3 GW (needs to have same mac address everywhere)
+    # So generate the mac address based on the given offset and add the
+    # configured MAC_LLADR_OFFSET to it.
+    mac_int = int(constants.MAC_LLADDR_OFFSET.replace(':', ''), 16)
+    ll = ('%x' % int(mac_int + offset)).zfill(12)
+    lladdr = ":".join([ll[i:i + 2] for i in range(0, len(ll), 2)])
+
+    # Check what the mac address currently is.
+    dev = priv.get_link_device(intf)
+    curr_lladdr = priv.get_attr(dev, 'IFLA_ADDRESS')
+
+    if lladdr != curr_lladdr:
+        LOG.info("Updating mac address for intf %s to address %s",
+                 intf, lladdr)
+        priv.set_link_attribute(intf, address=lladdr)
+
+        # Also update the 'scope link' address on the interface.
+        ll = lladdr.replace(':', '')
+        ll_ip_address = ipaddress.IPv6Address(
+            f'fe80::{ll[0:4]}:{ll[4:8]}:{ll[8:12]}')
+        ll_net = ipaddress.IPv6Network('fe80::/10')
+
+        # Fetch all ipv6 addresses and check if we already configured the
+        # link-local address.
+        addresses = priv.get_ip_addresses(
+            index=dev['index'], family=constants.AF_INET6)
+        for addr in addresses:
+            ip_addr = ipaddress.IPv6Address(
+                priv.get_attr(addr, 'IFA_ADDRESS'))
+
+            if (addr['scope'] != 0 and
+                    ip_addr in ll_net and
+                    ip_addr != ll_ip_address):
+
+                LOG.info('Update scope link address on intf %s from %s to %s',
+                         intf, ip_addr, ll_ip_address)
+
+                # Delete the old link local ip address from the interface
+                priv.delete_ip_address(ip_addr.compressed, intf,
+                                       prefixlen=addr['prefixlen'],
+                                       scope=addr['scope'])
+
+                # Attach our anycast link local address to the interface
+                priv.add_ip_address(ll_ip_address.compressed, intf,
+                                    prefixlen=addr['prefixlen'],
+                                    scope=addr['scope'])
+
+
+def disable_learning_vxlan_intf(intf):
+    '''ip link set vni200 type bridge_slave neigh_suppress on learning off'''
+    ovn_bgp_agent.privileged.linux_net.set_brport_attribute(
+        intf, neigh_suppress=True, learning=False
+    )
+
+
 def ensure_routing_table_for_bridge(ovn_routing_tables, bridge, vrf_table):
     # check a routing table with the bridge name exists on
     # /etc/iproute2/rt_tables
@@ -368,6 +427,23 @@ def enable_proxy_arp(device):
     ovn_bgp_agent.privileged.linux_net.set_kernel_flag(flag, 1)
 
 
+def enable_routing_for_interfaces(*interfaces):
+    # Configure sysctl
+    keys = [
+        ('net.ipv4.ip_forward', 1),
+        ('net.ipv4.conf.all.forwarding', 1),
+        ('net.ipv6.conf.all.forwarding', 1),
+    ]
+    for intf in interfaces:
+        intf_key = intf.replace('.', '/')
+        keys.append((f'net.ipv4.conf.{intf_key}.forwarding', 1))
+        keys.append((f'net.ipv6.conf.{intf_key}.forwarding', 1))
+
+    for k, v in keys:
+        LOG.debug('Configure sysctl %s=%s', k, v)
+        ovn_bgp_agent.privileged.linux_net.set_kernel_flag(k, v)
+
+
 @tenacity.retry(
     retry=tenacity.retry_if_exception_type(
         netlink_exceptions.NetlinkDumpInterrupted),