rustbgpd
Cookbook

EVPN fabric route reflector (control-plane only)

This recipe builds a control-plane-only VXLAN-EVPN fabric route reflector.

This recipe builds a control-plane-only VXLAN-EVPN fabric route reflector.

When this is you: You operate a VXLAN-EVPN leaf/spine fabric. Its VTEPs (FRR, SR Linux, GoBGP, or rustbgpd leaves) do the dataplane, and you want a lean, API-first reflector distributing the RFC 7432 routes between them. Scope, stated up front: this recipe is the RR role. The RR holds no EVI state, learns no MACs, and forwards no packets — it reflects EVPN Types 1–6 between clients, including Type 6 SMET relay. rustbgpd also has a bidirectional VTEP mode (alpha, Linux/VXLAN-only, with IRB and multi-homing — see evpn-enablement.md); that is a different deployment and not this document.

Proven by: M29 (EVPN RR capability + ListEvpnRoutes vs FRR), M30 (Type 2 MAC reflection end-to-end between kernel VXLAN VTEPs), M31 (MAC mobility + sticky preservation), M32 (multi-homing Type 1 EAD / Type 4 ES reflection), M82 (VLAN-aware-bundle reflection with non-zero Ethernet Tags — including rustbgpd's first vendor-NOS leg, Nokia SR Linux 25.10), and the M33 scale gate (50k reflected Type 2 routes + 60 s of 1,000-rps churn). The config below is a trimmed form of examples/rr-evpn-fabric/config.toml, which also disables [policy.explain] explicitly and uses the implicit gRPC socket path.

Type 6 SMET relay remains alpha. Its M113 raw-peer proof checks reflected wire bytes with an independent TShark decoder, including withdrawal and error recovery. The earlier receipts above cover other route types. M113 does not establish vendor interoperability; no SMET origination, IGMP/MLD proxy, or multicast forwarding is implemented. See the SMET boundary.

Config

[global]
asn = 65000
router_id = "10.0.0.100"
listen_port = 179
cluster_id = "10.0.0.100"
ebgp_requires_policy = false

[global.telemetry]
prometheus_addr = "127.0.0.1:9179"
log_format = "json"

# Owner-only local socket (default mode 0600): clients are authorized as the
# implicit "local-operator" principal — no [security.grpc] block needed.
[global.telemetry.grpc_uds]
path = "/var/lib/rustbgpd/grpc.sock"

# All VTEPs are iBGP RR clients on the l2vpn_evpn family. Empty
# [[evpn_instances]] (none configured) selects pure RR mode: no local
# EVI state, no kernel programming, reflection only.
[[neighbors]]
address = "10.0.0.1"
remote_asn = 65000
description = "vtep-leaf-01"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = true

[[neighbors]]
address = "10.0.0.2"
remote_asn = 65000
description = "vtep-leaf-02"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = true

[[neighbors]]
address = "10.0.0.3"
remote_asn = 65000
description = "vtep-leaf-03"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = true

For larger fabrics, template the leaves with a peer group and an auto-accept range over the VTEP loopback subnet — the unicast RR recipe shows the [peer_groups] + [[dynamic_neighbors]] shape; swap the families for ["l2vpn_evpn"].

Verify

$ export RUSTBGPD_ADDR=unix:///var/lib/rustbgpd/grpc.sock
$ rbgp neighbor                      # every leaf Established, l2vpn_evpn negotiated
$ rbgp evpn                          # all reflected EVPN routes
$ rbgp evpn --route-type 2           # MAC/IP advertisements only
$ rbgp evpn --route-type 3           # IMET (BUM flooding) entries
$ rbgp evpn --rd 65000:100           # one EVI's view
$ rbgp evpn --neighbor 10.0.0.1      # what leaf-01 contributed

Expected shape: one row per (RD, route-type, key) with the originating peer; after two leaves come up with the same L2VNI you should see each leaf's Type 3 IMET route reflected to the other, then Type 2 rows appearing as MACs are learned.

Two things that look odd but are correct:

  • The same MAC under two Ethernet Tags is two routes. In VLAN-aware-bundle service the non-zero Ethernet Tag is part of the route identity (ADR-0092); the RR keys and reflects them separately, tag-verbatim, and a single-tag withdraw removes exactly that entry (M82).
  • Attributes pass through untouched. RD, label/VNI, next-hop, RTs, MAC-mobility sequence numbers, ESI — the RR adds ORIGINATOR_ID and CLUSTER_LIST per RFC 4456 and changes nothing else. DF election, ARP/ND suppression, and mobility sequencing are the VTEPs' business.

Watch

The Grafana overview session and RIB-scale rows apply as-is (bgp_session_state_transitions_total, bgp_rib_outbound_registered_peers, update-group gauges). Per-VNI EVPN metric families (evpn_*) are VTEP-mode surface and stay empty in the RR role — deliberately not on the overview dashboard.

Route churn is the fabric health signal on an EVPN RR. EVPN route events have their own category, which a default event stream does not include: watch rbgp events watch --category evpn (or the EVPN history, rbgp events evpn --limit 200, filterable with --route-type 2, --rd, and --neighbor) during rollouts. MAC-mobility wars show up as a tight add/withdraw loop on one MAC key with a climbing sequence number.

Failure modes

A leaf's routes aren't reaching another leaf. Start with rbgp evpn received 10.0.0.1 for accepted post-policy source routes and rbgp evpn advertised 10.0.0.2 for committed destination output. Both commands accept type/RD filters and return a bounded page; use the returned --page-token to continue. Then explain the exact key with its source and destination. For a MAC-only Type 2 route:

rbgp evpn explain mac-ip --rd 65000:100 --mac 02:00:00:00:00:11 \
  --received-from 10.0.0.1 --advertised-to 10.0.0.2

Add --ip for a MAC+IP key; use the route's actual --ethernet-tag when it is nonzero. Other route types have their own exact selectors. The result separates retained accepted input, installed best, fresh selection, current export gates, and committed output. A missing accepted source does not prove the leaf never sent the route, and committed output does not prove the other leaf received it. Check selection_deferred and outbound_dirty before expecting installed or committed state to match a current comparison.

To inspect SMET relay, filter on Type 6 and explain its exact key:

rbgp evpn received 10.0.0.1 --route-type 6
rbgp evpn explain smet --rd 65000:100 --source '*' --group 239.1.2.3 \
  --originator-ip 2001:db8::1 --advertised-to 10.0.0.2

'*' is the route's literal wildcard source, not a query pattern. Source/group families must agree when both are concrete; the originator family is independent.

Two common reflection stops are family — the destination did not negotiate l2vpn_evpn — and RR rules — the source and destination's route_reflector_client settings do not permit this reflection. The trace shows the stopping gate. An FRR leaf missing neighbor X activate under address-family l2vpn evpn can establish a session while receiving no EVPN routes, as demonstrated by the M29 fixture.

A third stop, rt_membership, applies when the destination leaf also negotiated rtc: it receives only EVPN routes with a Route Target inside the RT membership it advertised (Type 4 routes match on their ES-Import RT), and nothing until it advertises some. rbgp rib rtc --neighbor <leaf> lists the membership that leaf sent; the default (zero-length) membership restores the unfiltered feed.

Vendor NOS quirks. From the M82 SR Linux leg: SR Linux enforces one EVI per mac-vrf (bundle identity = shared RT + Ethernet Tag, per-BD RDs) and needs an explicit transport local-address on the BGP group, or it sources the session from its system0 address and the RR's neighbor stanza never matches. Recorded in upstream-findings.md and the M82 lab fixtures.

Reflected state survives leaf restarts? Add GR to the neighbor stanzas (graceful_restart = true and friends, as in the unicast recipe) if your leaves support GR for EVPN; without it, a leaf bounce withdraws its routes fabric-wide and they re-learn on re-establish.

Source on GitHub

On this page