EVPN fabric route reflector (control-plane only)
This recipe builds a control-plane-only VXLAN-EVPN fabric route reflector.
This recipe builds a control-plane-only VXLAN-EVPN fabric route reflector.
When this is you: You operate a VXLAN-EVPN leaf/spine fabric. Its
VTEPs (FRR, SR Linux, GoBGP, or rustbgpd leaves) do the dataplane, and
you want a lean, API-first reflector distributing the RFC 7432 routes
between them. Scope, stated up front: this recipe is the RR role. The
RR holds no EVI state, learns no MACs, and forwards no packets — it
reflects EVPN Types 1–6 between clients, including Type 6 SMET relay. rustbgpd
also has a bidirectional VTEP mode (alpha, Linux/VXLAN-only, with IRB
and multi-homing — see
evpn-enablement.md); that is a different
deployment and not this document.
Proven by: M29
(EVPN RR capability + ListEvpnRoutes vs FRR), M30 (Type 2 MAC
reflection end-to-end between kernel VXLAN VTEPs), M31 (MAC mobility +
sticky preservation), M32 (multi-homing Type 1 EAD / Type 4 ES
reflection), M82 (VLAN-aware-bundle reflection with non-zero Ethernet
Tags — including rustbgpd's first vendor-NOS leg, Nokia SR Linux
25.10), and the M33 scale gate (50k reflected Type 2 routes + 60 s of
1,000-rps churn). The config below is a trimmed form of
examples/rr-evpn-fabric/config.toml,
which also disables [policy.explain] explicitly and uses the implicit gRPC
socket path.
Type 6 SMET relay remains alpha. Its M113 raw-peer proof checks reflected wire bytes with an independent TShark decoder, including withdrawal and error recovery. The earlier receipts above cover other route types. M113 does not establish vendor interoperability; no SMET origination, IGMP/MLD proxy, or multicast forwarding is implemented. See the SMET boundary.
Config
[global]
asn = 65000
router_id = "10.0.0.100"
listen_port = 179
cluster_id = "10.0.0.100"
ebgp_requires_policy = false
[global.telemetry]
prometheus_addr = "127.0.0.1:9179"
log_format = "json"
# Owner-only local socket (default mode 0600): clients are authorized as the
# implicit "local-operator" principal — no [security.grpc] block needed.
[global.telemetry.grpc_uds]
path = "/var/lib/rustbgpd/grpc.sock"
# All VTEPs are iBGP RR clients on the l2vpn_evpn family. Empty
# [[evpn_instances]] (none configured) selects pure RR mode: no local
# EVI state, no kernel programming, reflection only.
[[neighbors]]
address = "10.0.0.1"
remote_asn = 65000
description = "vtep-leaf-01"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = true
[[neighbors]]
address = "10.0.0.2"
remote_asn = 65000
description = "vtep-leaf-02"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = true
[[neighbors]]
address = "10.0.0.3"
remote_asn = 65000
description = "vtep-leaf-03"
hold_time = 180
families = ["l2vpn_evpn"]
route_reflector_client = trueFor larger fabrics, template the leaves with a peer group and an
auto-accept range over the VTEP loopback subnet — the
unicast RR recipe shows the
[peer_groups] + [[dynamic_neighbors]] shape; swap the families for
["l2vpn_evpn"].
Verify
$ export RUSTBGPD_ADDR=unix:///var/lib/rustbgpd/grpc.sock
$ rbgp neighbor # every leaf Established, l2vpn_evpn negotiated
$ rbgp evpn # all reflected EVPN routes
$ rbgp evpn --route-type 2 # MAC/IP advertisements only
$ rbgp evpn --route-type 3 # IMET (BUM flooding) entries
$ rbgp evpn --rd 65000:100 # one EVI's view
$ rbgp evpn --neighbor 10.0.0.1 # what leaf-01 contributedExpected shape: one row per (RD, route-type, key) with the originating peer; after two leaves come up with the same L2VNI you should see each leaf's Type 3 IMET route reflected to the other, then Type 2 rows appearing as MACs are learned.
Two things that look odd but are correct:
- The same MAC under two Ethernet Tags is two routes. In VLAN-aware-bundle service the non-zero Ethernet Tag is part of the route identity (ADR-0092); the RR keys and reflects them separately, tag-verbatim, and a single-tag withdraw removes exactly that entry (M82).
- Attributes pass through untouched. RD, label/VNI, next-hop, RTs, MAC-mobility sequence numbers, ESI — the RR adds ORIGINATOR_ID and CLUSTER_LIST per RFC 4456 and changes nothing else. DF election, ARP/ND suppression, and mobility sequencing are the VTEPs' business.
Watch
The Grafana overview session and RIB-scale rows apply
as-is (bgp_session_state_transitions_total,
bgp_rib_outbound_registered_peers, update-group gauges). Per-VNI
EVPN metric families (evpn_*) are VTEP-mode surface and stay empty
in the RR role — deliberately not on the overview dashboard.
Route churn is the fabric health signal on an EVPN RR. EVPN route events
have their own category, which a default event stream does not include: watch
rbgp events watch --category evpn (or the EVPN history,
rbgp events evpn --limit 200, filterable with --route-type 2, --rd, and
--neighbor) during rollouts. MAC-mobility wars show up as a tight
add/withdraw loop on one MAC key with a climbing sequence number.
Failure modes
A leaf's routes aren't reaching another leaf. Start with
rbgp evpn received 10.0.0.1 for accepted post-policy source routes and
rbgp evpn advertised 10.0.0.2 for committed destination output. Both
commands accept type/RD filters and return a bounded page; use the returned
--page-token to continue. Then explain the exact key with its source and
destination. For a MAC-only Type 2 route:
rbgp evpn explain mac-ip --rd 65000:100 --mac 02:00:00:00:00:11 \
--received-from 10.0.0.1 --advertised-to 10.0.0.2Add --ip for a MAC+IP key; use the route's actual --ethernet-tag when it
is nonzero. Other route types have their own
exact selectors.
The result separates retained accepted input, installed best, fresh selection,
current export gates, and committed output. A missing accepted source does not
prove the leaf never sent the route, and committed output does not prove the
other leaf received it. Check selection_deferred and outbound_dirty before
expecting installed or committed state to match a current comparison.
To inspect SMET relay, filter on Type 6 and explain its exact key:
rbgp evpn received 10.0.0.1 --route-type 6
rbgp evpn explain smet --rd 65000:100 --source '*' --group 239.1.2.3 \
--originator-ip 2001:db8::1 --advertised-to 10.0.0.2'*' is the route's literal wildcard source, not a query pattern. Source/group
families must agree when both are concrete; the originator family is independent.
Two common reflection stops are family — the destination did not negotiate
l2vpn_evpn — and RR rules — the source and destination's
route_reflector_client settings do not permit this reflection. The trace
shows the stopping gate. An FRR leaf missing neighbor X activate under
address-family l2vpn evpn can establish a session while receiving no EVPN
routes, as demonstrated by the M29 fixture.
A third stop, rt_membership, applies when the destination leaf also
negotiated rtc: it receives only EVPN routes with a Route Target inside the
RT membership it advertised (Type 4 routes match on their ES-Import RT), and
nothing until it advertises some. rbgp rib rtc --neighbor <leaf> lists the
membership that leaf sent; the default (zero-length) membership restores the
unfiltered feed.
Vendor NOS quirks. From the M82 SR Linux leg: SR Linux enforces
one EVI per mac-vrf (bundle identity = shared RT + Ethernet Tag,
per-BD RDs) and needs an explicit transport local-address on the BGP
group, or it sources the session from its system0 address and the RR's
neighbor stanza never matches. Recorded in
upstream-findings.md and the M82 lab
fixtures.
Reflected state survives leaf restarts? Add GR to the neighbor
stanzas (graceful_restart = true and friends, as in the
unicast recipe) if your leaves support
GR for EVPN; without it, a leaf bounce withdraws its routes
fabric-wide and they re-learn on re-establish.