SONiC FRR: Configure BGP on a SONiC Switch

SONiC runs FRR as its routing stack, so BGP on a SONiC switch is FRR BGP. Interfaces live in SONiC's CONFIG_DB and you set them with config. BGP lives in FRR and you set it with vtysh. Each half has its own save command.

This tutorial builds two spines, two leaves and two hosts on real SONiC switches, then walks the traps that make people search for sonic frr in the first place. All output below comes from a run on the community docker-sonic-vs image from the 202511 branch, with FRR 10.4.1.

The community image needs one step a data sheet will not tell you: bgpd is stopped at boot. On NetPilot the lab VM starts it for you.

Build the fabric

Ask the NetPilot agent:

Build two SONiC spines and two SONiC leaves. eBGP underlay, spines in AS 65000, leaf1 in AS 65001, leaf2 in AS 65002. Add one Linux host on each leaf, advertise each host subnet and loopback, and verify ECMP and host to host ping.

The agent designs the topology, writes the config commands and the FRR config per switch, deploys on containerlab and runs the checks. SONiC is preloaded on every lab VM, so there is no image to fetch.

Direct CLI is always available too. Open a terminal on any switch and run every command below by hand. To run the same fabric on your own Linux host, use the public lab:

git clone https://github.com/netpilot-labs/sonic-labs
cd sonic-labs/02-bgp-leaf-spine
sudo containerlab deploy -t bgp-leaf-spine.clab.yml

Run SONiC in containerlab covers the image download. Deploy returned in about 20 seconds and BGP was up about 60 seconds after it began.

Configure one leaf by hand

The lab does this at deploy from configs/leaf1.sh. These are the commands that script runs on leaf1, and each one works in a root shell on the switch. Ports first, in CONFIG_DB:

config loopback add Loopback0
config interface ip add Loopback0 10.255.0.11/32
config interface ip add Ethernet0 10.0.1.1/31
config interface startup Ethernet0
config interface ip add Ethernet4 10.0.3.1/31
config interface startup Ethernet4
config interface ip add Ethernet8 192.168.1.1/24
config interface startup Ethernet8

Then start bgpd. On a stock container vtysh otherwise answers bgpd is not running:

supervisorctl start bgpd

Then BGP, in FRR:

vtysh
configure terminal
router bgp 65001
 bgp router-id 10.255.0.11
 no bgp ebgp-requires-policy
 neighbor 10.0.1.0 remote-as 65000
 neighbor 10.0.3.0 remote-as 65000
 address-family ipv4 unicast
  network 10.255.0.11/32
  network 192.168.1.0/24
 exit-address-family
end

Set the router ID yourself. Left alone in the two switch test, FRR picked 172.20.20.2, the management address.

Verify BGP and ECMP

vtysh -c 'show bgp summary'

IPv4 Unicast Summary:
BGP router identifier 10.255.0.11, local AS number 65001 VRF default vrf-id 0
BGP table version 6
RIB entries 11, using 1408 bytes of memory
Peers 2, using 47 KiB of memory
 
Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.1.0        4      65000         6         6        6    0    0 00:00:17            3        6 N/A
10.0.3.0        4      65000         6         6        6    0    0 00:00:17            3        6 N/A
 
Total number of neighbors 2

Both spines send the same paths, and FRR installs both:

vtysh -c 'show ip route bgp'

IPv4 unicast VRF default:
B>* 10.255.0.1/32 [20/0] via 10.0.1.0, Ethernet0, weight 1, 00:00:16
B>* 10.255.0.2/32 [20/0] via 10.0.3.0, Ethernet4, weight 1, 00:00:16
B>* 10.255.0.12/32 [20/0] via 10.0.1.0, Ethernet0, weight 1, 00:00:16
  *                       via 10.0.3.0, Ethernet4, weight 1, 00:00:16
B>* 192.168.2.0/24 [20/0] via 10.0.1.0, Ethernet0, weight 1, 00:00:16
  *                       via 10.0.3.0, Ethernet4, weight 1, 00:00:16

From host1, the traceroute crosses one spine and arrives at host2:

traceroute to 192.168.2.10 (192.168.2.10), 30 hops max, 46 byte packets
 1  192.168.1.1  0.167 ms  0.089 ms  0.064 ms
 2  10.0.1.0  0.258 ms  0.224 ms  0.195 ms
 3  10.0.2.1  0.405 ms  0.337 ms  0.307 ms
 4  192.168.2.10  0.430 ms  0.424 ms  0.435 ms

Trap 1: sessions up, no routes

FRR on this image enforces RFC 8212. With bgp ebgp-requires-policy on, and no policy on the neighbors, the sessions come up and exchange nothing:

Neighbor        V         AS   MsgRcvd   MsgSent   TblVer  InQ OutQ  Up/Down State/PfxRcd   PfxSnt Desc
10.0.1.0        4      65000        11        10       10    0    0 00:00:10     (Policy) (Policy) N/A
10.0.3.0        4      65000        11        10       10    0    0 00:00:10     (Policy) (Policy) N/A

(Policy) is the clue. Either write inbound and outbound policy for each neighbor, or set no bgp ebgp-requires-policy under router bgp as this lab does. After clear bgp * the same line read 3 received and 6 sent.

Trap 2: redistribute connected leaks the management network

redistribute connected looks like the short way to advertise a loopback. It also advertises eth0. With it on leaf2, leaf1 learned the lab's management subnet over BGP:

Routing entry for 172.20.20.0/24
  Known via "bgp", distance 20, metric 0
  Last update 00:00:06 ago
    10.0.1.0, via Ethernet0, weight 1
    10.0.3.0, via Ethernet4, weight 1

Use network statements, or a route map on the redistribute. The lab uses network.

Trap 3: one link down, traffic gone

With ECMP you expect a link failure to cost nothing much. We pinged host2 from host1 every 200 ms and shut Ethernet0 on leaf1:

config interface shutdown Ethernet0

leaf1 dropped the session and moved to spine2 at once. The return traffic did not move. config interface shutdown takes the link down only on the port that runs it, so spine1 kept its port up, kept the session, and kept sending leaf1's traffic into a dead link. Replies stopped and had not come back when the test ended 14 seconds later. The BGP hold timer would have to expire first.

BFD fixes it. Turn it on at both ends of the link. On a stock container bfdd also needs starting, and it has no supervisor entry. NetPilot starts it for you:

/usr/lib/frr/bfdd -d -A 127.0.0.1

vtysh -c 'conf t' -c 'router bgp 65001' -c 'neighbor 10.0.1.0 bfd strict hold-time 1'

The matching line on spine1 uses AS 65000 and neighbor 10.0.1.1. Repeating the same test lost 10 pings, about 2 seconds, then traffic flowed through spine2. spine1 said why:

  Last reset 00:00:10,  Notification sent (Cease/BFD Down/Hard Reset)

A plain neighbor X bfd, without strict hold-time 1, keeps BGP up for 30 more seconds after BFD detects the loss. Our device guide records that from earlier runs on this image.

Save both halves

config save -y
vtysh -c write

Running command: /usr/local/bin/sonic-cfggen -d --print-data > /etc/sonic/config_db.json
Building Configuration...
Integrated configuration saved to /etc/frr/frr.conf
[OK]

The first file has the ports and the second has BGP. Read them back and you see each half in its own place. On this container image, a saved file is a record, not a reboot path: docker restart loses the node's links and config reload fails. That is why the public lab keeps its config in scripts and applies them at deploy.

FAQ

Does SONiC use FRR?

Yes. FRR is SONiC's routing stack. The SONiC wiki lists a Bgp container that runs the routing stack, and on the 202511 docker-sonic-vs image the version is FRRouting 10.4.1. You configure BGP in FRR through vtysh.

Why does my SONiC BGP session show Established with no prefixes?

FRR enforces RFC 8212 on this image. The summary shows (Policy) instead of a prefix count until each eBGP neighbor has an inbound and outbound policy, or you set no bgp ebgp-requires-policy under router bgp. In the test, leaf1 went from (Policy) to 3 prefixes received after that one line and a clear.

Why does show ip bgp summary fail on SONiC?

On the docker-sonic-vs image, show ip bgp summary answers No such command 'bgp', on a plain host and on NetPilot. Use vtysh -c 'show bgp summary', which is where BGP state lives.

How do I save BGP config on SONiC?

Run vtysh -c write, which saves FRR's config to /etc/frr/frr.conf. config save -y writes CONFIG_DB to /etc/sonic/config_db.json and does not include BGP on this setup. Run both when a change touched both.

How fast does BGP fail over on a SONiC virtual switch?

With neighbor bfd strict hold-time 1 on both ends, host traffic lost about 2 seconds, 10 pings at a 200 ms interval. Without BFD, the far end kept its session up and return traffic stayed blackholed for the whole 14 second test.

Next steps

• SONiC NOS lab online: the hosted route, built from a prompt
• SONiC architecture on a running switch: follow a route from FRR to the ASIC database
• Learn VXLAN EVPN on SONiC: add an overlay to this underlay
• FRRouting cloud labs: the same routing stack, standalone