Multi-Vendor SONiC Labs: Integrating with Cisco and Arista Without Hardware

SONiC adoption has moved from Microsoft Azure's private dialect to a mainstream NOS option, with the 650 Group projecting $8 billion in SONiC-related data-center switching revenue by 2027. But production SONiC deployments rarely exist in a SONiC-only fabric — they coexist with Cisco NX-OS leaves, Arista EOS spines, Nokia SR Linux aggregation, and commercial NOS tier-0 gear. Testing that interop without physical hardware has been the missing piece. Here is the multi-vendor SONiC lab pattern hyperscaler fabric engineering teams use to validate integration before deployment.

What is and is not covered by existing SONiC labs

The SONiC ecosystem has several existing lab environments, each with a specific focus:

The gap is clear: no existing lab combines SONiC with multiple commercial vendors in a single fabric, cloud-hosted, without DIY setup. That is the workflow this guide covers.

The multi-vendor SONiC fabric pattern

A representative hyperscaler integration scenario: SONiC leaves running on white-box hardware, connected to Cisco NX-OS or Arista EOS tier-1 spines, with traffic flowing across the vendor boundary over eBGP EVPN. The integration risks live at:

• eBGP EVPN session establishment across the vendor boundary (capability negotiation, keepalive timing)
• Route-target handling between SONiC's explicit RT configuration and commercial vendors' auto-derivation
• BUM traffic replication across SONiC VTEPs and commercial VTEPs
• ECMP hashing differences that lead to uneven utilization across parallel paths
• MAC mobility events when hosts move between SONiC and commercial leaves

Each one is a real source of integration issues. Each one is reproducible in a multi-vendor lab.

Building the lab in NetPilot

The prompt

Copy this into NetPilot (requires enterprise plan for SONiC):

Build a leaf-spine fabric with 2 Arista cEOS spines (spine1, spine2) and 3 leaves: 1 Cisco IOL leaf (leaf1), 1 SONiC-VS leaf (leaf2), and 1 Arista cEOS leaf (leaf3). All leaves run eBGP with the spines for the underlay (spines in AS 65000, each leaf in AS 65001-65003). Enable BGP EVPN address family between leaves and spines. Configure VLAN 100 mapped to VNI 10100 on all three leaves, subnet 192.168.100.0/24 with distributed anycast gateway 192.168.100.1. Place one host on each leaf. I want to verify: Type-2 EVPN route exchange works across vendor boundaries, host-to-host reachability between Cisco, SONiC, and Arista leaves, and MAC mobility when a host moves between leaves.

Deploy

Two minutes later the lab is live. You have SSH access to each leaf and spine.

Verify baseline

Ask the agent:

"Check EVPN state across all three leaves — SONiC, Cisco, and Arista. Show BGP EVPN sessions and whether Type-2 and Type-3 routes are exchanging across vendor boundaries."

The agent queries all three leaves in parallel — translating to each vendor's syntax (SONiC's vtysh-wrapped commands, Cisco's show bgp l2vpn evpn, Arista's show bgp evpn) — and returns a consolidated table. If any leaf's EVPN database is empty while its BGP session is up, the agent flags it: route-target mismatch is the most common cause.

Direct CLI is always available for per-device inspection:

# SONiC leaf (uses FRR vtysh)
vtysh
show bgp l2vpn evpn summary
show bgp l2vpn evpn
show evpn vni

show bgp l2vpn evpn summary
show bgp l2vpn evpn
show nve peers

show bgp evpn summary
show bgp evpn
show vxlan vtep

The three most common SONiC integration bugs

Bug 1: Route-target configuration mismatch

SONiC requires explicit route-target configuration — no auto-derivation. Arista EOS and Cisco NX-OS auto-derive RTs by default. The fix is to override auto-derivation on the commercial vendors to match SONiC's explicit RT, or configure SONiC with the derived RT value.

On SONiC:

{
  "VRF": {
    "TENANT1": {
      "vni": "10100",
      "rd": "100.100.100.2:100",
      "import_rt": "65001:10100",
      "export_rt": "65001:10100"
    }
  }
}

On the Cisco/Arista side, match the RT explicitly:

router bgp 65001
  vrf TENANT1
    route-target import 65001:10100
    route-target export 65001:10100

Bug 2: BUM replication mode mismatch

SONiC defaults to ingress replication (head-end replication). Some vendor defaults may use PIM-based replication, depending on version and platform. Unaligned modes cause BUM traffic to silently drop — unicast works, broadcast does not.

Explicitly set both sides to ingress replication:

# SONiC
config vxlan evpn_nvo add evpn_nvo vtep0
# (uses ingress replication by default)

# Arista
service multicast 
  no redundancy

Re-verify:

# Test BUM traffic - e.g., ARP
ping 192.168.100.255
# or test DHCP / mDNS / etc.

Bug 3: Anycast gateway MAC address mismatch

In a fabric with distributed anycast gateway (DAG), all leaves share the same virtual MAC for the gateway IP. If SONiC and the commercial vendors are not configured with the identical virtual MAC, hosts will ARP and get a different answer from different leaves — traffic gets blackholed on VRRP-like failover.

On SONiC:

{
  "VXLAN_TUNNEL_MAP": {
    "vtep0|map_10100_Vlan100": {
      "vlan": "Vlan100",
      "vni": "10100"
    }
  },
  "VLAN_INTERFACE": {
    "Vlan100": {
      "vrf_name": "TENANT1"
    },
    "Vlan100|192.168.100.1/24": {
      "gwip": "192.168.100.1"
    }
  }
}
# Plus anycast-gateway-mac config matching other leaves

On Cisco:

fabric forwarding anycast-gateway-mac 0000.0000.1234

The MAC address value must be byte-identical across all three leaves. Mismatches produce intermittent connectivity that looks like "random packet loss" from the application layer.

CI/CD integration

The reason fabric engineers adopt multi-vendor virtual labs is that they slot into CI/CD. NetPilot's REST API lets you:

1. Trigger a new lab from a git commit
2. Deploy candidate SONiC images against a fixed topology
3. Run validation scripts against the lab (check BGP state, test traffic flows, verify MAC learning)
4. Tear down when done
5. Fail the build if any check fails

This is standard NetDevOps but previously required hardware in the loop. Now the hardware is a container running in a managed cloud VM per build.

For the full NetDevOps integration patterns see the AI Network Emulator product page.

FAQ

Can NetPilot really run SONiC in a multi-vendor fabric?

Yes, on the enterprise plan. SONiC is supported via BYOI (bring your own image) — NetPilot's ContainerLab foundation supports any vendor image that runs in ContainerLab, and SONiC-VS (the virtual SONiC image) is compatible. Contact sales to add SONiC to your workspace.

How does SONiC integration testing in NetPilot compare to Microsoft's sonic-mgmt testbed?

The sonic-mgmt testbed is excellent for SONiC-only regression testing of the control plane. NetPilot is complementary — it adds the multi-vendor integration layer (Cisco, Juniper, Arista, Nokia alongside SONiC) and the cloud self-serve deployment model. Most teams run both: sonic-mgmt for upstream SONiC changes, NetPilot for integration tests against their specific vendor mix.

Do I need physical SONiC switches for meaningful integration testing?

For functional integration testing (BGP sessions, EVPN routes, MAC learning, BUM replication, ARP/DAG behavior), no — virtual SONiC in a container gives you the real control plane with real configuration. For performance benchmarking at line rate (100G/400G hardware forwarding performance), you still need physical switches. The division of labor: integration + functional testing in a virtual lab, performance testing on hardware.

Can I test SONiC upgrades before deploying to production?

Yes. Deploy candidate SONiC images in a NetPilot lab with topology matching your production fabric. Run your validation suite against the new image in isolation. When it passes, deploy to production with confidence. Enterprise plans support storing multiple SONiC image versions for upgrade testing.

What about Nokia SR Linux in the same fabric?

Fully supported. Nokia SR Linux is natively available on all NetPilot plans (no enterprise required). Add it to the prompt: "Include 1 Nokia SR Linux leaf for comparison with SONiC's feature set."

Copy-paste ready: The multi-vendor SONiC integration prompt is the template for the SONiC + Cisco + Arista scenario in this guide.

Running a hyperscaler fabric engineering team? The Network Research Lab hub covers the full enterprise workflow — contact sales for dedicated environments and SONiC BYOI support.