Cisco Nexus(NXOS) : Seamless handoff (Interworking) between BGP EVPN based VxLAN fabric and SR-MPLS network.

Introduction

Segment Routing has now gained significant traction, with many greenfield ISP core deployments leaning towards SR-MPLS or even SRv6. While traditional MPLS networks are still in operation, data centers can no longer avoid interfacing with SR-MPLS-based transport networks. One of the key advantages of adopting an SR-MPLS handoff — instead of a pure Inter-AS Option A model — is operational efficiency: we avoid the need to establish and maintain a separate BGP session for every VRF that requires extension, significantly simplifying the control-plane scaling.

I have attempted a lab to demonstrate SRMPLS type handoff with Cisco Nexus based data center running Vxlan with BGP EVPN control plane.

This lab was done in CML (Cisco Modelling lab) . The Nexus 9000 device used here is running 9.3(6) nxos release.

Other devices used are IOS-XE as P1 and P2 device, PE1 is the IOS-XR router.

Topology

Below topology shows the traffic flow between the DC (green) and Transport(Brown) area. I have kept two BPE(Border Provider Edge) but for lab I have shown the configuration of only one BPE.

In this setup I am using BGP LU (labeled unicast) to advertise the labels between the different domains. You can also use OSPF or ISIS for the same purpose.

Note : Host2 is a router and has two interfaces in two different vrf, the global vrf interface is simulating host in EVPN environment and the gig0/1 interface is in vrf Tenant1 is simulating host beyond PE1.

Configurations

Border PE(NXOSv)


install feature-set mpls
feature-set mpls

nv overlay evpn
feature ospf
feature bgp
feature pim
feature mpls l3vpn
feature mpls segment-routing

feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay
!
fabric forwarding anycast-gateway-mac 0000.1234.5678
ip pim rp-address 10.254.245.2 group-list 239.1.1.0/25
ip pim rp-address 10.254.245.2 group-list 239.1.2.0/25
ip pim ssm range 232.0.0.0/8
mpls label range 16000 25000
vlan 1,10,50,500
!
segment-routing
mpls
connected-prefix-sid-map
address-family ipv4
10.10.10.10/32 index 10

vlan 10
vn-segment 100010
vlan 50
vn-segment 100050
vlan 500
vn-segment 50000
!
route-map REDIST permit 10
match tag 12345
vrf context TENANT1
vni 50000
ip pim ssm range 232.0.0.0/8
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
route-target import 50000:50000
route-target export 50000:50000

!
interface Vlan10
no shutdown
mtu 9150
vrf member TENANT1
ip address 192.168.10.254/24 tag 12345
ip pim sparse-mode
fabric forwarding mode anycast-gateway
!
interface Vlan50
no shutdown
mtu 9150
vrf member TENANT1
ip address 192.168.50.254/24 tag 12345
ip pim sparse-mode
fabric forwarding mode anycast-gateway
!
interface Vlan500
no shutdown
vrf member TENANT1
no ip redirects
ip forward
!
interface nve1
no shutdown
host-reachability protocol bgp
source-interface loopback1
member vni 50000 associate-vrf
member vni 100010
mcast-group 239.1.1.10
member vni 100050
mcast-group 239.1.1.50
!
interface Ethernet1/1
mtu 9150
ip address 10.11.18.2/30
ip ospf cost 4
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
no shutdown
!
interface Ethernet1/2
ip address 21.1.1.1/30
mpls ip forwarding
no shutdown
!
interface Ethernet1/3
switchport
switchport mode trunk
no shutdown
!
interface loopback0
ip address 10.10.10.10/32
ip router ospf UNDERLAY area 0.0.0.0
ip pim sparse-mode
!
interface loopback1
description NVE-LOOPBACK
ip address 10.10.10.11/32
ip router ospf UNDERLAY area 0.0.0.0
!
interface loopback100
vrf member TENANT1
ip address 100.1.1.2/32 tag 12345
!
router ospf UNDERLAY
router bgp 65100
router-id 10.10.10.10
address-family ipv4 unicast
network 10.10.10.10/32
allocate-label all
neighbor 7.7.7.7
remote-as 65100
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 12.12.12.12
remote-as 65200
update-source loopback0
ebgp-multihop 5
address-family vpnv4 unicast
send-community
send-community extended
import l2vpn evpn reoriginate
address-family vpnv6 unicast
send-community
send-community extended
import l2vpn evpn reoriginate
neighbor 21.1.1.2
remote-as 65200
update-source Ethernet1/2
address-family ipv4 labeled-unicast
send-community
send-community extended
vrf TENANT1
address-family ipv4 unicast
redistribute direct route-map REDIST

!

P Router(XE Virtual router)

!
interface Loopback0
ip address 12.12.12.12 255.255.255.255
!
interface GigabitEthernet1
ip address 21.1.1.2 255.255.255.252
negotiation auto
mpls bgp forwarding
no mop enabled
no mop sysid
!
interface GigabitEthernet2
ip address 21.1.1.6 255.255.255.252
negotiation auto
mpls bgp forwarding
no mop enabled
no mop sysid
!
interface GigabitEthernet3
ip address 22.1.1.1 255.255.255.252
negotiation auto
no mop enabled
no mop sysid
!
!
segment-routing mpls
!
connected-prefix-sid-map
address-family ipv4
12.12.12.12/32 index 1 range 1
exit-address-family
!
!
router ospf 1
router-id 12.12.12.12
segment-routing area 0 mpls
segment-routing mpls
network 12.12.12.12 0.0.0.0 area 0
network 21.1.1.2 0.0.0.0 area 0
network 22.1.1.1 0.0.0.0 area 0
!
router bgp 65200
bgp router-id 12.12.12.12
bgp log-neighbor-changes
no bgp default route-target filter
neighbor 10.10.10.10 remote-as 65100
neighbor 10.10.10.10 ebgp-multihop 5
neighbor 10.10.10.10 update-source Loopback0
neighbor 11.11.11.11 remote-as 65100
neighbor 11.11.11.11 ebgp-multihop 5
neighbor 11.11.11.11 update-source Loopback0
neighbor 14.14.14.14 remote-as 65200
neighbor 14.14.14.14 update-source Loopback0
neighbor 21.1.1.1 remote-as 65100
neighbor 21.1.1.1 update-source GigabitEthernet1
neighbor 21.1.1.5 remote-as 65100
neighbor 21.1.1.5 update-source GigabitEthernet2
neighbor 22.1.1.6 remote-as 65200
neighbor 22.1.1.6 update-source GigabitEthernet3
!
address-family ipv4
network 12.12.12.12 mask 255.255.255.255
neighbor 10.10.10.10 activate
neighbor 10.10.10.10 soft-reconfiguration inbound
neighbor 11.11.11.11 activate
neighbor 14.14.14.14 activate
neighbor 21.1.1.1 activate
neighbor 21.1.1.1 send-community both
neighbor 21.1.1.1 send-label
neighbor 21.1.1.5 activate
neighbor 21.1.1.5 send-community both
neighbor 21.1.1.5 send-label
neighbor 22.1.1.6 activate
neighbor 22.1.1.6 next-hop-self
neighbor 22.1.1.6 send-label
exit-address-family
!
address-family vpnv4
neighbor 10.10.10.10 activate
neighbor 10.10.10.10 send-community extended
neighbor 11.11.11.11 activate
neighbor 11.11.11.11 send-community extended
neighbor 14.14.14.14 activate
neighbor 14.14.14.14 send-community extended
neighbor 14.14.14.14 route-reflector-client
neighbor 14.14.14.14 next-hop-self
exit-address-family
!

PE Router(XR 9000v Router)

!
vrf TENANT1
address-family ipv4 unicast
import route-target
50000:50000
!
export route-target
50000:50000
!
!
!
!
interface Loopback0
ipv4 address 14.14.14.14 255.255.255.255
!
interface Loopback100
vrf TENANT1
ipv4 address 100.1.1.1 255.255.255.255
!
!
interface GigabitEthernet0/0/0/0
ipv4 address 22.1.1.6 255.255.255.252
!
interface GigabitEthernet0/0/0/1
vrf TENANT1
ipv4 address 192.167.1.1 255.255.255.0
!
!
route-policy PASS_ALL
pass
end-policy
!
router ospf CORE
router-id 14.14.14.14
segment-routing mpls
segment-routing sr-prefer
address-family ipv4 unicast
area 0
segment-routing mpls
interface Loopback0
prefix-sid index 3
!
interface GigabitEthernet0/0/0/0
!
!
!
router bgp 65200
bgp router-id 14.14.14.14
address-family ipv4 unicast
network 14.14.14.14/32
allocate-label all
!
address-family vpnv4 unicast
!
neighbor 22.1.1.1
remote-as 65200
address-family ipv4 labeled-unicast
route-policy PASS_ALL in
route-policy PASS_ALL out
!
!
neighbor 12.12.12.12
remote-as 65200
update-source Loopback0
address-family vpnv4 unicast
route-policy PASS_ALL in
route-policy PASS_ALL out
!
!
vrf TENANT1
rd auto
address-family ipv4 unicast
redistribute connected
redistribute static
!
!
!
segment-routing
global-block 16000 23999
!
end

Verification

Here is the overall path taken by the packet, first the packet will take the segment routing path carrying SR and VPN labels then it enters the DC with the VRF label 492288 advertised by BPE for the VRF.

Traceroute from the Host Connected to PE


BPE2# show bgp vpnv4 unicast detail vrf TENANT1
BGP routing table information for VRF default, address family VPNv4 Unicast
Route Distinguisher: 11.11.11.11:3 (VRF TENANT1)
BGP routing table entry for 100.1.1.1/32, version 13
Paths: (2 available, best #2)
Flags: (0x800c021a) (high32 0x000020) on xmit-list, is in urib, is best urib route, is in HW, exported, has label
vpn: version 29, (0x00000000100002) on xmit-list
local label: 492288

Host#traceroute vrf Cust1 192.168.10.15
Type escape sequence to abort.
Tracing the route to 192.168.10.15
VRF info: (vrf in name/id, vrf out name/id)
1 192.167.1.1 4 msec 3 msec 3 msec
2 22.1.1.5 [MPLS: Labels 16001/22 Exp 0] 5 msec 5 msec 5 msec
3 22.1.1.1 [MPLS: Label 22 Exp 0] 5 msec 5 msec 7 msec
4 21.1.1.5 [MPLS: Label 492288 Exp 0] 5 msec 6 msec 6 msec
5 192.168.10.254 12 msec 10 msec 10 msec
6 192.168.10.15 12 msec * 14 msec

RP/0/RP0/CPU0:PE1#show bgp vrf TENANT1 192.168.10.0/24 detail
Tue Aug 12 16:14:55.496 UTC
BGP routing table entry for 192.168.10.0/24, Route Distinguisher: 14.14.14.14:0
Versions:
Process bRIB/RIB SendTblVer
Speaker 38 38
Flags: 0x00043001+0x00010000;
Last Modified: Aug 12 15:03:16.227 for 01:11:39
Paths: (2 available, best #2)
Not advertised to any peer
Path #1: Received by speaker 0
Flags: 0x4000000000020005, import: 0x80
Not advertised to any peer
65100
12.12.12.12 (metric 3) from 12.12.12.12 (12.12.12.12), if-handle 0x00000000
Received Label 17
Origin incomplete, metric 0, localpref 100, valid, internal, imported
Received Path ID 0, Local Path ID 0, version 0
Extended community: RT:50000:50000 RT:65100:50000
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 10.10.10.10:3
Path #2: Received by speaker 0
Flags: 0x4000000005060005, import: 0x80
Not advertised to any peer
65100
12.12.12.12 (metric 3) from 12.12.12.12 (12.12.12.12), if-handle 0x00000000
Received Label 22
Origin incomplete, metric 0, localpref 100, valid, internal, best, group-best, import-candidate, imported
Received Path ID 0, Local Path ID 1, version 38
Extended community: RT:50000:50000 RT:65100:50000
Source AFI: VPNv4 Unicast, Source VRF: default, Source Route Distinguisher: 11.11.11.11:3

P2#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 22.1.1.1-A 0 Gi3 22.1.1.1
17 Pop Label 22.1.1.6-A 0 Gi2 22.1.1.6
16001 Pop Label 12.12.12.12/32 424749 Gi3 22.1.1.1
16003 Pop Label 14.14.14.14/32 447236 Gi2 22.1.1.6


P1#show bgp vpnv4 unicast all 192.168.10.0/24
BGP routing table entry for 10.10.10.10:3:192.168.10.0/24, version 17
Paths: (1 available, best #1, no table)
Advertised to update-groups:
1 2
Refresh Epoch 1
65100
10.10.10.10 (via default) from 10.10.10.10 (10.10.10.10)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:50000:50000 RT:65100:50000
mpls labels in/out 17/492287
rx pathid: 0, tx pathid: 0x0
Updated on Aug 12 2025 15:03:03 UTC
BGP routing table entry for 11.11.11.11:3:192.168.10.0/24, version 19
Paths: (1 available, best #1, no table)
Advertised to update-groups:
1 2
Refresh Epoch 1
65100
11.11.11.11 (via default) from 11.11.11.11 (11.11.11.11)
Origin incomplete, metric 0, localpref 100, valid, external, best
Extended Community: RT:50000:50000 RT:65100:50000
mpls labels in/out 22/492288
rx pathid: 0, tx pathid: 0x0
Updated on Aug 12 2025 15:03:17 UTC

P1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 22.1.1.2-A 0 Gi3 22.1.1.2
17 492287 10.10.10.10:3:192.168.10.0/24 \
0 Gi1 21.1.1.1
18 24001 14.14.14.14:0:100.1.1.1/32 \
0 Gi3 22.1.1.2
19 24001 14.14.14.14:0:192.167.1.0/24 \
31116 Gi3 22.1.1.2
20 492287 10.10.10.10:3:100.1.1.2/32 \
0 Gi1 21.1.1.1
21 492287 10.10.10.10:3:192.168.50.0/24 \
0 Gi1 21.1.1.1
22 492288 11.11.11.11:3:192.168.10.0/24 \
7770 Gi2 21.1.1.5

For the configuration of all other devices in this topology and packet captures look at this git link.

https://github.com/Shambhu-Dev/Labs.git

Here is the sample packet of the ping when it passes through the SR network and when it passes through the Vxlan network

ICMP through the SR network
ICMP inside Vxlan network

Conclusion

An SR-MPLS handoff from the data center to the transport network provides a scalable and protocol-efficient way to extend reachability without resorting to per-VRF BGP sessions, as required in traditional Inter-AS Option A. By using BGP Labeled Unicast (LU) at the edge, the data center can exchange labeled transport routes with the provider, enabling seamless forwarding of traffic across independent IGP domains while preserving Segment Routing benefits. This approach simplifies interconnection, reduces control-plane overhead, and supports a future-ready migration path toward SRv6 when the core evolves.

Leave a Reply