VXLAN Journey - Part 06 - VXLAN and vPC Leaf Switches
This is the last part before we scale up our topology with dedicated Spine, Leaf, Border Gateway and Border Leaf switches which represent - how VXLAN network is built in modern data center networks. In this blog - we will explore how to configure a pair of vPC switches for VXLAN/MPBGP-EVPN. We need to do some extra configurations in the switches because in vPC enabled switches, they each member switch run their own control plane (no shared control plane when we connect switches using stacking).
Network Topology
Our topology looks like below -
01 - Network Topology
Other network settings are -
BGP ASN - 64512
loopback0 - 10.81.0.X/32
loopback1 - 10.81.1.X/32
loopback1 secondary (vPC switches only) - 10.81.1.23/32
Anycast GW MAC 2020.0000.00aa
Vlan 100 - L3VNI 100000 - Vrf-Blue
Vlan 101 - L2VNI 100101 - 10.85.101.0/24
Vlan 102 - L2VNI 100102 - 10.85.102.0/24
vPC keep-alive - over mgmt0 interface - 10.10.11.X/24
vPC peer-link - port-channel 500 (e1/3 & e1/4)
System NVE Infra-Vlans - vlan 3600
vPC peer-link SVI for VXLAN - interface vlan 3600 - 10.81.2.0/30
Now we will start configuring the switches. Only new settings will be explained, for other settings check-out other blogs in the series.
Leaf-Sw-01 Configuration
hostname Leaf-Sw-01
!
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay
!
fabric forwarding anycast-gateway-mac 2020.0000.00aa
!
vlan 100
name Vrf-Blue_L3-Vlan
vn-segment 100000
vlan 101
name Vrf-Blue_Vlan-101_10.85.101.0/24
vn-segment 100101
!
route-map Rmap_Tag-12345 permit 10
description Used-To-Redistribute-Connected-Routes-In-Vrf
match tag 12345
!
vrf context Vrf-Blue
description Vrf-Blue
vni 100000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
interface Vlan100
description Vrf-Blue_L3-Interface
no shutdown
mtu 9216
vrf member Vrf-Blue
ip forward
!
interface Vlan101
description Vrf-Blue_Vlan-101_10.85.101.0/24_Anycast-Gw
no shutdown
mtu 9216
vrf member Vrf-Blue
ip address 10.85.101.1/24 tag 12345
fabric forwarding mode anycast-gateway
!
interface nve1
no shutdown
description Vtep-Vxlan
host-reachability protocol bgp
source-interface loopback1
member vni 100000 associate-vrf
member vni 100101
ingress-replication protocol bgp
!
interface Ethernet1/1
description To_Leaf-Sw-02_e1/1
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/2
description To_Leaf-Sw-02_e1/2
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/3
description To_Leaf-Sw-03_e1/1
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/4
description To_Leaf-Sw-04_e1/2
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/15
description To_Vlan-101-Pc-01_eth0
switchport access vlan 101
spanning-tree port type edge
mtu 9216
!
interface loopback0
description Underlay-Routing-Loopback
ip address 10.81.0.1/32
ip router ospf UNDERLAY area 0.0.0.0
!
interface loopback1
description Overlay-Vtep-Loopback
ip address 10.81.1.1/32
ip router ospf UNDERLAY area 0.0.0.0
!
router ospf UNDERLAY
router-id 10.81.0.1
log-adjacency-changes detail
!
router bgp 64512
router-id 10.81.0.1
log-neighbor-changes
!!! We do not have Spine or ibgp route-reflector switches.
!!! We need ibgp full mesh neighborship.
neighbor 10.81.0.2
remote-as 64512
description Leaf-Sw-02
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.81.0.3
remote-as 64512
description Leaf-Sw-03
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
vrf Vrf-Blue
address-family ipv4 unicast
redistribute direct route-map Rmap_Tag-12345
maximum-paths ibgp 2
!
evpn
vni 100101 l2
rd auto
route-target import auto
route-target export auto
!
Leaf-Sw-02 Configuration
hostname Leaf-Sw-02
!
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay
!
!!! Enable necessary features related to vpc.
feature lacp
feature vpc
!
vrf context management
!
!!! mgmt0 will be used for vPC keepalive.
interface mgmt0
vrf member management
ip address 10.10.11.2/24
!
!!! vpc domain configuration.
vpc domain 1
!!! Lower priority switch becomes vPC primary switch.
role priority 10
!!! vPC keepalive is sent using vrf management and mgmt0 interface.
peer-keepalive destination 10.10.11.3 source 10.10.11.2
!!! Enable peer-switch, peer-router and layer3 peer-router enhancements.
peer-switch
peer-gateway
layer3 peer-router
!!! vPC port-channel interface bring up will be delayed for 150 seconds.
!!! This is to allow layer 3 routing protocols to converge before allowing traffic on server facing vPC port-channel interfaces.
delay restore 150
!!! During vPC recovery if only secondary switch comes back online, all vPC port-channel interfaces will be permanently down, until primary switch comes back online.
!!! With auto-recovery; instead of permanently disabling all vPC port-channels and wait for primary switch to be online; secondary switch will enable it's vPC port-channels after waiting for specified time in seconds.
auto-recovery reload-delay 360
!!! Synchronize ARP information with vPC member switches.
ip arp synchronize
!
interface Ethernet1/3
description Po500_To_Leaf-Sw-03_e1/3
switchport mode trunk
channel-group 500 mode active
!
interface Ethernet1/4
description Po500_To_Leaf-Sw-03_e1/4
switchport mode trunk
channel-group 500 mode active
!
!!! vPC peer-link is a port-channel interface using e1/3 and e1/4 physical interfaces.
interface port-channel500
description Po500_Vpc_Peer-Link
!!! Mode must be trunk which allows all the vlans.
switchport mode trunk
!!! Spanning-tree port type must be network.
spanning-tree port type network
!!! Use this port-channel interface as peer-link.
vpc peer-link
!
!!! We need a SVI over peer-link to transport VXLAN encapsulated traffic for failover scenarios.
vlan 3600
name Vpc_Peer_Link_Svi_Vlan
!
!!! Vlan reserved as infra-vlan; otherwise Cisco NXOS does not allow VXLAN encapsulated traffic over vlan interface/SVI.
system nve infra-vlans 3600
!
!!! The actual SVI configuration.
interface Vlan3600
description Vpc_Peer_Link_Svi
no shutdown
mtu 9216
no ip redirects
ip address 10.81.2.1/30
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
!
fabric forwarding anycast-gateway-mac 2020.0000.00aa
!
vlan 100
name Vrf-Blue_L3-Vlan
vn-segment 100000
vlan 101
name Vrf-Blue_Vlan-101_10.85.101.0/24
vn-segment 100101
!!! Even though vlan 102 is only available on Leaf-Sw-03, it must be defined in both vPC member switch.
vlan 102
name Vrf-Blue_Vlan-102_10.85.102.0/24
vn-segment 100102
!
route-map Rmap_Tag-12345 permit 10
description Used-To-Redistribute-Connected-Routes-In-Vrf
match tag 12345
!
vrf context Vrf-Blue
description Vrf-Blue
vni 100000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
interface Vlan100
description Vrf-Blue_L3-Interface
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip forward
!
interface Vlan101
description Vrf-Blue_Vlan-101_10.85.101.0/24_Anycast-Gw
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip address 10.85.101.1/24 tag 12345
fabric forwarding mode anycast-gateway
!
!!! Even though vlan 102 is only available on Leaf-Sw-03, it must be defined in both vPC member switch.
interface Vlan102
description Vrf-Blue_Vlan-102_10.85.102.0/24_Anycast-Gw
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip address 10.85.102.1/24 tag 12345
fabric forwarding mode anycast-gateway
!
interface loopback0
description Underlay-Routing-Loopback
ip address 10.81.0.2/32
ip router ospf UNDERLAY area 0.0.0.0
interface loopback1
description Overlay-Vtep-Loopback
ip address 10.81.1.2/32
!!! loopback 1 will have a secondary IP address.
!!! Secondary IP address will be the same in both vPC switches.
ip address 10.81.1.23/32 secondary
ip router ospf UNDERLAY area 0.0.0.0
!
interface nve1
no shutdown
description Vtep-Vxlan
host-reachability protocol bgp
!!! Advertising virtual-rmac is a must for vPC switches.
!!! rt-2 routes (both mac and mac+ip) will be advertised loopback 1 secondary shared IP address as bgp-evpn next-hop.
!!! RMAC for those rt-2 routes will be virtual-rmac created from secondary IP address.
advertise virtual-rmac
source-interface loopback1
member vni 100000 associate-vrf
member vni 100101
ingress-replication protocol bgp
!!! Even though vlan 102/l2vni 100102 is only available on Leaf-Sw-03, it must be defined in both vPC member switch.
member vni 100102
ingress-replication protocol bgp
!
interface Ethernet1/1
description To_Leaf-Sw-01_e1/1
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/2
description To_Leaf-Sw-01_e1/2
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
router ospf UNDERLAY
router-id 10.81.0.2
log-adjacency-changes detail
!
router bgp 64512
router-id 10.81.0.2
log-neighbor-changes
address-family l2vpn evpn
!!! advertise-pip is a must for vPC switches.
!!! rt-5 routes (subnet-prefix) will be announced using loopback1's primary IP address as bgp-evpn next-hop.
!!! The rmac for rt-5 routes will be vPC switches individual local router mac.
advertise-pip
!!! We do not have Spine or ibgp route-reflector switches.
!!! We need ibgp full mesh neighborship; even with vpc member switches.
neighbor 10.81.0.1
remote-as 64512
description Leaf-Sw-01
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.81.0.3
remote-as 64512
description Leaf-Sw-03
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
vrf Vrf-Blue
address-family ipv4 unicast
redistribute direct route-map Rmap_Tag-12345
maximum-paths ibgp 2
evpn
vni 100101 l2
rd auto
route-target import auto
route-target export auto
!!! Even though vlan 102/l2vni 100102 is only available on Leaf-Sw-03, it must be defined in both vPC member switch.
vni 100102 l2
rd auto
route-target import auto
route-target export auto
!
!!! Vlan-101-Pc-02 is connected with both switches e1/15 interfaces.
interface Ethernet1/15
description Po15_Vpc15_To_Vlan-101-Pc-02_eth0
switchport access vlan 101
mtu 9216
channel-group 15 mode active
!
!!! Creating a vpc port-channel interface for Vlan-101-Pc-02.
interface port-channel15
description Po15_Vpc15_To_Vlan-101-Pc-02_bond0
switchport access vlan 101
spanning-tree port type edge
mtu 9216
vpc 15
!
Leaf-Sw-03 Configuration
hostname Leaf-Sw-03
!
nv overlay evpn
feature ospf
feature bgp
feature interface-vlan
feature vn-segment-vlan-based
feature lacp
feature vpc
feature nv overlay
!
fabric forwarding anycast-gateway-mac 2020.0000.00aa
!
vlan 100
name Vrf-Blue_L3-Vlan
vn-segment 100000
vlan 101
name Vrf-Blue_Vlan-101_10.85.101.0/24
vn-segment 100101
vlan 102
name Vrf-Blue_Vlan-102_10.85.102.0/24
vn-segment 100102
vlan 3600
name Vpc_Peer_Link_Svi_Vlan
!
route-map Rmap_Tag-12345 permit 10
description Used-To-Redistribute-Connected-Routes-In-Vrf
match tag 12345
!
vrf context Vrf-Blue
description Vrf-Blue
vni 100000
rd auto
address-family ipv4 unicast
route-target both auto
route-target both auto evpn
!
vrf context management
!
system nve infra-vlans 3600
!
vpc domain 1
peer-switch
role priority 20
peer-keepalive destination 10.10.11.2 source 10.10.11.3
delay restore 150
peer-gateway
layer3 peer-router
auto-recovery reload-delay 360
ip arp synchronize
!
interface Vlan100
description Vrf-Blue_L3-Interface
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip forward
!
interface Vlan101
description Vrf-Blue_Vlan-101_10.85.101.0/24_Anycast-Gw
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip address 10.85.101.1/24 tag 12345
fabric forwarding mode anycast-gateway
!
interface Vlan102
description Vrf-Blue_Vlan-102_10.85.102.0/24_Anycast-Gw
no shutdown
mtu 9216
vrf member Vrf-Blue
no ip redirects
ip address 10.85.102.1/24 tag 12345
fabric forwarding mode anycast-gateway
!
interface Vlan3600
description Vpc_Peer_Link_Svi
no shutdown
mtu 9216
no ip redirects
ip address 10.81.2.2/30
no ipv6 redirects
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
!
interface port-channel15
description Po15_Vpc15_To_Vlan-101-Pc-02_bond0
switchport access vlan 101
spanning-tree port type edge
mtu 9216
vpc 15
!
interface port-channel500
description Po500_Vpc_Peer-Link
switchport mode trunk
spanning-tree port type network
vpc peer-link
!
interface nve1
no shutdown
description Vtep-Vxlan
host-reachability protocol bgp
advertise virtual-rmac
source-interface loopback1
member vni 100000 associate-vrf
member vni 100101
ingress-replication protocol bgp
member vni 100102
ingress-replication protocol bgp
!
interface Ethernet1/1
description To_Leaf-Sw-01_e1/3
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/2
description To_Leaf-Sw-01_e1/4
no switchport
mtu 9216
medium p2p
no ip redirects
ip unnumbered loopback0
ip ospf network point-to-point
ip router ospf UNDERLAY area 0.0.0.0
no shutdown
!
interface Ethernet1/3
description Po500_To_Leaf-Sw-02_e1/3
switchport mode trunk
channel-group 500 mode active
!
interface Ethernet1/4
description Po500_To_Leaf-Sw-02_e1/4
switchport mode trunk
channel-group 500 mode active
!
!!! Vlan-102-Pc-01 is connected with Leaf-Sw-03 only using a single physical interface e1/14.
interface Ethernet1/14
description To_Vlan-102-Pc-01_eth0
switchport access vlan 102
spanning-tree port type edge
mtu 9216
!!! If the switch is vPC secondary switch, orphan ports will be suspended until peer-link between vPC members is restored.
!!! When a host is connected with only with one of the vPC switch (single-homed).
vpc orphan-port suspend
!
interface Ethernet1/15
description Po15_Vpc15_To_Vlan-101-Pc-02_eth1
switchport access vlan 101
mtu 9216
channel-group 15 mode active
!
interface mgmt0
vrf member management
ip address 10.10.11.3/24
!
interface loopback0
description Underlay-Routing-Loopback
ip address 10.81.0.3/32
ip router ospf UNDERLAY area 0.0.0.0
!
interface loopback1
description Overlay-Vtep-Loopback
ip address 10.81.1.3/32
ip address 10.81.1.23/32 secondary
ip router ospf UNDERLAY area 0.0.0.0
!
router ospf UNDERLAY
router-id 10.81.0.3
log-adjacency-changes detail
!
router bgp 64512
router-id 10.81.0.3
log-neighbor-changes
address-family l2vpn evpn
advertise-pip
neighbor 10.81.0.1
remote-as 64512
description Leaf-Sw-01
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
neighbor 10.81.0.2
remote-as 64512
description Leaf-Sw-02
update-source loopback0
address-family l2vpn evpn
send-community
send-community extended
vrf Vrf-Blue
address-family ipv4 unicast
redistribute direct route-map Rmap_Tag-12345
maximum-paths ibgp 2
evpn
vni 100101 l2
rd auto
route-target import auto
route-target export auto
vni 100102 l2
rd auto
route-target import auto
route-target export auto
!
Verification
Firstly we will verify our vPC configuration.
Leaf-Sw-03# show vpc
Legend:
(*) - local vPC is down, forwarding via vPC peer-link
vPC domain id : 1
Peer status : peer adjacency formed ok
vPC keep-alive status : peer is alive
Configuration consistency status : success
Per-vlan consistency status : success
Type-2 consistency status : success
vPC role : secondary
Number of vPCs configured : 1
Peer Gateway : Enabled
Dual-active excluded VLANs : -
Graceful Consistency Check : Enabled
!!! Auto-recovery and Delay-restore timers.
Auto-recovery status : Enabled, timer is off.(timeout = 360s)
Delay-restore status : Timer is off.(timeout = 150s)
Delay-restore SVI status : Timer is off.(timeout = 10s)
Delay-restore Orphan-port status : Timer is off.(timeout = 0s)
Operational Layer3 Peer-router : Enabled
Virtual-peerlink mode : Disabled
vPC Peer-link status
---------------------------------------------------------------------
id Port Status Active vlans
-- ---- ------ -------------------------------------------------
!!! vpc peer-link port-channel
1 Po500 up 1,100-102,3600
vPC status
----------------------------------------------------------------------------
Id Port Status Consistency Reason Active vlans
-- ------------ ------ ----------- ------ ---------------
!!! vpc port-channel for Vlan-101-Pc-02.
15 Po15 up success success 101
Please check "show vpc consistency-parameters vpc <vpc-num>" for the
consistency reason of down vpc and for type-2 consistency reasons for
any vpc.
Leaf-Sw-03# show vpc role
vPC Role status
----------------------------------------------------
vPC role : secondary
Dual Active Detection Status : 2
vPC system-mac : 00:23:04:ee:be:01
vPC system-priority : 32667
vPC local system-mac : 50:00:20:00:1b:08
!!! Local switch vPC priority.
vPC local role-priority : 20
vPC local config role-priority : 20
vPC local Sticky-Bit : FALSE
vPC peer system-mac : 50:00:1f:00:1b:08
!!! Remote switch (peer) vPC priority. This is primary switch.
vPC peer role-priority : 10
vPC peer config role-priority : 10
Leaf-Sw-03# show vpc peer-keepalive
vPC keep-alive status : peer is alive
--Peer is alive for : (1203) seconds, (938) msec
--Send status : Success
--Last send at : 2026.09.28 18:11:00 860 ms
!!! vPC keep alive is sent using mgmt0 interface.
--Sent on interface : mgmt0
--Receive status : Success
--Last receive at : 2026.09.28 18:11:00 861 ms
--Received on interface : mgmt0
--Last update from peer : (0) seconds, (995) msec
vPC Keep-alive parameters
--Destination : 10.10.11.2
--Keepalive interval : 1000 msec
--Keepalive timeout : 5 seconds
--Keepalive hold timeout : 3 seconds
!!! Vrf management used for vPC keep alive.
--Keepalive vrf : management
--Keepalive udp port : 3200
--Keepalive tos : 192
Now we will look at our infra-vlan which is required when VXLAN encapsulated traffic passes through vlan interface (SVI).
Leaf-Sw-03# show system nve infra-vlans
Currently active infra Vlans: 3600
Currently active infra Vlans: 3600
Both the vPC switches are using the same virtual rmac - it will be the same in both Leaf-Sw-02 and Leaf-Sw-03. The virtual rmac is derived from the shared secondary IP address - 10.81.1.23. The virtual rmac will be - 0200+10.81.1.23 (in hex) = 0200.0a51.0117.
Leaf-Sw-02# show nve interface nve 1 detail
Interface: nve1, State: Up, encapsulation: VXLAN
VPC Capability: VPC-VIP-Only [notified]
!!! Unique Local rmac.
Local Router MAC: 5000.1f00.1b08
Host Learning Mode: Control-Plane
Source-Interface: loopback1 (primary: 10.81.1.2, secondary: 10.81.1.23)
Source Interface State: Up
Virtual RMAC Advertisement: Yes
NVE Flags:
Interface Handle: 0x49000001
Source Interface hold-down-time: 180
Source Interface hold-up-time: 30
Remaining hold-down time: 0 seconds
!!! Shared virtual rmac derived from shared secondary ip address 10.81.1.23.
Virtual Router MAC: 0200.0a51.0117
Interface state: nve-intf-add-complete
Fabric convergence time: 135 seconds
Fabric convergence time left: 0 seconds
Leaf-Sw-03# show nve interface nve 1 detail
Interface: nve1, State: Up, encapsulation: VXLAN
VPC Capability: VPC-VIP-Only [notified]
!!! Unique Local rmac.
Local Router MAC: 5000.2000.1b08
Host Learning Mode: Control-Plane
Source-Interface: loopback1 (primary: 10.81.1.3, secondary: 10.81.1.23)
Source Interface State: Up
Virtual RMAC Advertisement: Yes
NVE Flags:
Interface Handle: 0x49000001
Source Interface hold-down-time: 180
Source Interface hold-up-time: 30
Remaining hold-down time: 0 seconds
!!! Shared virtual rmac derived from shared secondary ip address 10.81.1.23.
Virtual Router MAC: 0200.0a51.0117
Interface state: nve-intf-add-complete
Fabric convergence time: 135 seconds
Fabric convergence time left: 0 seconds
We are using advertise-pip and advertise virtual-rmac. The following will happen to rt-2 and rt-5 evpn routes -
- rt-2 (mac) - bgp evpn next-hop will be the shared secondary ip address (10.81.1.23) configured on loopback1 interface. rt-2 (mac) routes do not have rmac.
- rt-2 (mac+ip) - bgp evpn next-hop will be the shared secondary ip address (10.81.1.23) configured on loopback1 interface. And router-mac will be the virtual-rmac (0200.0a51.0117).
- rt-5 (prefix) - bgp evpn next-hop will be the primary ip address (10.81.1.2 and 10.81.1.3) configured on loopback1 interface. And router-mac will be the local-rmac (5000.1f00.1b08 and 5000.2000.1b08) of the switches.
Let's verify them one by one from Leaf-Sw-01.
Vlan-101-Pc-02 has ip/mac - 10.85.101.12/503e.8b00.2300.
Leaf-Sw-01# show bgp l2vpn evpn 503e.8b00.2300
BGP routing table information for VRF default, address family L2VPN EVPN
!!! EVPN rt-2 (mac) route received from Leaf-Sw-02.
Route Distinguisher: 10.81.0.2:32868
!!! 503e.8b00.2300 mac-address of Vlan-101-Pc-02.
BGP routing table entry for [2]:[0]:[0]:[48]:[503e.8b00.2300]:[0]:[0.0.0.0]/216, version 852
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 1 destination(s)
Imported paths list: L2-100101
AS-Path: NONE, path sourced internal to AS
!!! bgp-evpn next-hop is 10.81.1.23 secondary ip address configured on loop1.
10.81.1.23 (metric 41) from 10.81.0.2 (10.81.0.2)
Origin IGP, MED not set, localpref 100, weight 0
Received label 100101
Extcommunity: RT:64512:100101 SOO:10.81.1.23:0 ENCAP:8
Path-id 1 not advertised to any peer
!!! EVPN rt-2 (mac) route received from Leaf-Sw-03.
Route Distinguisher: 10.81.0.3:32868
!!! 503e.8b00.2300 mac-address of Vlan-101-Pc-02.
BGP routing table entry for [2]:[0]:[0]:[48]:[503e.8b00.2300]:[0]:[0.0.0.0]/216, version 854
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 1 destination(s)
Imported paths list: L2-100101
AS-Path: NONE, path sourced internal to AS
!!! bgp-evpn next-hop is 10.81.1.23 secondary ip address configured on loop1.
10.81.1.23 (metric 41) from 10.81.0.3 (10.81.0.3)
Origin IGP, MED not set, localpref 100, weight 0
Received label 100101
Extcommunity: RT:64512:100101 SOO:10.81.1.23:0 ENCAP:8
Path-id 1 not advertised to any peer
As this is a rt-2 (mac) route, it will be programmed into the switch's mac address-table.
Leaf-Sw-01# show mac address-table vlan 101
Legend:
* - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
age - seconds since last seen,+ - primary entry using vPC Peer-Link,
(T) - True, (F) - False, C - ControlPlane MAC, ~ - vsan,
(NA)- Not Applicable A - ESI Active Path, S - ESI Standby Path
TL - True Learned, PS - Peer Sync, RO - Re-originate
VLAN MAC Address Type age Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
!!! 503e.8b00.2300 is learnt from nve peer 10.81.1.23.
C 101 503e.8b00.2300 dynamic NA F F nve1(10.81.1.23)
* 101 509f.d200.2100 dynamic NA F F Eth1/15
G 101 5000.1e00.1b08 static - F F sup-eth1(R)
Leaf-Sw-01 will have 3-nve peers.
Leaf-Sw-01# show nve peers
Interface Peer-IP State LearnType Uptime Router-Mac
--------- -------------------------------------- ----- --------- -------- -----------------
!!! Leaf-Sw-02 is peer.
nve1 10.81.1.2 Up CP 01:36:34 5000.1f00.1b08
!!! Leaf-Sw-03 is peer.
nve1 10.81.1.3 Up CP 01:36:35 5000.2000.1b08
!!! 10.81.1.23 is a peer shared by both Leaf-Sw-02 and Leaf-Sw-03.
nve1 10.81.1.23 Up CP 01:36:35 0200.0a51.0117
The rt-2 (mac+ip) will behave exactly same. The only difference will be it will carry one extra bgp extended community that will be rmac. And the rmac will be 0200.0a51.0117 (derived from 0200+shared ip 10.81.1.23 in hex).
Leaf-Sw-01# show bgp l2vpn evpn 10.85.101.12
BGP routing table information for VRF default, address family L2VPN EVPN
!!! EVPN rt-2 (mac+ip) route received from Leaf-Sw-02.
Route Distinguisher: 10.81.0.2:32868
!!! Vlan-101-Pc-02 IP/mac - 10.85.101.12/503e.8b00.2300.
BGP routing table entry for [2]:[0]:[0]:[48]:[503e.8b00.2300]:[32]:[10.85.101.12]/272, version 860
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 3 destination(s)
Imported paths list: Vrf-Blue L2-100101 L3-100000
AS-Path: NONE, path sourced internal to AS
10.81.1.23 (metric 41) from 10.81.0.2 (10.81.0.2)
Origin IGP, MED not set, localpref 100, weight 0
Received label 100101 100000
Extcommunity: RT:64512:100000 RT:64512:100101 SOO:10.81.1.23:0 ENCAP:8
!!! rmac is the virtual-rmac derived from shared secondary ip address on loop1.
Router MAC:0200.0a51.0117
Path-id 1 not advertised to any peer
!!! EVPN rt-2 (mac+ip) route received from Leaf-Sw-03.
Route Distinguisher: 10.81.0.3:32868
!!! Vlan-101-Pc-02 IP/mac - 10.85.101.12/503e.8b00.2300.
BGP routing table entry for [2]:[0]:[0]:[48]:[503e.8b00.2300]:[32]:[10.85.101.12]/272, version 863
Paths: (1 available, best #1)
Flags: (0x000202) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 3 destination(s)
Imported paths list: Vrf-Blue L2-100101 L3-100000
AS-Path: NONE, path sourced internal to AS
10.81.1.23 (metric 41) from 10.81.0.3 (10.81.0.3)
Origin IGP, MED not set, localpref 100, weight 0
Received label 100101 100000
Extcommunity: RT:64512:100000 RT:64512:100101 SOO:10.81.1.23:0 ENCAP:8
!!! rmac is the virtual-rmac derived from shared secondary ip address on loop1.
Router MAC:0200.0a51.0117
Path-id 1 not advertised to any peer
rt-2 (mac+ip) will enter in the vrf's routing table.
Leaf-Sw-01# show ip route 10.85.101.12 vrf Vrf-Blue
IP Route Table for VRF "Vrf-Blue"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.85.101.12/32, ubest/mbest: 1/0
*via 10.81.1.23%default, [200/0], 03:31:14, bgp-64512, internal, tag 64512, segid: 100000 tunnelid: 0xa510117 encap:
VXLAN
Now we can have a look on how next-hop 10.81.1.23 (shared secondary ip) is reachable in the underlay network - we have 4 way ICMP for that.
Leaf-Sw-01# show ip route 10.81.1.23
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.81.1.23/32, ubest/mbest: 4/0
*via 10.81.0.2, Eth1/1, [110/41], 04:36:29, ospf-UNDERLAY, intra
*via 10.81.0.2, Eth1/2, [110/41], 04:36:29, ospf-UNDERLAY, intra
*via 10.81.0.3, Eth1/3, [110/41], 04:36:29, ospf-UNDERLAY, intra
*via 10.81.0.3, Eth1/4, [110/41], 04:36:29, ospf-UNDERLAY, intra
rt-5 (prefix) route will be announced by both vPC switches using their primary ip address used by the loopback1 interface and rmac will be the switches individual local rmac.
Leaf-Sw-01# show bgp l2vpn evpn 10.85.102.0
BGP routing table information for VRF default, address family L2VPN EVPN
!!! rt-5 for 10.85.102.0/24 received for Leaf-Sw-02.
Route Distinguisher: 10.81.0.2:4
BGP routing table entry for [5]:[0]:[0]:[24]:[10.85.102.0]/224, version 834
Paths: (1 available, best #1)
Flags: (0x000002) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 2 destination(s)
Imported paths list: Vrf-Blue L3-100000
Gateway IP: 0.0.0.0
AS-Path: NONE, path sourced internal to AS
!!! next-hop is loop1 primary ip address of Leaf-Sw-02.
10.81.1.2 (metric 41) from 10.81.0.2 (10.81.0.2)
Origin incomplete, MED 0, localpref 100, weight 0
Received label 100000
!!! rmac is local router mac of Leaf-Sw-02.
Extcommunity: RT:64512:100000 ENCAP:8 Router MAC:5000.1f00.1b08
Path-id 1 not advertised to any peer
!!! rt-5 for 10.85.102.0/24 received for Leaf-Sw-03.
Route Distinguisher: 10.81.0.3:4
BGP routing table entry for [5]:[0]:[0]:[24]:[10.85.102.0]/224, version 948
Paths: (1 available, best #1)
Flags: (0x000002) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not in HW
Advertised path-id 1
Path type: internal, path is valid, is best path, no labeled nexthop
Imported to 2 destination(s)
Imported paths list: Vrf-Blue L3-100000
Gateway IP: 0.0.0.0
AS-Path: NONE, path sourced internal to AS
!!! next-hop is loop1 primary ip address of Leaf-Sw-03.
10.81.1.3 (metric 41) from 10.81.0.3 (10.81.0.3)
Origin incomplete, MED 0, localpref 100, weight 0
Received label 100000
!!! rmac is local router mac of Leaf-Sw-03.
Extcommunity: RT:64512:100000 ENCAP:8 Router MAC:5000.2000.1b08
Path-id 1 not advertised to any peer
rt-5 (prefix) will enter in the vrf's routing table.
Leaf-Sw-01# show ip route 10.85.102.0 vrf Vrf-Blue
IP Route Table for VRF "Vrf-Blue"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.85.102.0/24, ubest/mbest: 2/0
*via 10.81.1.2%default, [200/0], 07:35:45, bgp-64512, internal, tag 64512, segid: 100000 tunnelid: 0xa510102 encap:
VXLAN
*via 10.81.1.3%default, [200/0], 00:06:26, bgp-64512, internal, tag 64512, segid: 100000 tunnelid: 0xa510103 encap:
VXLAN
That's all about vPC and VXLAN.
From Part - 07 - we will work with actual Spine-Leaf topology as now we have covered most of the foundational basics for VXLAN-EVPN.
Comments
Post a Comment