Mixing layer-3 and layer-2 over a single aggregated link? Possible???

I have tried a strange combination recently and was very surprised to see that it works - mixing layer-2 and layer-3 over an LACP aggregated interface. I have no explanation why it works. Does any of my reader have any idea?

Let's introduce our topology first-

Fig 01 - Link-aggregation Topology
Now the problem definition - 

We are using two Arista  switches. In Arista-SW-L3 we will create a L3 (layer 3) link-aggregation and in Arista-SW-L2 we will create a L2 (layer 2) link-aggregation over the same pair of interfaces (eth1 and eth2). Then we will verify the connectivity by doing ping from the L2 side (vlan interfaces) to the L3 side (port-channel interfaces). And come to conclusion that we can mix and match L2 and L3 over a single link-aggregation interface.

 Let's configure the L3 side of our aggregated link - 

 Arista-SW-L3 configuration

 We configure the switch by running the following commands -

 interface Ethernet1
   no switchport --Creating L3 port
   channel-group 1 mode active --Enabling link-aggregation with LACP
!
interface Ethernet2
   no switchport
   channel-group 1 mode active
!
interface Port-Channel1
   no switchport
!
interface Port-Channel1.501
   encapsulation dot1q vlan 501 --Enable a L3 interface which can process frames with VLAN id 501
   ip address 10.50.1.1/24
!
interface Port-Channel1.502
   encapsulation dot1q vlan 502
   ip address 10.50.2.1/24
!

Now we will configure the L2 side of our aggregated link -

Arista-SW-L2 configuration

We configure the switch by running the following commands -

vlan 501-502
!
interface Vlan501
   ip address 10.50.1.2/23
!
interface Vlan502
   ip address 10.50.2.2/23
!
interface Ethernet1
   channel-group 1 mode active
!
interface Ethernet2
   channel-group 1 mode active
!
interface Port-Channel1
   switchport mode trunk --Aggregated interface works at L2 only
   switchport trunk allowed vlan 501-502 --Allowing VLAN 501 and 502 over L2 aggregated trunk
!

Verification

From our L3 switch we can verify our port-channel 1 interface is not enabled for spanning tree, hence it is a L3 interface.

Arista-SW-L3#sh spanning-tree 
MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     5000.00cb.38c2
             This bridge is the root

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     5000.00cb.38c2
             Hello Time  2.000 sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role       State      Cost      Prio.Nbr Type
---------------- ---------- ---------- --------- -------- --------------------
--No interface is running spanning tree

We can also verify our aggregated interface is a L3 interface by running commands below -  

Arista-SW-L3#sh int status
Port       Name   Status       Vlan     Duplex Speed  Type                Flags Encapsulation
Et1               connected    in Po1   full   unconf EbraTestPhyPort                        
Et2               connected    in Po1   full   unconf EbraTestPhyPort                        
Po1               connected    routed   full   unconf N/A --L3 interface                                    
Po1.501           connected    routed   full   unconf dot1q-encapsulation       501 --L3 interface         
Po1.502           connected    routed   full   unconf dot1q-encapsulation       502 --L3 interface         

Arista-SW-L3#sh ip int brief
Interface                    IP Address         Status     Protocol         
Port-Channel1           unassigned         up           up              
Port-Channel1.501    10.50.1.1/24       up           up           
Port-Channel1.502    10.50.2.1/24       up           up           


Now we will look at the aggregated interface status in our L2 switch.

Arista-SW-L2#sh spanning-tree 
MST0
  Spanning tree enabled protocol mstp
  Root ID    Priority    32768
             Address     5000.00d5.5dc0
             This bridge is the root

  Bridge ID  Priority    32768  (priority 32768 sys-id-ext 0)
             Address     5000.00d5.5dc0
             Hello Time  2.000 sec  Max Age 20 sec  Forward Delay 15 sec

Interface        Role       State      Cost      Prio.Nbr Type
---------------- ---------- ---------- --------- -------- --------------------
Po1              designated forwarding 1999      128.100  P2p Edge --Interface is running spanning tree          

Arista-SW-L2#sh int status 
Port       Status          Vlan     
Et1        connected    in Po1   
Et2        connected    in Po1   
Po1       connected    trunk --Interface is a L2 trunk    

Arista-SW-L2#sh ip int brief
Interface              IP Address         Status     Protocol        
Vlan501              10.50.1.2/23       up           up              
Vlan502              10.50.2.2/23       up           up

Let's verify the connectivity by running a simple ping command from the L2 switch -

Arista-SW-L2#ping 10.50.1.1
PING 10.50.1.1 (10.50.1.1) 72(100) bytes of data.
80 bytes from 10.50.1.1: icmp_seq=1 ttl=64 time=21.3 ms
80 bytes from 10.50.1.1: icmp_seq=2 ttl=64 time=24.1 ms
80 bytes from 10.50.1.1: icmp_seq=3 ttl=64 time=21.2 ms

Arista-SW-L2#ping 10.50.2.1
PING 10.50.1.1 (10.50.2.1) 72(100) bytes of data.
80 bytes from 10.50.2.1: icmp_seq=3 ttl=64 time=20.2 ms
80 bytes from 10.50.2.1: icmp_seq=4 ttl=64 time=19.0 ms
80 bytes from 10.50.2.1: icmp_seq=5 ttl=64 time=18.1 ms
Arista-SW-L2#

Conclusion

So, it is now verified that we can have a working aggregated-link where one side is L3 and the other side is L2. But the question is why does it works. I don't have any explanation at the time of writing. I am guessing may be it is because on the L3 side of the link we are processing frames tagged with VLAN ids (501 and 502) by using the command "encapsulation dot1q vlan 501". May be one of my reader can give me some clues and explain it to me.
Location: Sundsvall, Sweden

Comments

  1. MahfuzJune 1, 2019 at 5:14 AM

    Interesting configuration. ICMP/PING are working fine but this if you transmit other packets that contain header and need to be serialised what will happen?
    You might try to transmit some diameter packet and take a tcp dump and see the packet travel is serialised or splitted. Just an idea.

    ReplyDelete
  2. MahfuzJune 1, 2019 at 5:22 AM

    For packet simulation you might use:
    http://packeth.sourceforge.net/packeth/Home.html

    ReplyDelete

Post a Comment

Popular posts from this blog

Fortigate firewall AAA Configuration for management with TACACS+ protocol and Cisco ISE

Image
In this blog I will write about how to implement AAA services in Fortigate firewalls using Cisco ISE as an authentication server with TACACS+ protocol. First good to know our limitations. AAA means - Authentication (supported by ISE TACACS+), Authorization (partially supported by ISE TACACS+; Fortigate implements Admin-Profiles; so what a user can/cannot do is defined locally in the firewall with "Admin Profiles"; ISE just instructs the firewall to allocate one of those profiles), Accounting (not supported; instead configure syslog to get accounting data, Fortigate firewalls generate syslog messages for configuration changes made by a user). Let's jump to our configuration. Our topology is very simple; one Fortigate firewall and Cisco ISE server connected with the same management network which looks like below -  01 - Network Topology We have a management network 192.168.199.0/24. The firewall is at  .230  and Cisco ISE is at  .49  IP address. We are running...
Read more

Stacking switches Part - VI (Dell OS10 VLT - Virtual Link Trunking)

Image
In this blog I will configure MLAG (multi chassis link aggregation) for Dell OS10 switches. As every other vendor choses a fancy name of their implementation of MLAG; Dell is no exception to this. Dell calls their implementation VLT - Virtual Link Trunking . Let's look at our network topology - Two Dell OS10 switches will run peering between them and will run VLT/MLAG between them. Two Debian 10 linux machine will be connected with LACP bonding with both switches. They will simulate the client connection. Our topology looks like below - 01 - Dell OS10 VLT Topology Let's look at some terminology before configuration - VLT peer – The two switches participating in VLT are peer to each other. In a VLT domain a maximum of two switches are allowed. VLT interconnect (VLTi) – VLTi synchronizes state information between VLT peers. It synchronizes layer 2 and layer 3 control-plane information. These state/control-plane information includes MAC table, ARP table, IPv6 neighbors etc. VLT ...
Read more

Network device configuration management with Oxidized (Basic)

Image
I always like to work with open source tools. As a network engineer, we need to maintain a device configuration management system where we take backup of configuration for different networking devices like routers, switches, firewalls etc. We want to have our backups ready so that we can restore a device configuration when a device fails or replace with new one or for auditing purpose to track recent configuration changes. Today I will talk about such a open source tools named - Oxidized. Interested readers can have a look at it's GitHub page . It is a network configuration management tool with support for diverse networking equipments. Oxidized is developed with programming language Ruby. It is very configurable, extensible and can be integrated with management tools like Librenms . I will do a two part blog about Oxidized. In this part I will cover basic setup of Oxidized and start taking configuration backup of networking equipments. Network Topology Our topology l...
Read more