Private VLAN on Arista EOS Switch (How ISP isolates traffic between subscribers)
The other day, I was looking at the arp table entry for my home router and found a funny thing going on -
$ show arp
Address HWtype HWaddress Iface
10.10.10.1 ether 04-96-AF-EF-C6-53 eth0
10.10.10.10 ether 04-96-AF-EF-C6-53 eth0
Above arp table is an example hiding the actual IP addresses and MAC addresses. Here my IP address is let's say - 10.10.10.50/24 and GW is 10.10.10.1. Now the question comes why another subscriber of the ISP in the same subnet (10.10.10.10) also have same mac address as my GW (04-96-AF-EF-C6-53). We are in the same IP network/VLAN (ISP definitely configured those), but my router cannot see the actual MAC address of other devices in the network.
The reason is that my ISP is isolating devices in layer-2 using a technique called private vlan. All of my layer-2 and layer-3 traffic within my subnet is proxied through the ISP router. We the different subscribers cannot see each other's broadcast traffic at layer-2; only ISP router can see that and proxies that information back and forth between it's subscribers.
Private VLAN terminology
We need to learn some concepts behind private vlan before implementing it in our lab.
VLAN - In a switch the ports which belong to a vlan are under a single layer-2 broadcast domain. For example - port 1, 2 and 3 in a switch belong to vlan 101 and they are in the same layer-2 broadcast domain; there is no layer-2 isolation between them.
Private VLAN - In a switch the ports which belong to a private vlan are not under a single layer-2 broadcast domain. Even though they are under the same vlan, there exists layer-2 isolation between them depending on what kind of private vlan ports they are. For example - port 1, 2 and 3 in a switch belong to private vlan 101 (isolated), then they are isolated at layer-2 and devices connected to these ports cannot communicate with each other.
In private vlan we have three types of VLAN -
Primary VLAN - A vlan which can communicate with the associated secondary isolated or community vlans. A switch port that carries the primary vlan is called promiscuous port.
Community VLAN - A vlan which can communicate with the associated primary vlan and also in the same community vlan. A switch port that carries the community vlan is called community port.
Isolated VLAN - A vlan which can communicate with the associated primary vlan only. Communication between ports in the same isolated vlan is prohibited. A switch port that carries the isolated vlan is called isolated port.
Topology
Let's create a lab and get a first hand experience about private vlan.
01 - Private VLAN Topology
In above topology we have a single IP network (192.168.5.0/24). But three vlans ; 100 - primary, 101 - isolated and 102 - community. According to our private vlan terminology -
Promiscuous port - which is eth1 of SW_01 where our gateway for the network - GW01 is connected. GW01 can reach all the devices in 192.168.5.0/24 network whether the switch ports are isolated or community.
Community port - which is eth3 of both SW_01 and SW_02. Here devices connected to these ports can reach each other and the device connected to the promiscuous port.
Isolated port - which is eth2 of both SW_01 and SW_02. Here devices connected to these ports can only reach the device connected to the promiscuous port. The isolated ports cannot communicate between them.
Configuration
SW_01 Configuration
vlan 100
!!! Primary VLAN
name Primary
!
vlan 101
!!! Isolated VLAN
name Isolated
!!! Associating isolated vlan with primary vlan
private-vlan isolated primary vlan 100
!
vlan 102
!!! Community vlan
name Community
!!! Associating community vlan with primary vlan
private-vlan community primary vlan 100
!
interface Ethernet1
!!! Promiscuous port which is a access port in primary vlan
switchport mode access
switchport access vlan 100
!
interface Ethernet2
!!! Isolated port which is a access port in isolated vlan
switchport mode access
switchport access vlan 101
!
interface Ethernet3
!!! Community port which is a access port in community vlan
switchport mode access
switchport access vlan 102
!
interface Ethernet8
!!! Trunk between switches
!!! which must carry all the vlans primary, isolated and community
switchport mode trunk
switchport trunk allowed vlan 100-102
!
We have a separate gateway in our topology (GW01). One of the switch can also be a gateway for the network also. In that case we will configure a SVI in the switch as below - (only applicable if one of the switch is also gateway)
!!! Only required if the switch is also gateway
interface Vlan100
!!! Configure IP adress
ip address 192.168.5.1/24
!!! Which secondary community and isolated vlans
!!! are associated with primary vlan SVI
pvlan mapping 101-102
SW_02 Configuration
Configuration is exactly same as SW_01. It is given for reference only.
vlan 100
name Primary
!
vlan 101
name Isolated
private-vlan isolated primary vlan 100
!
vlan 102
name Community
private-vlan community primary vlan 100
!
interface Ethernet2
switchport mode access
switchport access vlan 101
!
interface Ethernet3
switchport mode access
switchport access vlan 102
!
interface Ethernet8
switchport mode trunk
switchport trunk allowed vlan 100-102
!
GW01 Configuration
interface GigabitEthernet1
!!! We are just configuring IP address on the physical interface
!!! SW_01 is configured to send frames belonging to VLAN 100 untagged to GW
ip address 192.168.5.1 255.255.255.0
!!! Disables ICMP redirect messages from interface
no ip redirects
Verification
As the gateway is connected to a promiscuous port of SW_01, it can reach all devices in both community and private ports/vlan.
GW01#ping 192.168.5.3
Type escape sequence to abort.
!!! communication towards any associated isolated vlan is allowed
Sending 5, 100-byte ICMP Echos to 192.168.5.3, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/8/15 ms
GW01#ping 192.168.5.5
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 192.168.5.5, timeout is 2 seconds:
!!! communication towards any associated community vlan is allowed
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 6/6/7 ms
GW01#show arp
Protocol Address Age (min) Hardware Addr Type Interface
Internet 192.168.5.2 0 5097.9900.0d00 ARPA GigabitEthernet1
Internet 192.168.5.3 0 50c5.1600.0f00 ARPA GigabitEthernet1
Internet 192.168.5.4 0 50ef.8f00.1000 ARPA GigabitEthernet1
Internet 192.168.5.5 0 508e.ab00.1100 ARPA GigabitEthernet1
Now we will test reachability from a community port (eth3 - SW_02 - 192.168.5.5 - PC04_COMM) -
PC04_COMM# ping 192.168.5.4 -c 3
PING 192.168.5.4 (192.168.5.4) 56(84) bytes of data.
!!! Communication towards same community vlan is allowed
64 bytes from 192.168.5.4: icmp_seq=1 ttl=64 time=5.70 ms
64 bytes from 192.168.5.4: icmp_seq=2 ttl=64 time=6.50 ms
64 bytes from 192.168.5.4: icmp_seq=3 ttl=64 time=5.81 ms
PC04_COMM# ping 192.168.5.1 -c 3
PING 192.168.5.1 (192.168.5.1) 56(84) bytes of data.
!!! Communication towards associated primary vlan is allowed
64 bytes from 192.168.5.1: icmp_seq=1 ttl=255 time=5.90 ms
64 bytes from 192.168.5.1: icmp_seq=2 ttl=255 time=5.83 ms
64 bytes from 192.168.5.1: icmp_seq=3 ttl=255 time=5.57 ms
PC04_COMM# ping 192.168.5.3 -c 3
PING 192.168.5.3 (192.168.5.3) 56(84) bytes of data.
!!! Communication towards any isolated vlan is not allowed
From 192.168.5.5 icmp_seq=1 Destination Host Unreachable
From 192.168.5.5 icmp_seq=2 Destination Host Unreachable
From 192.168.5.5 icmp_seq=3 Destination Host Unreachable
Now we will test reachability from a isolated port (eth2 - SW_02 - 192.168.5.3 - PC02_ISO) -
PC02_ISO# ping 192.168.5.1 -c 3
!!! Communication towards associated primary vlan is allowed
PING 192.168.5.1 (192.168.5.1) 56(84) bytes of data.
64 bytes from 192.168.5.1: icmp_seq=1 ttl=255 time=6.03 ms
64 bytes from 192.168.5.1: icmp_seq=2 ttl=255 time=12.5 ms
64 bytes from 192.168.5.1: icmp_seq=3 ttl=255 time=5.97 ms
PC02_ISO# ping 192.168.5.4 -c 3
!!! Communication towards any community vlan is not allowed
PING 192.168.5.4 (192.168.5.4) 56(84) bytes of data.
From 192.168.5.3 icmp_seq=1 Destination Host Unreachable
From 192.168.5.3 icmp_seq=2 Destination Host Unreachable
PC02_ISO# ping 192.168.5.2 -c 3
!!! Communication towards same isolated vlan is not allowed
PING 192.168.5.2 (192.168.5.2) 56(84) bytes of data.
From 192.168.5.3 icmp_seq=1 Destination Host Unreachable
From 192.168.5.3 icmp_seq=2 Destination Host Unreachable
From 192.168.5.3 icmp_seq=3 Destination Host Unreachable
We can verify the private vlan settings from our switches also.
SW01#show vlan private-vlan
Primary Secondary Type Ports
------- --------- ----------- -------------------------------
100 101 isolated Et1, Et2, Et8
100 102 community Et1, Et3, Et8
If one of our switch is also gateway for the network, we can view the mappings/associations configured for SVI using the command below -
SW01#show pvlan mapping interfaces
Interface Secondary Vlans
--------- ---------------
Vlan100 101-102
How the switch tags/untags layer-2 frames in private vlan
We need to understand how the switch forwards the layer-2 frame in our topology. For example - a ping from 192.168.5.3 (PC02_ISO) to GW01 (192.168.5.1).
ICMP request flow -
- 192.168.5.3 -> 192.168.5.1 (icmp request)
- SW_02 receives the frame in eth2 interface without any vlan tag.
- SW_02 tags the frame with vlan 101 (isolated) and forwards it via eth8 interface.
- SW_01 receives the frame, looking at the vlan tag (101) it understands that this is a isolated vlan and can only forward the packet to the associated primary vlan. SW_01 strips the vlan tag from the frame and forwards it to eth1 interface.
- The GW01 receives the frame at gi1 interface without any vlan tag.
ICMP reply flow -
- 192.168.5.1 -> 192.168.5.3 (icmp reply)
- SW_01 receives the frame in eth1 interface without any vlan tag.
- SW_01 tags the frame with vlan 100 (primary) and forwards it via eth8 interface.
- SW_02 receives the frame, looking at the vlan tag (100) it understands that the frame is coming from a primary vlan. But how to forward it now??? In Arista all the mac adress learning happens in primary vlan only. So, it looks up the destination mac from mac adress table and forwards it to eth2 interface without any vlan tag.
- The PC02_ISO receives the frame at e0 interface without any vlan tag.
SW02#show mac address-table
Mac Address Table
------------------------------------------------------------------
Vlan Mac Address Type Ports Moves Last Move
---- ----------- ---- ----- ----- ---------
!!! 5007.5a00.1200 - GW01, 50c5.1600.0f00 - PC02_ISO
!!! All mac adress belongs to vlan 100
!!! No mac adress learning for VLAN 101 or 102
100 5007.5a00.1200 DYNAMIC Et8 1 0:17:49 ago
100 50c5.1600.0f00 DYNAMIC Et2 1 0:17:49 ago
PC02_ISO# arp -n
Address HWtype HWaddress Flags Mask Iface
192.168.5.1 ether 50:07:5a:00:12:00 C e0
Isolated port communication through Gateway
Now we will try to answer our question - how my ISP was isolating traffic at layer-2 between their subscribers? And why I was seeing the ISP's gateway's mac address in the ARP table for all the other devices connected to the same subnet.
Let's use our topology - say we have isolated ports/vlans only. Then we already got half of the answer to our question. From PC02_ISO (192.168.5.3) we run a ping toward PC01_ISO (192.168.5.2) - ping will not be successful and we will have a incomplete entry in PC02_ISO's arp table. Our switches are not allowing communication between isolated ports.
PC02_ISO# arp -n
Address HWtype HWaddress Flags Mask Iface
192.168.5.2 (incomplete) e0
192.168.5.1 ether 50:07:5a:00:12:00 C e0
What is the difference between our topology and the ISP setup - in the ISP setup the ARP table was complete with ISP's gateway's MAC adress and we were able to communicate even with devices in the same isolated vlans.
My ISP gateway has configured something called "ip local-proxy-arp" which means when a device in the same layer-2 network sends a ARP broadcast; the gateway will answer with it's own mac adress for those request. And all the network communication between clients of the isolated vlan will pass through the gateway and they can communicate with each other. Any kind of traffic generated by clients in that isolated vlan will always pass through the gateway. For example; ping from 192.168.5.3 to 192.168.5.2 will be like this - the gateway 192.168.5.1 will always be between clients invisibly.
- ICMP request - 192.168.5.3 -> 192.168.5.1 -> 192.168.5.2
- ICMP reply - 192.168.5.2 -> 192.168.5.1 -> 192.168.5.3
Below shows how to configure the local-proxy-arp in our gateway -
GW01#show running-config interface gigabitEthernet 1
interface GigabitEthernet1
ip address 192.168.5.1 255.255.255.0
no ip redirects
!!! local-proxy-arp is configured on the interface
ip local-proxy-arp
PC02_ISO:~# ping 192.168.5.2 -c 3
PING 192.168.5.2 (192.168.5.2) 56(84) bytes of data.
!!! Devices in the same isolated vlan can communicate with each other
64 bytes from 192.168.5.2: icmp_seq=1 ttl=64 time=12.2 ms
64 bytes from 192.168.5.2: icmp_seq=2 ttl=64 time=7.14 ms
64 bytes from 192.168.5.2: icmp_seq=3 ttl=64 time=6.78 ms
PC02_ISO:~# arp -n
Address HWtype HWaddress Flags Mask Iface
!!! 192.168.5.2's mac address is same as the gateway's mac address
!!! Gateway is proxying traffic between clients by responding to ARP requests
192.168.5.2 ether 50:07:5a:00:12:00 C e0
192.168.5.1 ether 50:07:5a:00:12:00 C e0
That's it for today. So, next time around log in to your internet router; do a packet capture; look at the ARP table. If you see setup like above - then the ISP is doing a good job by isolating layer-2 traffic between it's customers. Otherwise, give them a call and ask for an explanation.
Comments
Post a Comment