Configuring Cisco Embedded Event Manager (EEM) - Programming IOS

In today's network; everyone tries to automate as much as possible. I will cover one such topic today - EEM (Embedded Event Manager) in Cisco devices. EEM is supported by Cisco IOS, XR and NX-OS.

EEM allows automation for Cisco device administrators by tracking and monitoring events in network devices and take actions when those events occur. For example; we want to shutdown an network interface at a scheduled time, change interface description when new LLDP neighbors are detected, backup device configuration etc.

Overview of EEM (Embedded Event Manager)

EEM - we need to understand two topics:
  • EEM event detectors - We use event detector to subscribe to an event; for example - interface went down, routing neighbor is lost etc. When such an event is occurred; we will do some automated actions.
  • EEM actions - When an EEM event occurs; we will do some predefined tasks automatically; these are called EEM actions. For example; send an email alert when an interface down event is detected.
Below picture from Cisco© explains the concept of EEM in a easy way -

01 - Overview Of EEM
01 - Overview of EEM

We can configure EEM in two ways -
  • Applet - Most easiest way. This method will be used mostly in this blog.
  • TCL scripts - Complex and we get the full power of Cisco's implementation of TCL programming language.

Basic EEM Configuration

Let's start with a basic example - We want to generate an arbitrary syslog message. We will configure following in our switch -

event manager applet Applet_Syslog
 !!! We are not subscribing to any event; we will run manually the applet
 event none
 !!! Adding comment for our action
 action 1.0 comment "Generate a syslog message with EEM applet"
 !!! Take an EEM action by generating a syslog message 
 action 1.1 syslog msg "Welcome to EEM with Applet!!!"

Now we will run the applet manually by using command below and see a syslog message has been generated.

SW01# event manager run Applet_Syslog                                        
SW01#
*Jul  5 09:27:37.505: %HA_EM-6-LOG: Applet_Syslog: Welcome to EEM with Applet!!!

Now we will convert our applet to a TCL script to show how to use TCL scripts with EEM.

Let's create a TCL script file in the flash -

SW01#tclsh
SW01(tcl)#puts [open "flash:Eem_Syslog.tcl" w+] {
+>::cisco::eem::event_register_none
+>
+>namespace import ::cisco::eem::*
+>namespace import ::cisco::lib::*
+>
+>array set arr_einfo [event_reqinfo]
+>
+># "Generate a syslog message with EEM applet"
+>action_syslog msg "Welcome to EEM with TCL script!!!"
+>}
SW01(tcl)#tclquit

Now have a look at the file in IOS - 

SW01#more Eem_Syslog.tcl

 !!! We are not subscribing to any event; we will run manually the applet
::cisco::eem::event_register_none

!!! Importing necessary EEM libraries
namespace import ::cisco::eem::*
namespace import ::cisco::lib::*

array set arr_einfo [event_reqinfo]

# "Generate a syslog message with EEM applet"
action_syslog msg "Welcome to EEM with TCL script!!!"

After that we need to register both the directory where our TCl script is saved and the script itself.

!!! Register the directory where the script resides
SW01(config)#event manager directory user policy flash0:/
!!! Register the script
SW01(config)#event manager policy Eem_Syslog.tcl

After that we can run the script same way as the applet.

SW01#event manager run Eem_Syslog.tcl
SW01#
*Jul  5 14:44:24.920: %HA_EM-6-LOG: Eem_Syslog.tcl: Welcome to EEM with TCL script!!!


Advanced EEM Configuration

Now we will do two examples of EEM which are event driven.

Example 1 - shutdown interface at a specific time

We will shutdown an interface (gi0/1) at a scheduled time (at 20:00 everyday). And we will turn it back on gi0/1 at 07:00 everyday. Let's create our applet -

event manager applet Applet_Shutdown_Gi0/1
 !!! An event will occur with cron at 20:00 everyday 
 event timer cron cron-entry "0 20 * * *"
 action 1.0 comment "Shutdown Gi0/1 port at 20:00 everyday"
 !!! We will run below commands/actions to shutdown the interface
 action 1.1 cli command "enable"
 action 1.2 cli command "configure terminal"
 action 1.3 cli command "interface Gi0/1"
 action 1.4 cli command "shutdown"
 action 1.5 cli command "end"
 action 1.6 syslog msg "Interface Gi1/0/1 has been shutdown with scheduled applet"

event manager applet Applet_Open_Gi0/1
 !!! An event will occur with cron at 07:00 everyday 
 event timer cron cron-entry "0 7 * * *"
 action 1.0 comment "Open Gi0/1 port at 07:00 everyday"
 !!! We will run below commands/actions to turn on the interface
 action 1.1 cli command "enable"
 action 1.2 cli command "configure terminal"
 action 1.3 cli command "interface Gi0/1"
 action 1.4 cli command "no shutdown"
 action 1.5 cli command "end"
 action 1.6 syslog msg "Interface Gi1/0/1 has been opened with scheduled applet"

We will see the below message in logs when it is 20:00 o'clock.

SW01#
Jul  5 20:00:00.287: %HA_EM-6-LOG: Applet_Shutdown_Gi0/1: Interface Gi1/0/1 has been shutdown with scheduled applet

Example 2 - Change interface description from LLDP neighborship

We want to change our interface description when a LADP neighbor is added or deleted. Our interface description will be in the format - (LLDP-Neighbor-Name)-(LLDP-Neighbor-Port-Id) - SW02-Gi0/1. Now time to write our applet - one for adding interface description whenever a new LLDP neighbor is discovered for an interface and one for removing interface description when a LLDP neighbor is deleted for an interface.

event manager applet Applet_LLDP_Port_Description_Add
 description "Automatically add interface description using LLDP neighbor information"
 !!! Monitor lldp add events and match interface name that contains GigabitEthernet
 event neighbor-discovery interface regexp GigabitEthernet.* lldp add
 action 1.0 comment "Triming domain name from LLDP system name"
 !!!We will exclude the domain name part from LLDP neighbor using regular expression
 action 1.1 regexp "(^[^.]*)|(^[^.]*$)" "$_nd_lldp_system_name" match neighbor_host_name
 action 2.0 comment "Update interface description and save the configuration"
 action 2.1 cli command "enable"
 action 2.2 cli command "configure terminal"
 action 2.3 cli command "interface $_nd_local_intf_name"
 action 2.4 cli command "description $neighbor_host_name-$_nd_port_id"
 action 2.5 cli command "end"
 action 2.6 cli command "write memory"
 action 2.7 syslog msg "EEM applet added interface description for - $_nd_local_intf_name"
 action 2.8 exit 0

event manager applet Applet_LLDP_Port_Description_Remove
 description "Automatically delete interface description using LLDP neighbor information"
 !!! Monitor lldp delete events and match interface name that contains GigabitEthernet
 event neighbor-discovery interface regexp GigabitEthernet.* lldp delete
 action 1.0 comment "Remove interface description and save the configuration"
 action 1.1 cli command "enable"
 action 1.2 cli command "configure terminal"
 action 1.3 cli command "interface $_nd_local_intf_name"
 action 1.4 cli command "no description"
 action 1.5 cli command "end"
 action 1.6 cli command "write memory"
 action 1.7 syslog msg "EEM script deleted interface description for - $_nd_local_intf_name"
 action 1.8 exit 0

Now some explanation about - $_nd_local_intf_name, $_nd_port_id. What are these and how can we know about them. These are built-in variables provided by different EEM events. We can get a the complete list of these variables for neighbor-discovery (both LLDP and CDP) by using the command below -

SW01#show event manager detector neighbor-discovery detailed (output truncated)
No.  Name                Version   Node        Type    
1    neighbor-discovery  01.00     node0/0     RP      

        Tcl Configuration Syntax: 
        ::cisco::eem::event_register_neighbor_discovery 
                 [tag <tag-val>] 
                 interface <interface-pattern> 
                 [cdp {add | update | delete | all}] 
                 [lldp {add | update | delete | all}]
          

        LLDP EVENT VARIABLES: 
        protocol 
        lldp_system_name 
        lldp_system_description 
        lldp_port_description 
       
        COMMON VARIABLES: 
        $_nd_notification 
        $_nd_intf_linkstatus 
        $_nd_intf_linestatus 
        $_nd_local_intf_name 
        $_nd_short_local_intf_name 
        $_nd_port_id 
        
        LLDP EVENT VARIABLES: 
        $_nd_protocol 
        $_nd_proto_notif 
        $_nd_proto_new_entry 
        $_nd_lldp_chassis_id 
        $_nd_lldp_system_name 
        $_nd_lldp_system_description 
        $_nd_lldp_ttl 
        $_nd_lldp_port_description 

Using above command we can get a list of all the built-in variables and use them according to our automation needs.

EEM threads and VTY lines

In most Cisco devices we have 16 VTY lines. We need to beware of an extreme case scenario. For example - our switch has deleted 20 new LLDP neighbors at the same time. Then for each LLDP neighbor update  applet operation; EEM will create a thread, use one VTY lines. But we have maximum 16 lines and we require 20 threads with 20 vty lines. In such case; our applet execution will fail. Instead what we can configure is that - EEM can create maximum 5 threads and use 5 vty lines. Other update operations/threads will wait until current ones are finished; then EEM will create new threads for the updates with are in queue. In this case; we will use queuing so that EEM does not exhaust VTY lines. The command to configure maximum number of threads of EEM is -

!!! Allow maximum of 5 threads/5 vty lines of EEM
SW01(config)#event manager scheduler applet thread class default number 5

Now time to test our LLDP update/delete applet. We forcefully clear LLDP table; so that we can see both remove/add operation.

SW01#clear lldp table

After some time when LLDP neighbor table is populated - We can see our interface description -

SW01#show interfaces description 
Interface                      Status         Protocol Description
Gi0/0                           down           down     
Gi0/1                           up               up         SW02-Gi0/1
Gi0/2                           up               up         SW03-Gi0/2
Gi0/3                           down           down     
Gi1/0                           down           down     
Gi1/1                           down           down     
Gi1/2                           down           down     
Gi1/3                           down           down     

We can also see the syslog messages from our applets -

SW01#show logging | include %HA_EM-6-LOG
Jul  5 23:41:54.072: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Remove: EEM script deleted interface description for - GigabitEthernet0/1
Jul  5 23:41:56.021: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Remove: EEM script deleted interface description for - GigabitEthernet0/2
Jul  5 23:42:17.156: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add: EEM applet added interface description for - GigabitEthernet0/1
Jul  5 23:42:23.222: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add: EEM applet added interface description for - GigabitEthernet0/2 
 
Verification

We can view all our configured EEM policies in the device using command -

SW01#show event manager policy registered (output truncated)

No. Class   Type  Event Type  Trap  Time Registered              Name

1    applet  user  none           Off    Mon Jul 5 09:25:36   Applet_Syslog
 policyname {Applet_Syslog} sync {yes}
 maxrun 20.000
 action 1.0 comment "Generate a syslog message with EEM applet"
 action 1.1 syslog msg "Welcome to EEM with Applet!!!"

3    applet  user   timer cron   Off   Mon Jul 5 15:03:28  Applet_Shutdown_Gi0/1
 cron entry {0 20 * * *}
 maxrun 20.000
 action 1.0 comment "Shutdown Gi0/1 port at 20:00 everyday"
 action 1.1 cli command "enable"
 action 1.2 cli command "configure terminal"
 action 1.3 cli command "interface Gi0/1"
 action 1.4 cli command "shutdown"
 action 1.5 cli command "end"
 action 1.6 syslog msg "Interface Gi1/0/1 has been shutdown with scheduled applet"

As expected detailed debugging of EEM also provided by Cisco.

!!! Turn on EEM debugging for CLI operations. Our applets use CLI.
SW01#debug event manager action cli
Debug EEM action cli debugging is on

!!! Filtering EEM CLI debug logs which shows the CLI commands executed by Applet.
SW01#show logging | include DEBUG(cli_lib)*
Jul  5 23:56:40.607: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : 
Jul  5 23:56:40.607: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01>
Jul  5 23:56:40.607: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01>enable
Jul  5 23:56:40.618: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01#
Jul  5 23:56:40.618: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01#configure terminal
Jul  5 23:56:40.633: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : Enter configuration commands, one per line.  End with CNTL/Z.
Jul  5 23:56:40.633: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01(config)#
Jul  5 23:56:40.633: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01(config)#interface GigabitEthernet0/1
Jul  5 23:56:40.744: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01(config-if)#
Jul  5 23:56:40.744: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01(config-if)#description SW02-Gi0/1
Jul  5 23:56:40.857: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01(config-if)#
Jul  5 23:56:40.857: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01(config-if)#end
Jul  5 23:56:40.867: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : OUT : SW01#
Jul  5 23:56:40.867: %HA_EM-6-LOG: Applet_LLDP_Port_Description_Add : DEBUG(cli_lib) : : IN  : SW01#write memory

That's it for today. Happy automation with Cisco EEM.

References
Location: Sundsvall, Sweden

Comments

Post a Comment

Popular posts from this blog

Fortigate firewall AAA Configuration for management with TACACS+ protocol and Cisco ISE

Image
In this blog I will write about how to implement AAA services in Fortigate firewalls using Cisco ISE as an authentication server with TACACS+ protocol. First good to know our limitations. AAA means - Authentication (supported by ISE TACACS+), Authorization (partially supported by ISE TACACS+; Fortigate implements Admin-Profiles; so what a user can/cannot do is defined locally in the firewall with "Admin Profiles"; ISE just instructs the firewall to allocate one of those profiles), Accounting (not supported; instead configure syslog to get accounting data, Fortigate firewalls generate syslog messages for configuration changes made by a user). Let's jump to our configuration. Our topology is very simple; one Fortigate firewall and Cisco ISE server connected with the same management network which looks like below -  01 - Network Topology We have a management network 192.168.199.0/24. The firewall is at  .230  and Cisco ISE is at  .49  IP address. We are running the lates
Read more

Stacking switches Part - VI (Dell OS10 VLT - Virtual Link Trunking)

Image
In this blog I will configure MLAG (multi chassis link aggregation) for Dell OS10 switches. As every other vendor choses a fancy name of their implementation of MLAG; Dell is no exception to this. Dell calls their implementation VLT - Virtual Link Trunking . Let's look at our network topology - Two Dell OS10 switches will run peering between them and will run VLT/MLAG between them. Two Debian 10 linux machine will be connected with LACP bonding with both switches. They will simulate the client connection. Our topology looks like below - 01 - Dell OS10 VLT Topology Let's look at some terminology before configuration - VLT peer – The two switches participating in VLT are peer to each other. In a VLT domain a maximum of two switches are allowed. VLT interconnect (VLTi) – VLTi synchronizes state information between VLT peers. It synchronizes layer 2 and layer 3 control-plane information. These state/control-plane information includes MAC table, ARP table, IPv6 neighbors etc. VLT
Read more

Arista EOS AAA configuration for management with TACACS+ protocol and Cisco ISE (Part I)

Image
Today I will write about AAA configuration (SSH authentication, authorization and accounting) for Arista EOS switches with Cisco ISE as authentication server and AAA protocol will be TACACS . Our topology is very simple. One Arista switch and One Cisco ISE server is running on the same network. Our topology looks like below - 01 - Network Topology We have a management network 192.168.199.0/24. The switch is at .134 and Cisco ISE is at .49 IP address. We are running the latest version of ISE - version 3.0. Let's define our AAA requirements - Arista switches comes with two pre defined user roles - "network-admin" and "network-operator" . We will use those two roles. Every user after successful authentication from tacacs server (ISE), will be authorized as either "network-admin" or "network-operator" role. In turn these two roles will determine read-write or read-only access to the switch. N ote: With TACACS, we can also do individual cl
Read more