Network Device Configuration Templating with YAML, Jinja2, Python and Ansible

It was long time I had some fun writing some programming code. The other day I was thinking about one of my daily assignment as a network engineer - configuring network devices. Most of the time when we configure a network device -  we start from a predefined configuration template which is copied from an existing device (the actual configuration); then open it in a text editor and do a search and replace operation to change the configuration according to the new device and it's configurations.

What is wrong with this approach? First of all most of the time there is a risk that we will forget to change something; because we are doing a search and replace in a text editor and we have not made a list what are the things/configurations only need to change. Secondly there is no reusability; every time I need to open this text file and make changes by hand.

Let's think for a moment. What are the things we actually change - when we configure a new device from an existing configuration? Things like - interface IP address, interface descriptions, vlan, routing information, switch port configuration etc. are actually changed. Other information like - logging, ssh server, aaa settings etc. remain the same most of the time.

Now programming comes to our rescue. From above we already know some configuration items are changing (which are variable items) and some items are not changing (which are static items). Isn't it wonderful if we can just define a configuration file with variables. Then when creating a new configuration just provide the value of the variables and in return we get the full configuration for the new device. No more search and replace; we will just work with items which are changing from device to device and one template file will generate configuration for all of our devices. Welcome to the world of configuration templating with YAML, Jinja2, Python and Ansible !!!


What is configuration templating

I will try to cover the basic things which will allow most of us do basic templating. We need three things for generating configuration by using templates -

  • YAML data file - A file containing configuration items which are changed during a configuration task. In other words a file which contains our variables written in YAML syntax.

  • Jinja template file - A file which contains the whole configuration of a network device. The difference between a jinja template and normal device configuration file is that - in template we will use variables, looping, conditional etc. programming constructs. These constricts will be replaced when we generate a device configuration.
  • Configuration generator - This is the secret sauce that binds together a YAML data source & Jinja template and generates the actual configuration according to our need. It our case we can use either a python script or ansible playbook to achieve the same result.

Installing the prerequisite

We are using Linux - Debian 10 as our development environment. We need the following commands to install the necessary packages -

apt update
apt install python3-pip ansible
pip3 install Jinja2 PyYAML

That's it - we are done with installation.


Let's do some templating

We have a Cisco switch; we will generate it's configuration using templating. 

Let's have a look at our YAML data file which contains the values of the different items; which change during configuration of a Cisco IOS switch. The file name is - my-data.yml.

---
HostName: Test-Sw-01

Vlans:
  - VlanId: 10
    VlanName: MGMT
  - VlanId: 20
    VlanName: ACCESS
 
SwInterfaces:
  - IntName: Gi0/1
    IntDescription: "To-PC01"
    IntTrunk: false
  - IntName: Gi0/2
    IntDescription: "To-PC02"
    IntTrunk: false
  - IntName: Gi0/3
    IntDescription: "To-PC03"
    IntTrunk: false
  - IntName: Gi0/10
    IntDescription: "Uplink-GW"
    IntTrunk: true  

MgmtVlan: 10

MgmtVlanIp: 192.168.10.5

DefaultGw: 192.168.10.1

In the data YAML file; green color represents a variable/configuration item and aqua color represents it's value.

Then we will look at our Jinja template file which references our variables defined in the YAML file. The file name is - my-template.j2.

hostname {{ HostName }}
!
logging origin-id hostname
logging console debugging
logging buffered debugging
logging monitor debugging
logging trap debugging
!
{% for Vlan in Vlans -%}
vlan {{ Vlan['VlanId'] }}
  name {{ Vlan['VlanName'] }}
!
{% endfor -%}
!
{% for Interface in SwInterfaces -%}
interface {{ Interface['IntName'] }}
  description {{ Interface['IntDescription'] }}
  no shutdown
  {% if Interface['IntTrunk'] -%}
  switchport mode trunk
  {% else -%}
  switchport mode access
  switchport access vlan {{ Vlans[1]['VlanId'] }}
  {% endif -%}
!
{% endfor -%}
!
interface vlan {{MgmtVlan}}
   ip address {{ MgmtVlanIp}} 255.255.255.0
   no shutdown
!
ip default-gateway {{DefaultGw}}
!

We can see that in the jinja template we are referencing the variables defined (aqua color) in the yaml data file and some programming constructs like loop, if..else to do the repetitive tasks (creating multiple vlans) and do some verification (port - access or trunk). 

Then we will use either Ansible playbook or a Python script to change the values of the variables defined in the template file from the data file.

When using Ansible; our playbook YAML looks like below. The file name is my-playbook.yml.

---
- hosts: localhost
  vars_files:
    - my-data.yml
  tasks:
    - name: Create a Cisco Switch Configuration
      template:
        src: my-template.j2
        dest: my-output.txt

If we run the following linux command; then we will get a file named my-output.txt in the current directory which contains the actual configuration of our Cisco switch.

$ ansible-playbook my-playbook.yml 
PLAY [localhost] *********************************

TASK [Gathering Facts] ***************************
ok: [localhost]

TASK [Create a Cisco Switch Configuration] ***********************************************
ok: [localhost]

PLAY RECAP *************************************
localhost                  : ok=2    changed=1    unreachable=0    failed=0

$ cat my-output.txt 
hostname Test-Sw-01
!
logging origin-id hostname
logging console debugging
logging buffered debugging
logging monitor debugging
logging trap debugging
!
vlan 10
  name MGMT
!
vlan 20
  name ACCESS
!
!
interface Gi0/1
  description To-PC01
  no shutdown
  switchport mode access
  switchport access vlan 20
!
interface Gi0/2
  description To-PC02
  no shutdown
  switchport mode access
  switchport access vlan 20
!
interface Gi0/3
  description To-PC03
  no shutdown
  switchport mode access
  switchport access vlan 20
!
interface Gi0/10
  description Uplink-GW
  no shutdown
  switchport mode trunk
!
!
interface vlan 10
   ip address 192.168.10.5 255.255.255.0
   no shutdown
!
ip default-gateway 192.168.10.1
!

We can see that all the variables in the template file is replaced with their corresponding values from the data file (like hostname, gateway, interface description etc.). From now on we will just change the values in the YAML data file and get configuration for our different switches according to our needs. No more needing of touching the actual configuration file.

The ansible part can also be done with a Python script. The script file name is my-generate.py.

#!/usr/bin/env python3

import sys
import yaml
from jinja2 import Environment, FileSystemLoader
from jinja2 import TemplateNotFound

# Initialize the Jinja2 environment to load templates
# from the current directory
env = Environment(loader=FileSystemLoader('.'))
template = env.get_template(sys.argv[1])

# Load the context YAML file into a Python dictionary
with open(sys.argv[2], 'r') as datafile:
    context = yaml.full_load(datafile)

# Render the template and save the resulting document into a file
rendered_template = template.render(**context)
config_file = open(sys.argv[3], "w+")
config_file.write(rendered_template)
config_file.close()

We can run this script as follows -

$ ./my-generate my-template.j2 my-data.yml my-output.txt

Both the Ansible and the Python script will output the same thing. The advantage with ansible is that you do not need to have any python programming knowledge.


Note: 
  • I am attaching all the example files in a zip achieve. As both YAML and Python are very sensitive to indentation; it is recommended to download the zip file and run the codes from there.
  • As huge shout out to PacketLife.net blog. I have actually used the python code from his blog.

Reference

1. Downloadable files shown in the blog - templating-examples.zip.

Location: Sundsvall, Sweden

Comments

Post a Comment

Popular posts from this blog

Fortigate firewall AAA Configuration for management with TACACS+ protocol and Cisco ISE

Image
In this blog I will write about how to implement AAA services in Fortigate firewalls using Cisco ISE as an authentication server with TACACS+ protocol. First good to know our limitations. AAA means - Authentication (supported by ISE TACACS+), Authorization (partially supported by ISE TACACS+; Fortigate implements Admin-Profiles; so what a user can/cannot do is defined locally in the firewall with "Admin Profiles"; ISE just instructs the firewall to allocate one of those profiles), Accounting (not supported; instead configure syslog to get accounting data, Fortigate firewalls generate syslog messages for configuration changes made by a user). Let's jump to our configuration. Our topology is very simple; one Fortigate firewall and Cisco ISE server connected with the same management network which looks like below -  01 - Network Topology We have a management network 192.168.199.0/24. The firewall is at  .230  and Cisco ISE is at  .49  IP address. We are running...
Read more

Stacking switches Part - VI (Dell OS10 VLT - Virtual Link Trunking)

Image
In this blog I will configure MLAG (multi chassis link aggregation) for Dell OS10 switches. As every other vendor choses a fancy name of their implementation of MLAG; Dell is no exception to this. Dell calls their implementation VLT - Virtual Link Trunking . Let's look at our network topology - Two Dell OS10 switches will run peering between them and will run VLT/MLAG between them. Two Debian 10 linux machine will be connected with LACP bonding with both switches. They will simulate the client connection. Our topology looks like below - 01 - Dell OS10 VLT Topology Let's look at some terminology before configuration - VLT peer – The two switches participating in VLT are peer to each other. In a VLT domain a maximum of two switches are allowed. VLT interconnect (VLTi) – VLTi synchronizes state information between VLT peers. It synchronizes layer 2 and layer 3 control-plane information. These state/control-plane information includes MAC table, ARP table, IPv6 neighbors etc. VLT ...
Read more

Network device configuration management with Oxidized (Basic)

Image
I always like to work with open source tools. As a network engineer, we need to maintain a device configuration management system where we take backup of configuration for different networking devices like routers, switches, firewalls etc. We want to have our backups ready so that we can restore a device configuration when a device fails or replace with new one or for auditing purpose to track recent configuration changes. Today I will talk about such a open source tools named - Oxidized. Interested readers can have a look at it's GitHub page . It is a network configuration management tool with support for diverse networking equipments. Oxidized is developed with programming language Ruby. It is very configurable, extensible and can be integrated with management tools like Librenms . I will do a two part blog about Oxidized. In this part I will cover basic setup of Oxidized and start taking configuration backup of networking equipments. Network Topology Our topology l...
Read more