Device Discovery with netbox

netbox is a cool asset management system that serves as source of truth. Configurations are derived from netbox and deployed with ansible or saltstack. But how does netbox get the information about all devices? In this article I want to discuss a method to discover new devices based on known devices in netbox.

Motivation

Of course, you can add all devices to netbox manually. That is the hard way and it is time consuming. The alternative is a discovery mechanism that crawls through your network and tells netbox about yet unknown devices. netbox offers the onboarding process for this purpose.

My appraoch is to ask netbox for all devices that are already known. I then ask all these known devices for devices in their neighborhood. The Local Link Discovery Protocol (LLDP) is the key. All devices talk to each other, so all devices know about their direct neighborhood.

Known Devices in netbox

Let’s start with the querey of know devices. After I’ve imported some libraries and initialized the variables all devices from netbox are stored in the devices variable.

import socket
import pynetbox
import json
import requests

from napalm.base import get_network_driver
from pysnmp.hlapi import *

url="http://127.0.0.1:8001"
token="123456"

nb = pynetbox.api(url, token=token)
devices = nb.dcim.devices.all()
knowndevices = [device.name for device in devices]
todoDevices = []

ob_url = 'http://127.0.0.1:8001/api/plugins/onboarding/onboarding/'
headers = {'Accept': 'application/json', 'Authorization': 'Token 123456'}

for device in devices:
  todoDevices.append({'name': device.name, 'ip': device.primary_ip.address.split('/')[0], 'os': device.platform.name.split('_')[1]})

print ("Known devices: ", knowndevices)

Asking netbox I import the knowndevices and rearrange it to a list of devices my program still has to ask about their neighbors.

The Core of the Discovery

The main part of the programm recursively loops over the devices that are left to ask:

while todoDevices:

  device = todoDevices[0]
  todoDevices.remove(device)

  lldpDevices = []
  getLldpNeigh (device, lldpDevices)

  for lldpDevice in lldpDevices:
    if lldpDevice['name'] not in knowndevices:
      print ('Found new device {} attached to {}'.format (lldpDevice['name'], device['name']))

      if lldpDevice['ip'] and lldpDevice['os']:
        payload = {'site': 'home', 'ip_address': lldpDevice['ip']}
        r = requests.post ( ob_url, headers=headers, data=payload )
        todoDevices.append({'name': lldpDevice['name'], 'ip': lldpDevice['ip'], 'os': lldpDevice['os']})

It uses the function getLldgNeigh() to get a list of neighbors. It compares every neighbor to the list of known devices. If it finds a yet unknown device it triggers the onboarding of the new device in netbox and adds it to the list of devices still to ask.

Getting the LLDP Information

The function getLldpNeigh() finally asks the device about its neighbors. My program uses the napalm utiliy to standardize the output of the various devices.

def getLldpNeigh ( device, lldpdevices ):
  name = device['name']
  ip = device['ip']
  os = device['os']
  driver = get_network_driver(os)
  print ('Checking device {} on IP {} with driver {}'.format(name, ip, os))

  dev = driver(hostname=name, username='user', password='password')
  dev.open()
  lldpNeighbors = dev.get_lldp_neighbors_detail()
  dev.close()

  for entry in lldpNeighbors:
    neighbor = lldpNeighbors[entry][0]['remote_system_name']
    name = neighbor.split('.')[0]

    remoteOS = lldpNeighbors[entry][0]['remote_system_description']
    if 'ios' in remoteOS.lower():
      os = 'ios'
    elif 'junos' in remoteOS.lower():
      os = 'junos'
    else:
      os = ''

    try:
      ip = socket.gethostbyname(name)
    except Exception as e:
      # print ('Error resolving host name {}'.format(name))
      ip = ''
    else:
      ip = ip

    lldpdevices.append ({'name': name, 'ip': ip, 'os': os})

My method is not limited to the LLDP neighborhood, but also can include the ARP table, the MAC address table or the routing table of devices. Just add the according functions.

As you might have noticed, I am not a good python programmer. Improvements are welcome! Please mail me at ms@sys4.de.

Michael Schwartzkopff, 15 Nov 2020