Monitoring Certificates with Zabbix

Abstract

There's nothing more embarrasing than running a public website with expired certificates. Huge red warning message pop-up all over the place, users are concerned and might even avoid your site. The admin frantically trying to find out how to renew the certificate, something they either never done before or forgot how to do it. The same applies to mail servers that offer TLS encryption for SMTP or IMAP.

Monitoring Certificates with Zabbix

There's nothing more embarrassing than running a public website with expired certificates. Huge red warning message pop-up all over the place, users are concerned and might even avoid your site. The admin frantically trying to find out how to renew the certificate, something they either never done before or forgot how to do it. The same applies to mail servers that offer TLS encryption for SMTP or IMAP.

Basic Idea

The basic idea is to monitor the validity of the certificate that a server offers as a item in Zabbix. The item returns the days how long the certificate is still valid. Triggers inform you if the remaining time falls below a certain limit.

Searching in the internet I found an blog entry and a script that solves the problem for web servers. The basic idea is to use the openssl command to retrieve the certificate of the web server, to print out the dates of the certificate and to parse the notAfter line. The number of that line can be compared to the actual date. The command line of the script is:

END_DATE="$(openssl s_client $S_CLIENT_OPTS </dev/null 2>$TMP | \
  openssl x509 -dates -noout | \
  sed -n 's/notAfter=//p')"

Checking other Protocols

Reading the man pages of openssl you can see that it offers a variety of other protocols, not only HTTP. So it is easy to extend the original script to check SMTP, IMAP, POP3, FTP, and XMPP servers for the validity of its certificates. The protocol to be checked is passed to the script in the option -P and a case command adjusts the S_CLIENT_OPTS that the openssl executes.

External Script

I also used the feature to check the issuer of the certificate. Putting it all together the script looks a follows:

#!/bin/bash
#
# Authors:
#       Michael Schwartzkopff <ms@sys4.de>
#       Marc Schiffbauer <m@sys4.de>
#

trap clean_exit EXIT

clean_exit() {
  [[ $TMP && -f $TMP ]] && rm -f "$TMP"
}

debug() {
  [[ $DEBUG -gt 0 ]] && echo "$*"
}

debugexec() {
  [[ $DEBUG -gt 0 ]] && "$*"
}

error() {
  echo "ERROR: $*"
}

die() {
  error "$*"
  exit 1
}

usage() {
  cat <<-EOF
    Usage:
    $(basename $0) [options]

    -H <hostname>         Hostname to connect to. Default: localhost
    -P <protocol>         Protocol to use (SSL, SMTP, IMAP, POP3, FTP, XMPP). Default: SSL
    -d                    Turn on debug mode
    -i                    Get certificate issuer instead of days left until certificate will expire
    -p <port>             Port to connect to. Defaults: 443 (SSL), 25 (SMTP), 143 (IMAP),
                          110 (POP3), 21 (FTP), 5269 (XMPP)

  EOF
  exit 0
}

while getopts "idhH:p:P:" opt; do
  case "$opt" in
    H) HOST="$OPTARG";;
    P) PROTO="$OPTARG";;
    d) DEBUG=1; set -x;;
    i) WHAT="ISSUER";;
    p) PORT="$OPTARG";;
    *) usage;;
  esac
done

# set default values
HOST=${HOST:-localhost}
PROTO=${PROTO:-SSL}
WHAT=${WHAT:-TIME}

debug "Checking protocol $PROTO on ${HOST}:${PORT}"

case $PROTO in
  SSL)
    PORT=${PORT:-443}
    S_CLIENT_OPTS=" -host $HOST -port $PORT -showcerts"
  ;;
  SMTP)
    PORT=${PORT:-25}
    S_CLIENT_OPTS="-connect $HOST:$PORT -starttls smtp"
  ;;
  IMAP)
    PORT=${PORT:-143}
    S_CLIENT_OPTS="-connect $HOST:$PORT -starttls imap"
  ;;
  POP3)
    PORT=${PORT:-110}
    S_CLIENT_OPTS="-connect $HOST:$PORT -starttls pop3"
  ;;
  FTP)
    PORT=${PORT:-21}
    S_CLIENT_OPTS="-connect $HOST:$PORT -starttls ftp"
  ;;
  XMPP)
    PORT=${PORT:-5269}
    S_CLIENT_OPTS="-connect $HOST:$PORT -starttls xmpp"
  ;;
  *)
    die "Unknown protocol"
  ;;
esac

debug "Certificate:"
debugexec "openssl s_client $S_CLIENT_OPTS </dev/null 2>$TMP"

case $WHAT in
  TIME)
    TMP="$(mktemp)"
    END_DATE="$(openssl s_client $S_CLIENT_OPTS </dev/null 2>$TMP | openssl x509 -dates -noout | sed -n 's/notAfter=//p')"
    NOW="$(date '+%s')"
    if [[ $END_DATE ]]; then
      SEC_LEFT="$(date '+%s' --date "${END_DATE}")"
      echo $((($SEC_LEFT-$NOW)/24/3600))
    else
      die "openssl error: $(cat $TMP)"
    fi
  ;;
  ISSUER)
    TMP="$(mktemp)"
    openssl s_client $S_CLIENT_OPTS </dev/null 2>$TMP | openssl x509 -issuer -noout | sed -n 's/.*CN=//p'
  ;;
  *)
    die "BUG: unknown WHAT value: $WHAT"
  ;;
esac

exit 0

Using a external check for this item will not hurt the performance of the Zabbix server since the checks are executed only once a day.

Integration into Zabbix

After copying the script to a location where the Zabbix server expects its external scripts (config option ExternalScripts) and allowing the Zabbix server to execute the script you can define a item to check the validity of the certificate of a web server of the type External check:

$ zext_ssl_cert.sh[-d,{HOST.CONN}]

If you want the check the validity of the certificate of a SMTP mail service you just pass some more options to the External check item:

$ zabbix_check_cert.sh[-H,{HOST.CONN},-P,SMTP]

You can look at the Zabbix SSL Cert template - it is part of the original dev.aperto.fr Zabbix extensions - to get the idea of the items and the triggers. Of course, it is only usable for web services, but you can easily extend it for the other servers.

Michael Schwartzkopff, 06. August 2013