UPS control with NUT, PFSense+ and Zabbix

Introduction

In this article I am going to explain how I got control of my UPS (Uninterruptible Power Supply) installation using NUT via PFSense+¹ so I can monitor and manage the UPS from the main router in my HomeLab. In addition I will show how I have configured Zabbix with a new NUT template for monitoring, alarm generation and long term metrics for my UPS.

The UPS I am using with my HomeLab is a PowerWalker VI 3000 RLE², a line-interactive UPS from BlueWalker GmbH. This UPS is connected via USB to the main router, both the main router and the KVM server together with all VMs and the main Desktop computer in my office are configured to shutdown automatically in a proper way if the battery charge goes under 10% or the left runtime on battery is less than 5min.

NUT (Network UPS Tools)

As the “Network UPS Tools” project³ says on its pages, “NUT is a collection of programs which provide a common interface for monitoring and administering UPS, PDU and SCD hardware”.

This collection of programs are divided in three main areas:

NUT drivers

They are used to communicate with the hardware and provide support for specific UPS models. They implement protocols and port specifications and make it possible for the upsd server program to communicate with the hardware.

You have multiple types of drivers supporting different protocols and connection types, SNMP, USBHID, Megatec/Q1 protocol, bridges to PowerMan, Apcupsd daemons, etc. Depending on your hardware and how it is connected to the machine running NUT you will have to use one or the other.

To configure the drivers you must use this configuration file:

• ups.conf: In this file you will have to define a section per UPS/PDU that the machine is responsible for managing. Details and documentation in UPS.CONF(5)⁴

NUT server

The NUT server is called upsd. This program is responsible for passing data between the drivers and the client programs via the network.

To configure the NUT server you can use these two configuration files:

• upsd.conf - upsd uses this file to control access to the server and set some other miscellaneous configuration values. Details and documentation in UPSD.CONF(5)⁵
• upsd.users - Administrative user definitions for NUT upsd. Details and documentation in UPSD.USERS(5)⁶

NUT clients

They are used to communicate with the NUT server.

One of the most important clients is the upsmon⁷ program, it provides an essential feature, “safe shutdowns when the power fails”. Details and documentation in UPSMON(8)⁷

NUT and PFSense+

The PFSense+ router in my HomeLab was the right place to control my UPS. The router should be the last component going down when shuting down the infrastructure and it is usually the one with the longest uptime.

The UPS/NUT infrastructure I wanted to implement looks like this:

• The PowerWalker VI 3000 RLE UPS is connected via an USB cable to the Main PFSense+ router.
• The Main PFSense+ router, the KVM server and the Desktop PC are connected with Power cables to the UPS.
• The Main PFSense+ router is responsible for managing the UPS. It uses the usbhid-ups driver to communicate with the UPS, it runs the upsd server and uses the upsmon client in master (primary) mode.
• The KVM server and the Desktop PC are connected to the PFSense+ router via TCP/IP and using the upsmon client in slave (secondary) mode.

PFSense+ has support for NUT with the system package “nut / sysutils / 2.8.0_2”. After installing this package via "System > Package Manager > Available Packages" I could configure NUT for the described infrastructure at "Services > UPS".

Under the "UPS Settings" tab on that page I could define the type of UPS (USB), the name I wanted to give to the UPS (PowerWalker-VI-3000-RLE) and the type of driver (usbhid-ups) it used.

After this and under the "Advanced settings" section on that page I had to define some extra parameters to implement the infrastructure I was planing to use. This part was a bit tricky because you have sections for additional configuration for upsmon.conf, ups.conf, upsd.conf and upsd.users but not information or documentation about what “Additional configuration” means at this moment, additional to what?

Luckily we can check via "Diagnostics > Command Prompt" what these files contain at this time. They can be found under /usr/local/etc/nut/ and a simple cat gives us the information generated and saved by default by PFSense/NUT so we can find out the “additional configuration” we need in our system.

#  cat /usr/local/etc/nut/upsmod.conf

MONITOR PowerWalker-VI-3000-RLE 1 local-monitor x2x2x2x2x2x2x2x2x2x2 master
SHUTDOWNCMD "/sbin/shutdown -p +0"
POWERDOWNFLAG /etc/killpower

# cat /usr/local/etc/nut/ups.conf

[PowerWalker-VI-3000-RLE]
driver=usbhid-ups
port=auto

# cat /usr/local/etc/nut/upsd.conf

LISTEN 127.0.0.1
LISTEN ::1

# cat /usr/local/etc/nut/upsd.users

[admin]
password=x3x3x3x3x3x3x3x3x3xj
actions=set
instcmds=all

[local-monitor]
password=x2x2x2x2x2x2x2x2x2x2
upsmon master

Well, this was informative. At first it looked like everything needed to start getting some information from the UPS locally was in place. The only problem was that the "UPS Status" tab at "Services > UPS" failed to get any information from the UPS.

After searching the Internet I found a message in a forum saying that you need this parameter user = root as “Additional configuration” for ups.conf for NUT to work with PFSense+. After adding this parameter, NUT started getting information from the UPS without problems.

The only things left to configure now were:

• To define an user in upsd.users I could use to connect with the clients running in the KVM server and my Desktop PC.
• Configure upsd.conf to listen via the router IP (10.100.100.1) in addition to the localhost interface so it could be available from the network.
• Add some extra parameters to upsmod.conf so I could send some notifications to SYSLOG for logging purposes.
• Configure nut.conf and upsmod.conf on the KVM server and my Desktop PC so they could comunicate with the NUT server running in the router and send some notifications locally to SYSLOG for logging purposes.

This is what I did under "Services > UPS / Advanced settings" to implement these last changes:

Under "Additional configuration lines for upsmon.conf" in PFSense+ I defined these extra lines:

FINALDELAY 10
HOSTSYNC 15

NOTIFYCMD "/usr/local/sbin/upssched"

NOTIFYMSG ONLINE "UPS State[ONLINE] - Normal state"
NOTIFYMSG ONBATT "UPS State[ONBATT] - On battery"
NOTIFYMSG LOWBATT "UPS State[LOWBATT] - Battery low"
NOTIFYMSG FSD "UPS State[FSD] - Starting 'Forced Shutdown'"
NOTIFYMSG COMMOK "UPS State[COMMOK] - Communication restored"
NOTIFYMSG COMMBAD "UPS State[COMMBAD] - Communication lost"
NOTIFYMSG SHUTDOWN "UPS State[SHUTDOWN] - Shutting down"
NOTIFYMSG REPLBATT "UPS State[REPLBATT] - Replace battery"

NOTIFYFLAG ONLINE SYSLOG
NOTIFYFLAG ONBATT SYSLOG
NOTIFYFLAG LOWBATT SYSLOG
NOTIFYFLAG FSD SYSLOG
NOTIFYFLAG COMMOK SYSLOG
NOTIFYFLAG COMMBAD SYSLOG
NOTIFYFLAG SHUTDOWN SYSLOG
NOTIFYFLAG REPLBATT SYSLOG

Under "Additional configuration lines for ups.conf" in PFSense+ I defined this extra line:

user = root

Under "Additional configuration lines for upsd.conf" in PFSense+ I defined this extra line:

LISTEN 10.100.100.1 3493

And under "Additional configuration lines for upsd.users" in PFSense+ I defined these extra lines:

[remote-monitor]
password = x1x1x1x1x1x1x1x1x1x1
upsmon slave

These are the changes I did on/etc/nut/nut.conf and /etc/nut/upsmod.conf on the KVM server and my Desktop PC so they could comunicate with the NUT server running in the router and send some notifications locally to SYSLOG for logging purposes:

root@server:/etc/nut# cat /etc/nut/nut.conf

MODE=netclient

root@server:/etc/nut# cat /etc/nut/upsmon.conf

MONITOR PowerWalker-VI-3000-RLE@10.100.100.1 1 remote-monitor x1x1x1x1x1x1x1x1x1x1 slave
SHUTDOWNCMD "/sbin/shutdown -P +0"
POWERDOWNFLAG /etc/killpower

NOTIFYCMD "/sbin/upssched"

NOTIFYMSG ONLINE "UPS State[ONLINE] - Normal state"
NOTIFYMSG ONBATT "UPS State[ONBATT] - On battery"
NOTIFYMSG LOWBATT "UPS State[LOWBATT] - Battery low"
NOTIFYMSG FSD "UPS State[FSD] - Starting 'Forced Shutdown'"
NOTIFYMSG COMMOK "UPS State[COMMOK] - Communication restored"
NOTIFYMSG COMMBAD "UPS State[COMMBAD] - Communication lost"
NOTIFYMSG SHUTDOWN "UPS State[SHUTDOWN] - Shutting down"
NOTIFYMSG REPLBATT "UPS State[REPLBATT] - Replace battery"

NOTIFYFLAG ONLINE SYSLOG
NOTIFYFLAG ONBATT SYSLOG
NOTIFYFLAG LOWBATT SYSLOG
NOTIFYFLAG FSD SYSLOG
NOTIFYFLAG COMMOK SYSLOG
NOTIFYFLAG COMMBAD SYSLOG
NOTIFYFLAG SHUTDOWN SYSLOG
NOTIFYFLAG REPLBATT SYSLOG

The two important parameters on the client side are MODE=netclient and MONITOR PowerWalker-VI-3000-RLE@10.100.100.1 1 remote-monitor x1x1x1x1x1x1x1x1x1x1 slave on these configuration files.

After these changes I had NUT up and running and my infrastructure configured to shutdown automatically in a proper way if the power supply disappears and my UPS doesn’t have enough battery charge.

Zabbix monitoring using NUT

You can implement some level of monitoring with upsmon and send notifications via multiple channels with upssched. I have done this in all my servers and receive information about different UPS status changes via SYSLOG.

But in addition I wanted to implement some real monitoring of several parameters from the UPS so I could have some graphs and proper alarms depending of the status reported by the UPS.

PFSense+ has support for the Zabbix-agent with the system package “zabbix-agent6 / net-mgmt / 1.0.6”. After installing the package via "System > Package Manager > Available Packages" I could configure the zabbix-agent at "Services > Zabbix Agent 6"

After this, I created a simple Zabbix Template and linked it to the PFSense+ server host I created in my Zabbix server. The template I created can be found under https://github.com/rafaelma/zabbix-template-ups-nut ⁸. For this template to work with PFSense+ you have to define these lines in the "Advanced features->Users Parameters" under "Services > Zabbix Agent 6":

UserParameter=get.ups.nut.output[*],[ -f /usr/local/bin/upsc ] && /usr/local/bin/upsc $2@$1 2>/dev/null
UserParameter=get.ups.nut.autodiscovery[*],[ -f /usr/local/bin/upsc ] && /usr/local/bin/upsc -l $1 2>/dev/null

This is the default dashboard I get in Zabbix for my UPS using this template:

This template defines some alerts/triggers that will be generated if the UPS is “On Battery” and discharging, “On Line” and charging the battery, if the load or charge of the UPS are over/under some limits or if the battery should be replaced. This is a good starting point with a minimum monitoring of the UPS.

Well, this is all for today, enjoy your PFSense+ with NUT if you have it, and if not, I recommend you to get an UPS and install PFSense+ as your router/firewall with NUT to protect your infrastructure when the power supply disappears.

Footnotes