SDB:Network UPS Tools

Jump to: navigation, search


Tested on openSUSE

Related articles


This article describes the required setup NUT (Network UPS Tools) for control and monitoring Uninterruptible Power Supply (UPS). As examples we consider UPS from MGE/EATON, but it is also applicable to configure any managed UPS models from other manufacturers.
NUT project

Introduction

UPS control concepts

There are two basic strategies for stopping a server when the input power source fails:

  1. Optimistic: Keep going as long, as possible with the available battery charge. When battery charge low, the server must be shutdown. When input power comes back, start the server, and hope, that input power will be available long enough to, at least partially, charge the battery. This is the most common strategy.
  2. Pessimistic: While the input power failure, the server must be shutdown as soon, as possible. When input power comes back, start the server, but to remain in readiness to shutdown again as soon, as the input power fails again. This strategy is preferred in case of bad power supply quality (very frequent input power outages), because allows you to save battery power sufficient to properly shutting down the server.
In this situation will be better to use a UPS, that supports delaying restart of the load until the batteries have charged to a certain minimum level.

Choice of an appropriate strategy depends on the quality of the mains supply and the capabilities of your UPS.

NUT architecture

"Simple" configuration

NUT has 3 levels, which comprise of the following components:

Drivers

  • provides interface with the hardware. See /usr/share/nut/driver.list

Server

  • UPS information server upsd, intermediary between the driver and clients components (man upsd).

Clients

"Advanced" configuration
  • Monitor — UPS monitor and shutdown controller, (man upsmon)
  • Tools — upsc, upscmd, upsrw.
  • CGI programs — scripts for Web administration: upsset.cgi, upsstats.cgi, upsimage.cgi (installed as a separate package).

Using and customizing the component depends from connect UPS to the protected equipment, see Monitoring diagrams

"Simple" configuration

One UPS protects one computer. In It used and configured components of all three levels - driver, upsd service and upsmon.

"Advanced" configuration

One UPS provides protection for multiple computers.

Master — control computer which has connected to the UPS communication, provides information about his condition all slave computers. The settings are similar to the system of "Simple" configuration except network settings access the service upsd.

Slave — managed computer receives information about UPS status from service upsd on "master" system. In it used and configured only the component of the third level upsmon.

Checking UPS support

Icon-warning.png
Warning: When selecting a suitable UPS should pay attention to the compatibility of drivers and equipment to execute commands turn off and turn on the UPS itself with specified delay.

For check your UPS support in NUT and what driver it needs, you can use the official Hardware compatibility list.


Installation

Software of project Network UPS Tools (NUT) is licensed the GPL version 2. With openSUSE 12.3 used version 2.6.5 which is distributed as an RPM package.

Using YaST

In accordance with the recommendations of the "YaST Software Management", locate and select the package nut, apply the changes to install it.

Using command line

sudo zypper in nut

Info about package NUT:

sudo zypper info nut
Loading repository data...
Reading installed packages...


Information for package nut:
----------------------------
Repository: openSUSE-12.3-1.7
Name: nut
Version: 2.6.5-4.1.1
Arch: x86_64
Vendor: openSUSE
Installed: Yes
Status: up-to-date
Installed Size: 4.0 MiB
Summary: Network UPS Tools Core (Unbreakable Power Supply Monitoring)
Description:
Core package of Network UPS Tools.
Network UPS Tools is a collection of programs which provide a common
interface for monitoring and administering UPS hardware.
Detailed information about supported hardware can be found in
/usr/share/doc/packages/nut.

Configuration

Setting up the equipment (ups)

Icon-warning.png
Warning: When configuring connection of UPS, possibly wrong or premature switching power off, so during debugging, do not connect configurable computer via it.

As an example, consider the setting for two UPS from EATON connected via USB. However, nut, also supports UPS which are connected via COM-port or network (SNMP).

Connect UPS to a computer and find it in the list of USB devices.

lsusb
...
Bus 003 Device 005: ID 0592:0002 Powerware Corp. UPS (X-Slot)
...

Select appropriate driver, it may depend by the type of communication connection:

grep -i -E 'powerware.*5110' /usr/share/nut/driver.list

where powerware — UPS manufacturer, 5110 — model.

"Powerware"     "ups"   "5"     "PW5110"        ""      "bcmxcp_usb"

/etc/ups/ups.conf

Specify name of the UPS which will provide the protection. Also, specify the required driver and its settings.

For "EATON Powerware 5110 UPS" (More detail in man bcmxcp_usb):

[PW5110]
        driver = bcmxcp_usb
        port = auto
        desc = "UPS:EATON Powerware 5110 USB"
        shutdown_delay = 90
  • shutdown_delay = 90 — The number of seconds that the UPS should wait between receiving the shutdown command and actually shutting off. Default value is 120 seconds.

For "EATON Ellipse ECO 1600 FR CSS"(More detail in man usbhid-ups):

[Eaton-66781]
        driver = usbhid-ups
        port = auto
        desc = "UPS:EATON Ellipse ASR 1500 USB"
        vendorid = 0463
        offdelay = 30
        ondelay = 40
  • offdelay = n — Where n seconds is the time after the kill power command is sent, and before the UPS is turned off. Default value is 20 seconds.
  • ondelay = n — Where n seconds is the delay for the UPS to switch on in case the power returns after the kill power command had been sent but before the actual switch off. This ensures the machines connected to the UPS are, in all cases, rebooted after a power failure. Default value is 30 seconds. Remember that ondelay must be greater than offdelay.

Data server configuration (upsd)

/etc/ups/nut.conf

The general section of the NUT configuration determines which part of the NUT is to be started.

The standalone mode calls for a local only configuration, with 1 UPS protecting the local system.

MODE = standalone

This implies starting the 3 NUT layers (driver, upsd and upsmon) and the matching configuration files.

If multiple, then for the control system (driver, upsd and upsmon):

MODE = master

And for all clients (only upsmon):

MODE = slave

/etc/ups/upsd.conf

Specify which ports the upsd daemon will listen on. By default, openSUSE 12.3 enables IPv6 so specify:

# Make sure this matches the IPv6 setting in YaST -> Network Devices ->
# Network Settings -> Global options -> IPv6 Protocol Settings
# LISTEN <address> [<port>]
LISTEN 127.0.0.1 3493
LISTEN ::1 3493

If you disenable IPv6 using YaST -> Network Devices -> Network Settings -> Global options -> IPv6 Protocol Settings, then specify only:

LISTEN 127.0.0.1 3493
# IPv6 turned off

In multiple host configuration for master specify:

LISTEN 0.0.0.0 3493

For slave host this config not used, because upsd daemon is not running.

Note: The logic of file upsd.conf is subtle. If no LISTEN addresses are specified in upsd.conf, the compiled in defaults are used. As soon as you specify one or more LISTEN directives, upsd will only listen to the specified addresses.

Data server security

Version: openSUSE 11.4 and older The information in this section applies only to versions prior to openSUSE 12.1

Restriction of access to upsd daemon:

ACL localhost 127.0.0.1/32
ACL localnet 192.168.0.0/24
ACL all 0.0.0.0/0
ACCEPT localhost localnet
REJECT all

Version: openSUSE 12.1 and newer In newer versions of openSUSE NUT uses TCP-Wrappers to provide controlled access to the daemon upsd. For configure use /etc/hosts.allow


/etc/ups/upsd.users

This file sets the permissions for upsd — the UPS network daemon. Users are defined here, are given passwords, and their privileges are controlled here too.

[upsmaster]
        password = master_sekret
        upsmon master

[upsslave]
        password = slave_sekret
        upsmon slave

You may want something better than master_sekret and slave_sekret as a password.

User upsslave required only for configurations with multiple hosts.

/etc/hosts.allow

/etc/hosts.deny

This file is not used and is not present in my configuration. All denials are specified in hosts.allow.

NUT-Monitor configuration (upsmon)

/etc/ups/upsmon.conf

This file’s primary job is to define the systems that upsmon will monitor and to tell it how to shut down the system when necessary. For standalone mode and master system in multiple host configuration:

#MONITOR PW5110@localhost 1 upsmaster master_secret master

For slave system in multiple host configuration:

#MONITOR PW5110@upsserver 1 upsslave slave_secret slave

Other options by default:

MINSUPPLIES 1
SHUTDOWNCMD "/sbin/shutdown -h +0"
POLLFREQ 5
POLLFREQALERT 5
HOSTSYNC 15
DEADTIME 15
POWERDOWNFLAG /etc/killpower
RBWARNTIME 43200
NOCOMMWARNTIME 300
FINALDELAY 5
NOTIFYFLAG ONLINE   SYSLOG+WALL
NOTIFYFLAG COMMOK   SYSLOG+WALL
NOTIFYFLAG SHUTDOWN SYSLOG+WALL
NOTIFYFLAG ONBATT   SYSLOG+WALL
NOTIFYFLAG LOWBATT  SYSLOG+WALL
NOTIFYFLAG FSD      SYSLOG+WALL
NOTIFYFLAG NOCOMM   SYSLOG+WALL
NOTIFYFLAG COMMBAD  SYSLOG+WALL

Pessimist strategy script

/etc/ups/upssched.conf

/usr/sbin/upssched-cmd

Icon-warning.png
Warning: Make sure UPS name matches in /etc/ups/ups.conf

How to shutdown the UPS unit

Version: 12.2 and more OpenSUSE 12.2 and 12.3 have fully embraced systemd and systemctl. This may be the cause of a bug which has appeared in the power-fail shutdown management in 12.2. The fix described here is only needed for openSUSE releases 12.2 and 12.3. It is not needed for previous releases.

The race

The UPS delayed shutdown order to be sent to the UPS unit is not an immediate order. If it were, the UPS would turn off immediately causing power loss and probable loss of data in the server. The order used is for a a "delayed shutdown", with a default delay of 20 seconds. This is often sufficient to allow the server to shutdown before the UPS, but there is no guarantee of this. The UPS and the server race to shutdown. We hope the server wins. Some simple ascii art shows the timing for a basic test rig and the scripted shutdown. The X's show the race window:

 systemctl               system
 poweroff                 halt
     |                 |XXXX|
     0   2   4   6   8 |10  12  14  16  18  20  22  24  26  28  30  32 secs
                       |                                       |
                   upsdrvctl                                  UPS
                   shutdown                                 shutdown

More ascii art shows the timing for the same test rig and the service unit shutdown:

 systemctl               system
 poweroff                 halt
     |     |XXXXXXXXXXXXXXXX|
     0   2 | 4   6   8  10  12  14  16  18  20  22  24  26  28  30  32 secs
           |                                       |
        upsdrvctl                                 UPS
        shutdown                                shutdown

The script solution has the advantage of reducing the race window from 9 to 3 secs. Note that the same security can also be achieved by the service unit by setting offdelay = 26 in /etc/ups/ups.conf.

You should chose one or the other of these solutions — not both.

Using a bash script

systemd provides drop-in directories for scripts which should be executed as late as possible during a system shutdown. /usr/lib/systemd/system-shutdown is for a user's scripts, and /lib/systemd/system-shutdown is for system scripts. Although shutting down a UPS unit is clearly a system matter and not a user matter, I use the /usr/lib/... directory since my script is not part of an officially distributed product.

systemd detects automatically that a script in one of these drop-in directories needs to be executed. There is no need to enable the script.

Place the following script ups-delayed.shutdown in /usr/lib/systemd/system-shutdown.

Version: openSUSE 12.2 and newer Beginning with release 12.2 there is no /usr/lib... so you have to choose /lib/...

#! /bin/bash
# ups-delayed.shutdown  Delayed turn off for the UPS unit.
# Needed for automatic system restart when wall power returns.
UPSDRVCTL_BIN=/usr/lib/ups/driver/upsdrvctl
$UPSDRVCTL_BIN shutdown

To test the shutdown, use command

systemctl poweroff

There is no record on the console or in /var/log/messages of the execution of the script. You have to check that the UPS really does shut down.

Using a systemd service unit

Create a systemd service unit to shut down the UPS. The unit is the new file /etc/systemd/system/ups-delayed-shutdown.service

# ups-delayed-shutdown.service

# OpenSUSE 12.3 powerdown: server shutdown requires UPS shutdown
# This service fixes a problem with openSUSE 12.3 NUT.
# Background: When wall power fails the system and the UPS must
# be shut down in such a way that when wall power returns,
# the server is turned back on again without human intervention.

# For this process to work, the UPS unit must receive a
# delayed-UPS-shutdown signal which will put it into a state where
# it will restart the server when wall power returns. A UPS unit 
# which is simply left beeping will not restart its load when wall 
# power returns.

[Unit]
Description=Initiate delayed UPS shutdown
Before=umount.target
DefaultDependencies=no

[Service]
Type=oneshot
ExecStart=/usr/lib/ups/driver/upsdrvctl shutdown

[Install]
WantedBy=final.target

This service must be enabled. Use command

systemctl --system reenable /etc/systemd/system/ups-delayed-shutdown.service
ln -s '/etc/systemd/system/ups-delayed-shutdown.service' '/etc/systemd/system/final.target.wants/ups-delayed-shutdown.service'

To test the shutdown, use command

systemctl poweroff

A line similar to the following should be written in /var/log/messages

 2013-08-11T13:20:20.557069+02:00 pinta systemd[1]: Starting Initiate delayed UPS shutdown...

and you should see the main box powered off, followed a few seconds later by the UPS power off. 10 seconds later, since wall power is still present, the server should restart automatically.


Debug and troubleshoot

UPS connection issues

For diagnosis of a USB connection need to test subsystem UDEV.

Check UDEV rules:

grep 0592 /etc/udev/rules.d/52-nut-usbups.rules
ATTR{idVendor}=="0592", ATTR{idProduct}=="0002", MODE="664", GROUP="daemon"
ATTR{idVendor}=="0592", ATTR{idProduct}=="0004", MODE="664", GROUP="daemon"

UDEV monitor allows to analyze events occuring when connecting and disconnecting the UPS:

udevadm monitor
monitor will print the received events for:
UDEV - the event which udev sends out after rule processing
KERNEL - the kernel uevent

Disconnect the cable from the UPS

KERNEL[1766.454306] remove   /devices/pci0000:00/0000:00:1d.1/usb3/3-2/3-2:1.0 (usb)
KERNEL[1766.455629] remove   /devices/pci0000:00/0000:00:1d.1/usb3/3-2 (usb)
UDEV  [1766.456081] remove   /devices/pci0000:00/0000:00:1d.1/usb3/3-2/3-2:1.0 (usb)
UDEV  [1766.457179] remove   /devices/pci0000:00/0000:00:1d.1/usb3/3-2 (usb)

Connect the cable to the UPS

KERNEL[1770.321286] add      /devices/pci0000:00/0000:00:1d.1/usb3/3-2 (usb)
KERNEL[1770.324196] add      /devices/pci0000:00/0000:00:1d.1/usb3/3-2/3-2:1.0 (usb)
UDEV  [1770.331446] add      /devices/pci0000:00/0000:00:1d.1/usb3/3-2 (usb)
UDEV  [1770.338683] add      /devices/pci0000:00/0000:00:1d.1/usb3/3-2/3-2:1.0 (usb)

Also, may be useful to analyze the problem testing of processing rules Udev:

udevadm test /devices/pci0000:00/0000:00:1d.1/usb3/3-2/3-2:1.0

Testing the driver


Appendix

Configuration files

The configuration files are to be found in directory /etc/ups/. Certain run-time files are in /var/lib/ups/ and there is a useful script at /usr/sbin/.

Files provided by NUT
Permissions Owner Group Name Purpose Use
drwxr-xr-x root root /etc/ups/ Directory
-rw-r--r-- root root /etc/ups/hosts.conf CGI programs Not used
-rw-r--r-- root root /etc/ups/nut.conf General
-rw-r--r-- root root /etc/ups/ups.conf Define UPSs
-rw------- upsd root /etc/ups/upsd.conf Access control Use if no IPv6
-rw------- upsd root /etc/ups/upsd.users Network users
-rw------- upsd root /etc/ups/upsmon.conf Monitoring
-rw-r--r-- root root /etc/ups/upssched.conf Commands
-rw-r--r-- root root /etc/ups/upsset.conf Web access Not used
drwx------ upsd daemon /var/lib/ups/ Directory
-rwxr--r-- upsd daemon /usr/sbin/upsd Binary
-rwxr--r-- upsd daemon /usr/sbin/upsmon Binary
-rwxr--r-- upsd daemon /usr/sbin/upssched Binary
-rwxr-xr-x root root /bin/upssched-cmd Sample script Not used
Custom script for NUT
Permissions Owner Group Name Purpose Use
-rwxr--r-- upsd daemon /usr/sbin/upssched-cmd Pessimist strategy script
Choice of UPS shutdown techniques required by NUT
Permissions Owner Group Name Purpose Use
-rw-r--r-- root root /usr/lib/systemd/system-shutdown/ups-delayed.shutdown UPS shutdown script Choose one of these
-rw-r--r-- root root /etc/systemd/system/ups-delayed-shutdown.service UPS shutdown script Choose one of these
TCP wrapper files used by NUT
Permissions Owner Group Name Purpose Use
-rw-r--r-- root root /etc/hosts.allow TCP wrappers access control
-rw-r--r-- root root /etc/hosts.deny TCP wrappers access control Not used

See also

Related articles

External links