You Are Here: Greyghost > Features Page > Stupid Things To Do With APC UPS Network Management Cards
Stupid Things To Do With APC UPS Network Management Cards (and lots more besides!)
American
Power Conversion's higher end UPS models, as well as those intended for
use in large datacenter environments, are usually equipped with a
special expansion slot known as a SmartSlot. A number of different
network management cards, as well as various environmental monitors,
out of band management, remote dial in and signaling multiplexer cards
can also be installed. In more recent times, APC's product offerings
for use in a SmartSlot have become fewer in number and what is left has
become less capable. The newer SmartSlot accessories will generally
work in an older UPS, while older accessories won't work in a newer UPS
due to communications protocol changes.
Most APC UPSes with SmartSlot
capability have just the one slot. While most SmartSlot modules can
function by themselves, a few of them must be used with something else in order to do anything
productive. One such device is the AP9612 environmental monitor and dry contact card.
That's why APC has made a few
different enclosures that allow for the use of multiple SmartSlot cards
with a UPS. There is a single unit enclosure (the APC AP9600) and
another that accepts up to three SmartSlot cards (an AP9604, available
in white or black). I recently came across a few AP9600 enclosures that
had been slumbering in their packages since 1998 or so. And while I've
seen a lot of it, I'm constantly amazed when a company pays good money
for something and never ends up actually putting it to use.
Different SmartSlot cards have differing levels of "priority", in terms
of how much (if any) control they exert over an attached UPS. APC had a
document describing this and in time I might post it here, since they
seem to be purging much of the support material for their older
products from ready view. (Note that all of this has relevance only in
the world of the older "APC Smart" or "UPSlink" Smart-UPS products.
There is not to my knowledge a multi-slot enclosure for the newer
Microlink UPS products, and not all the cards available to this
generation of product were updated and carried over to the newer
Microlink products. I do discuss Microlink in further detail down
below, though only insofar as to acknowledge its existence and the
discovery in 2016 that any Microlink UPS is also a Smart UPS at heart.
I don't know if that's still true here in late 2018.)
I'm pretty sure that APC doesn't
intend for you to install two SmartSlot cards with similar
functionality. None of their documentation mentions doing this, or for
that matter, telling you that you can't.
And so it was that I just had to try it.
This is what happens when you have an
AP9600 enclosure, an AP9605 SMNP management card, a Smart-UPS 750 with
an already configured AP9617 card and you're a nerd who just so
happens to be bored on a Saturday night. A few minutes with a
screwdriver brought me to this point.
An AP9617 card is resident in the
UPS's internal SmartSlot, while the much older AP9605 is resident in
the AP9600 expansion chassis.
Never you mind that wiring fault light. Grounded outlets are for sissies. 
To my great surprise, this
configuration seemed to work perfectly well. Above, in the browser, is the internal web server
(link may not work for readers outside the US) that's running on the
AP9617. Below, the AP9605 responds in the only way it can: via telnet.
Both cards are capable of
reporting on and controlling the UPS when requested to do so. It's less
clear what the serial console might do when connected to a computer
with terminal emulation software. APC makes mention of particular cards
having differing priority levels. I'm pretty sure that SmartSlot cards
providing much the same function would have identical priority levels,
at least in theory. Perhaps the card which is electrically "closer" to
the serial port would take precedence?
It is still possible to attach a
smart signalling cable to the expansion chassis and monitor the UPS
directly from an attached computer if you so desire.
I don't yet know, and due to the lack of hardware sufficient to test
the idea, probably will not try to see what two identical network
management cards would do.
And Now, The Hat Trick
The AP9600 enclosure has a
connection for an external power source providing 24 volts DC, with
center positive polarity. I believe the voltage itself probably isn't
too critical, as multiple regulators are present inside the AP9600.
Anything 12 volts or higher ought to work just fine, maybe a little
more to overcome the minimum voltage difference that might be imposed
by the regulator. Of course, I can assume no responsibility for the
actions of others, so on your own head be the results, especially if
you fry your SmartSlot card or AP9600 enclosure. In the picture below,
mine is running perfectly well from a 20 volt Lenovo laptop power
adapter.
To make a long story short, this lets
you power up the AP9600 and whatever you have installed inside it
without a UPS attached. I really thought that this would cause the card
to fail its boot process or report a status issue, but it did not!
(Incidentally, this card's previous owner, GTE/BBN, changed only the default user name and not the password.
I guess they thought that would be sufficient to keep it secure, and
maybe that was true if the card ran in a fully controlled environment.
I'd love to know where or in what it was originally used, though the
likelihood of that ever happening is miniscule.)
The management interface is functional, but you can't do much of anything besides configuring the various network parameters.
"If It Ain't Broke...You're Not Trying!"
Around 2009 or so, APC changed the communications protocol utilized
by
their Smart-UPS products from UPSLink to Microlink. To the
consternation of many, APC and Schneider Electric have utterly refused
to document any aspect of Microlink's design or implementation. Most if
not all of these UPS products are readily identified by the presence of
a backlit, alphanumeric LCD and various operation buttons, instead of
the usual LEDs and on/off buttons. Models using Microlink are primarily
categorized under a model number beginning with "SMT". There are also
SMX, SURTD and SMC models (more in the next paragraph).
There is also a line of slightly cut down Smart UPS "SMC" models that
have the simpler backlit LCD panel also seen in larger Back-UPS
products of the time. I suppose one could think of them as a "boy
trapped in a man's body". Despite the limited display, these have a
true sinewave inverter just like their SMT brethren. They lack a
SmartSlot (though perhaps the artifacts to add one might exist -- I've
not checked).
APC and Schneider Electric offered two long delayed responses to users
clamoring for a way to use their UPS with third party and open source
monitoring software: first came the AP9620 legacy communications card
(of which more below) and next in line was a firmware update for
selected UPS models and revisions that enabled the use of the Modbus
protocol. When enabled and where available, Modbus allowed third party
software to again communicate with the UPS. Legacy software programs
were confined to use of the AP9620 card. As of this writing, none of
these approaches applies to any SMC Smart-UPS model without a network
port for cloud-based monitoring (and perhaps not even then).
With these newer UPS models came new management cards known as the
AP9630 and AP9631. These are collectively known as the "Network
Management Card 2" and have what is probably a code name of "Rhodes 2"
printed on the circuit board. Exactly what "Rhodes" is or was, beyond a
code name, is unknown to me. There's also what appears to be a shrunken
version of the APC network management card, known as a "Mini Rhodes". (This person probably knows what Rhodes indicates.) These later management cards and their
firmware work with both new (Microlink) and old (UPSLink) protocols and
UPS hardware. An AP9635 card came later, offering a built in modem and dedicated Modbus ports.
These newer cards have muchfaster
microcontrollers at their heart and in the case of the AP9631 or
AP9635,
two USB ports that are used for firmware upgrades. Maybe someday
they'll enable the use of these ports for storing log data as well.
That would be very handy. I was received in what I perceived to be a
cool manner when I suggested enabling such with the USB ports. (The
microcontroller might well be clocked faster on these new management
cards, and apparently it has an encryption co-processor that's pulled
into service whenever HTTPS or SSH are enabled on the management card.
The previous AP9617/18/19 cards were the first to support encrypted
communications of any kind for management purposes, but they're
glacially slow with it enabled.)
You can't install an old network management card directly into a new
Microlink Smart-UPS model. There are two keyways both in the SmartSlot
and the rear bezel of compatible accessories. These prevent old cards
from being inserted, implying a major incompatibility. I decided to
find out if anything dramatic or otherwise would happen by
removing the rear bezel from my AP9605 board and installing it into an
SMT750 Smart-UPS.
I connected power with some trepidation. After all, a battery can
deliver essentially unlimited current for a short period of time and
that might happen if there were a major electrical incompatibility. To
my surprise, the card came online and was accessible from the network.
What it didn't do was provide any status or control of the SMT750. The
AP9605's software simply behaved as though no UPS were present.
The same experiment, later tried with an AP9617, resulted in a failure
message being logged: "UPS: Platform is not supported by AP9617/18/19.
Call APC tech support for more information." I strongly suspect this is
an artificial limitation, easily dispensed with if APC would update the
AP9617/9618/9619 firmware.
Now you know what happens when you install an old SmartSlot card into a
new UPS, even if you didn't care or ask. It simply doesn't work. The
older card sees nothing and a newer one tells you how naughty you've
been.
A compatibility board known as the AP9620 exists for Microlink UPSes.
It fits into the SmartSlot and gives them a veneer of compatibility
with older monitoring software and peripherals that only support
UPSLink. It is said not to be a perfect solution as it implements a
number of features differently or not at all. It may also disable some
of the more advanced features on a newer UPS. It's also not supposed to
be compatible with older management cards operating in external
enclosures, although my own tests indicate that it mostly works and
will communicate with an AP9617 card. The AP9620 even supplies 24 volt
power to operate an external management card enclosure! Even so, it's
still not possible to perform certain actions properly. For example,
turning the UPS off results in its coming right back on again.
Smart-UPS models featuring Microlink are sometimes also multilingual, and support use of the Mpdbus
protocol. Older Microlink-only models can sometimes be upgraded with
revised firmware. (If your SMT-series UPS is running firmware 8.0 or
later, it can be upgraded to support Modbus. For everyone else there is
the AP9620 card.)
Late Breaking News!
It turns out that the story of MicroLink (APC/Schneider's new and thus
far undocumented binary communication protocol) is much more
interesting than was first believed. Some research
into the subject (not by me) reveals that at least in the case of an
SMT2200 UPS, there is more than one microcontroller at play. One
microcontroller is used to operate the UPS display and control panel,
while the other monitors the UPS hardware itself. This second
microcontroller is the same thing as seen on every third generation
Smart-UPS ever made, and it communicates using the classic UPSLink (or
APC Smart) protocol! This capability isn't actually brought out for the
end user or their software, and attempting to use it (as demonstrated
in the linked video) results in the control panel not working. In fact,
you can see the control panel trying to place the UPS microcontroller
into its "smart" mode whenever it can't communicate with said
microcontroller.
I believe that APC encrypts their firmware for those UPS models where
upgrades exist, mainly by virtue of the file extension being .ENC. I
haven't tried to figure out how. I'd guess it's a relatively simple
cipher, if you could even call it that.
To make a long story short, at least as of this writing in July 2016, every MicroLink UPS is ALSO a UPSLink (or "APC Smart") UPS. It's just that the end user doesn't get to access the UPSlink communications.
December 2018 minor update: there's some suggestion that people have managed to decipher small parts of the Microlink protocol.
If you don't care about the control panel working, it is not
tremendously difficult to tap and utilize the UPSlink communications
protocol with such a unit.
The whole thing really looks to me like an awful hack, and a ridiculous
"stack" of different microcontrollers. I guess there's enough profit
margin in these things to make such an approach to their design
feasible. A suggestion has been made that perhaps the engineering
talent behind the original Smart-UPS 2G/3G design is no longer around,
and no one at APC understands the existing design well enough to
modernize it in a neater, cleaner and more professional way. I have no
idea if it's true. I would be surprised if it was.
Odds & Ends
It's very common to find older APC
UPS models being retired from service as a result of battery failure.
Genuine APC batteries cost a small fortune, sometimes almost as much as
or more than the UPS they came in! People who are less concerned about
having a "genuine APC" battery pack can build their own relatively
easily and much more cheaply. That's all you usually have to do to get
an old UPS back on its feet. Some of the various SmartUPS models should
ideally have their battery constant value reset
to ensure that the correct runtime and battery state-of-charge
information will be available. A few could stand replacement capacitors
and some models have a design flaw.
Older APC SNMP UPS network management
cards like the AP9605 shown on this page can usually be had for less
than $10, shipped. They are rather primitive compared to the newer
cards. Even so, all popular network UPS management tools support SNMP
quite readily. If you do need more capability, the AP9617 is also
pretty cheap these days. Unfortunately, newer SmartUPS models that have
informational display panels instead of simple LEDs communicate with a
different protocol and don't support the older management
cards. Newer management cards can handle both protocols and work fine
in older UPS models. They're just not all that cheap...yet.
The AP9605 utilizes a 13 MHz (!!)
80C188 CPU and a National Semiconductor DP8390 NIC on a chip. It
consists of two sandwiched together printed circuit boards -- one for
the NIC related stuff and the other for the processor and ROMs. Rather
surprisingly it is equipped with user-flashable ROMs. While APC no
longer mentions the AP9605 on their web site, the latest firmware
release is still available on their FTP site.
(Amazingly, the AP9603 Token Ring version is still mentioned as of this
writing in October 2015. You still won't actually find any downloads or
manuals linked from its support page.)
APC indicates that some versions of
the AP9603 Token Ring SNMP management card will require a hardware
upgrade in order to accept firmware upgrades to version 3.0. It is
unclear what this upgrade consists of or how one would get it. Perhaps
it is larger ROM chips or flashable ROMs? Every attempt I've made to get an AP9603 has failed.
Placing a SmartSlot Network
Management Card in an external module such as the AP9600 makes it quite
possible to take the card around to various SmartUPS units and check in
on them.
Unlike APC's later management cards,
the AP9605 is capable of communicating with any supported device having
a SmartSlot. Later management cards require you to change the firmware
bundle, which at 2400 bits per second over a serial link, is really rather
thoroughly unexciting.
Firmware updates for the AP9605 can be done over
serial, telnet (with the aid of TFTP) and seemingly XMODEM as well.
Even with TFTP, update speed is still very thoroughly unexciting and
you don't get any meaningful status indications as the telnet
connection drops almost immediately afterwards.
When installed in or attached to a SmartUPS 750 from February 2008,
some of the telnet and SNMP configuration fields displayed protracted
strings consisting of extended ASCII garbage. This behavior vanished
without the UPS connected. Could it be a partial incompatibility? A conflict between the two cards?
(The former I can't prove, although the displayed garbage disappeared
when the card was operating by itself. The latter doesn't seem terribly
likely, with the card operating all by its lonesome in a Smart-UPS
1500. One possibility that remains is that of an incompatibility caused
by the radical age differences of the UPSes I have as compared to this
management card. I also view this as rather unlikely, since the
configuration data in question resides in memory on the card itself and
not within the UPS. In any case, the card works despite the display of
gibberish in these fields. Screenshot to come...when I get around to
it.)
Before installing or removing any SmartSlot card, no matter how it happens to be attached, you're supposed to remove
power from the UPS completely. Cards that can be hot swapped have
longer contact fingers in a few positions as compared to those that
don't support hot installation or swapping. To be safe, power down any
attached loads
safely and turn the UPS off from its front panel controls. Then
disconnect the UPS itself from power. At that point you can either wait
a few minutes for the UPS to turn itself the rest of the way off, or
you may press and hold the OFF (0) button until you hear a click.
Lights on the SmartSlot card (if it has any) should go out. At that
point you can safely remove the SmartSlot card from your UPS.
If you are planning to reinstall the
SmartSlot card, don't be too surprised if the UPS powers itself up upon
insertion of the card! I've had a few units that did so, and while I
don't think it is in any way harmful if they do, it can certainly be
startling. It is rather less amusing when the SmartSlot ribbon cable
and its connector break loose from the clips holding them to the
SmartSlot "box". Then you have to open up the UPS (being very careful
of the hazardous circuitry and making SURE it is very disconnected from
all sources of power, including the batteries) and put it back -- or
hold it in place while you insert the SmartSlot card.
Should you have one of the newer
AP9618 or AP9619 network management cards, you may have noticed that a
"humidity" value appears in the Environmental sensing area. APC's
bundled temperature probes don't actually report the humidity value
because they lack the needed sensor. Yet the AP9618/19 firmware does
support collection of humidity data IF you can get a sensor with the
humidity module inside it. Thse could be ordered from APC as a separate
part back in the day, and they came with the AP9612 Measure-UPS board.
Humidity measurement is not possible on the current generation "NMC2"
card, model numbers AP9631 or 9632. (Or so I thought inititally: my attention was later brought to this item,
which might work to provide a humidity reading on the later APC
management cards. Still, it kind of begs the question why APC just
didn't include the more capable sensor with the AP9618/19 and AP9631 in
the first place.)
If the battery in your AP9617/18/19
or "U" series card runs down, it's a simple matter to replace it with a
new CR2032 type. The older AP9605 has no real time clock to worry about
(or even a software clock!) and the AP9606 uses "SnapHat" modules that
clip over a permanently installed timekeeping IC. Once again, AP963x
cards take a significant step back and use a soldered CR2032 (maybe a
2025, it's been a while since I've seen one) battery. As if it weren't
bad enough for these NMC2 series cards to have a permanently installed
battery, said battery is positioned right over the top of a surface
soldered flashable ROM. If the battery ever leaks, there goes your card!
APC employees have stated that the permanently installed battery has to
do with product safety regulations. I rather seriously doubt this.
There might be more "odds and ends"
to come in time. For now this is quite long and rambly enough, don't
you think? (Now that I've added random stuff discussing several
generations of product, this has become pretty messy.)
Yet More Diddling Around! (convert an AP9604S into a regular AP9604)
Martin, "aintbigaintclever",
from Youtube deduced how to convert an AP9604S enclosure for use with
an APC Silcon UPS to a regular AP9604 for use with other APC UPS
products. His e-mail is presented below as written:
"I’ve just converted an APC AP9604S (the Silcon variant of the AP9604
Triple Chassis) into an AP9604. Doing this involved working out the
schematics for both an AP9604 rev. 6 (borrowed from work) and the
AP9604S rev. 7 (from a scrapped Silcon, again from work). This enabled
me to work out the schematics for an AP9604 rev. 7 and how to convert
one to the other.
Additional components required are:-
2 x 3K3 resistor
4 x 1N4148 diode
1 x 10nF polyester capacitor
1 x KN2907A transistor (or a BC327 fitted back-to-front)
All other components are salvaged from elsewhere on the board where they’re no longer needed.
I stumbled across your website whilst trying to find out about firmware
for the AP9604S. If you want the schematics I’ve created, they’re
attached in PNG format. Video of the conversion will be uploaded to
YouTube later this week."
Martin's schematics are available here, here and here.
He retains any and all rights to his work, including copyright. You'll
need to ask his permission before using these schematics in your own
work, if you plan to make that work publicly accessible. I believe this is his video discussing the conversion process.
As always, your thoughts, comments, suggestions and additions are very welcome! Click here to contact
me!
Go Back >
Copyright © 2015-2018 by William R.
Walsh. Some rights reserved. Review the terms and conditions
information linked from the top level page of this server to determine
your rights to reproduce and reuse this material.. Last updated with
minor changes on 12/17/2018. Preivously updated: 07/01/2016.