You Are Here: Greyghost > Features Page > Stupid Things To Do With APC UPS Network Management Cards
Stupid Things To Do With APC UPS Network Management Cards (and lots more besides!)

American Power Conversion's higher end UPS models, as well as those intended for use in large datacenter environments, are usually equipped with a special expansion slot known as a SmartSlot. A number of different network management cards, as well as various environmental monitors, out of band management, remote dial in and signaling multiplexer cards can also be installed. In more recent times, APC's product offerings for use in a SmartSlot have become fewer in number and what is left has become less capable. The newer SmartSlot accessories will generally work in an older UPS, while older accessories won't work in a newer UPS due to communications protocol changes.

Most APC UPSes with SmartSlot capability have just the one slot. While most SmartSlot modules can function by themselves, a few of them must be used with something else in order to do anything productive. One such device is the AP9612 environmental monitor and dry contact card.

That's why APC has made a few different enclosures that allow for the use of multiple SmartSlot cards with a UPS. There is a single unit enclosure (the APC AP9600) and another that accepts up to three SmartSlot cards (an AP9604, available in white or black). I recently came across a few AP9600 enclosures that had been slumbering in their packages since 1998 or so. And while I've seen a lot of it, I'm constantly amazed when a company pays good money for something and never ends up actually putting it to use.

Different SmartSlot cards have differing levels of "priority", in terms of how much (if any) control they exert over an attached UPS. APC had a document describing this and in time I might post it here, since they seem to be purging much of the support material for their older products from ready view. (Note that all of this has relevance only in the world of the older "APC Smart" or "UPSlink" Smart-UPS products. There is not to my knowledge a multi-slot enclosure for the newer Microlink UPS products, and not all the cards available to this generation of product were updated and carried over to the newer Microlink products. I do discuss Microlink in further detail down below, though only insofar as to acknowledge its existence and the discovery in 2016 that any Microlink UPS is also a Smart UPS at heart. I don't know if that's still true here in late 2018.)

I'm pretty sure that APC doesn't intend for you to install two SmartSlot cards with similar functionality. None of their documentation mentions doing this, or for that matter, telling you that you can't.

And so it was that I just had to try it.

This is what happens when you have an AP9600 enclosure, an AP9605 SMNP management card, a Smart-UPS 750 with an already configured AP9617 card and you're a nerd who just so happens to be bored on a Saturday night. A few minutes with a screwdriver brought me to this point.

Two APC Network Management Cards attached to one Smart-UPS

An AP9617 card is resident in the UPS's internal SmartSlot, while the much older AP9605 is resident in the AP9600 expansion chassis.

Never you mind that wiring fault light. Grounded outlets are for sissies. Green Smiley Emoticon

Two APC network management cards in the same UPS and on the same network, with their management interfaces displayed.

To my great surprise, this configuration seemed to work perfectly well. Above, in the browser, is the internal web server (link may not work for readers outside the US) that's running on the AP9617. Below, the AP9605 responds in the only way it can: via telnet. Both cards are capable of reporting on and controlling the UPS when requested to do so. It's less clear what the serial console might do when connected to a computer with terminal emulation software. APC makes mention of particular cards having differing priority levels. I'm pretty sure that SmartSlot cards providing much the same function would have identical priority levels, at least in theory. Perhaps the card which is electrically "closer" to the serial port would take precedence?

It is still possible to attach a smart signalling cable to the expansion chassis and monitor the UPS directly from an attached computer if you so desire.

I don't yet know, and due to the lack of hardware sufficient to test the idea, probably will not try to see what two identical network management cards would do.

And Now, The Hat Trick

The AP9600 enclosure has a connection for an external power source providing 24 volts DC, with center positive polarity. I believe the voltage itself probably isn't too critical, as multiple regulators are present inside the AP9600. Anything 12 volts or higher ought to work just fine, maybe a little more to overcome the minimum voltage difference that might be imposed by the regulator. Of course, I can assume no responsibility for the actions of others, so on your own head be the results, especially if you fry your SmartSlot card or AP9600 enclosure. In the picture below, mine is running perfectly well from a 20 volt Lenovo laptop power adapter.

To make a long story short, this lets you power up the AP9600 and whatever you have installed inside it without a UPS attached. I really thought that this would cause the card to fail its boot process or report a status issue, but it did not!

AP9605 management card running without an attached UPS.

(Incidentally, this card's previous owner, GTE/BBN, changed only the default user name and not the password. I guess they thought that would be sufficient to keep it secure, and maybe that was true if the card ran in a fully controlled environment. I'd love to know where or in what it was originally used, though the likelihood of that ever happening is miniscule.)

APC AP9605 telnet management interface with no UPS attached.

The management interface is functional, but you can't do much of anything besides configuring the various network parameters.

"If It Ain't Broke...You're Not Trying!"

Around 2009 or so, APC changed the communications protocol utilized by their Smart-UPS products from UPSLink to Microlink. To the consternation of many, APC and Schneider Electric have utterly refused to document any aspect of Microlink's design or implementation. Most if not all of these UPS products are readily identified by the presence of a backlit, alphanumeric LCD and various operation buttons, instead of the usual LEDs and on/off buttons. Models using Microlink are primarily categorized under a model number beginning with "SMT". There are also SMX, SURTD and SMC models (more in the next paragraph).

There is also a line of slightly cut down Smart UPS "SMC" models that have the simpler backlit LCD panel also seen in larger Back-UPS products of the time. I suppose one could think of them as a "boy trapped in a man's body". Despite the limited display, these have a true sinewave inverter just like their SMT brethren. They lack a SmartSlot (though perhaps the artifacts to add one might exist -- I've not checked).

APC and Schneider Electric offered two long delayed responses to users clamoring for a way to use their UPS with third party and open source monitoring software: first came the AP9620 legacy communications card (of which more below) and next in line was a firmware update for selected UPS models and revisions that enabled the use of the Modbus protocol. When enabled and where available, Modbus allowed third party software to again communicate with the UPS. Legacy software programs were confined to use of the AP9620 card. As of this writing, none of these approaches applies to any SMC Smart-UPS model without a network port for cloud-based monitoring (and perhaps not even then).

With these newer UPS models came new management cards known as the AP9630 and AP9631. These are collectively known as the "Network Management Card 2" and have what is probably a code name of "Rhodes 2" printed on the circuit board. Exactly what "Rhodes" is or was, beyond a code name, is unknown to me. There's also what appears to be a shrunken version of the APC network management card, known as a "Mini Rhodes". (This person probably knows what Rhodes indicates.) These later management cards and their firmware work with both new (Microlink) and old (UPSLink) protocols and UPS hardware. An AP9635 card came later, offering a built in modem and dedicated Modbus ports.

These newer cards have muchfaster microcontrollers at their heart and in the case of the AP9631 or AP9635, two USB ports that are used for firmware upgrades. Maybe someday they'll enable the use of these ports for storing log data as well. That would be very handy. I was received in what I perceived to be a cool manner when I suggested enabling such with the USB ports. (The microcontroller might well be clocked faster on these new management cards, and apparently it has an encryption co-processor that's pulled into service whenever HTTPS or SSH are enabled on the management card. The previous AP9617/18/19 cards were the first to support encrypted communications of any kind for management purposes, but they're glacially slow with it enabled.)

You can't install an old network management card directly into a new Microlink Smart-UPS model. There are two keyways both in the SmartSlot and the rear bezel of compatible accessories. These prevent old cards from being inserted, implying a major incompatibility. I decided to find out if anything dramatic or otherwise would happen by removing the rear bezel from my AP9605 board and installing it into an SMT750 Smart-UPS.

AP9605 SmartSlot SMNP accessory installed without its rear bezel in a Smart-UPS SMT750 model.

I connected power with some trepidation. After all, a battery can deliver essentially unlimited current for a short period of time and that might happen if there were a major electrical incompatibility. To my surprise, the card came online and was accessible from the network. What it didn't do was provide any status or control of the SMT750. The AP9605's software simply behaved as though no UPS were present.

The same experiment, later tried with an AP9617, resulted in a failure message being logged: "UPS: Platform is not supported by AP9617/18/19. Call APC tech support for more information." I strongly suspect this is an artificial limitation, easily dispensed with if APC would update the AP9617/9618/9619 firmware.

Now you know what happens when you install an old SmartSlot card into a new UPS, even if you didn't care or ask. It simply doesn't work. The older card sees nothing and a newer one tells you how naughty you've been.

A compatibility board known as the AP9620 exists for Microlink UPSes. It fits into the SmartSlot and gives them a veneer of compatibility with older monitoring software and peripherals that only support UPSLink. It is said not to be a perfect solution as it implements a number of features differently or not at all. It may also disable some of the more advanced features on a newer UPS. It's also not supposed to be compatible with older management cards operating in external enclosures, although my own tests indicate that it mostly works and will communicate with an AP9617 card. The AP9620 even supplies 24 volt power to operate an external management card enclosure! Even so, it's still not possible to perform certain actions properly. For example, turning the UPS off results in its coming right back on again.

Smart-UPS models featuring Microlink are sometimes also multilingual, and support use of the Mpdbus protocol. Older Microlink-only models can sometimes be upgraded with revised firmware. (If your SMT-series UPS is running firmware 8.0 or later, it can be upgraded to support Modbus. For everyone else there is the AP9620 card.)

Late Breaking News!

It turns out that the story of MicroLink (APC/Schneider's new and thus far undocumented binary communication protocol) is much more interesting than was first believed. Some research into the subject (not by me) reveals that at least in the case of an SMT2200 UPS, there is more than one microcontroller at play. One microcontroller is used to operate the UPS display and control panel, while the other monitors the UPS hardware itself. This second microcontroller is the same thing as seen on every third generation Smart-UPS ever made, and it communicates using the classic UPSLink (or APC Smart) protocol! This capability isn't actually brought out for the end user or their software, and attempting to use it (as demonstrated in the linked video) results in the control panel not working. In fact, you can see the control panel trying to place the UPS microcontroller into its "smart" mode whenever it can't communicate with said microcontroller.

I believe that APC encrypts their firmware for those UPS models where upgrades exist, mainly by virtue of the file extension being .ENC. I haven't tried to figure out how. I'd guess it's a relatively simple cipher, if you could even call it that.

To make a long story short, at least as of this writing in July 2016, every MicroLink UPS is ALSO a UPSLink (or "APC Smart") UPS. It's just that the end user doesn't get to access the UPSlink communications.

December 2018 minor update: there's some suggestion that people have managed to decipher small parts of the Microlink protocol.

If you don't care about the control panel working, it is not tremendously difficult to tap and utilize the UPSlink communications protocol with such a unit.

The whole thing really looks to me like an awful hack, and a ridiculous "stack" of different microcontrollers. I guess there's enough profit margin in these things to make such an approach to their design feasible. A suggestion has been made that perhaps the engineering talent behind the original Smart-UPS 2G/3G design is no longer around, and no one at APC understands the existing design well enough to modernize it in a neater, cleaner and more professional way. I have no idea if it's true. I would be surprised if it was.

Odds & Ends

It's very common to find older APC UPS models being retired from service as a result of battery failure. Genuine APC batteries cost a small fortune, sometimes almost as much as or more than the UPS they came in! People who are less concerned about having a "genuine APC" battery pack can build their own relatively easily and much more cheaply. That's all you usually have to do to get an old UPS back on its feet. Some of the various SmartUPS models should ideally have their battery constant value reset to ensure that the correct runtime and battery state-of-charge information will be available. A few could stand replacement capacitors and some models have a design flaw.

Older APC SNMP UPS network management cards like the AP9605 shown on this page can usually be had for less than $10, shipped. They are rather primitive compared to the newer cards. Even so, all popular network UPS management tools support SNMP quite readily. If you do need more capability, the AP9617 is also pretty cheap these days. Unfortunately, newer SmartUPS models that have informational display panels instead of simple LEDs communicate with a different protocol and don't support the older management cards. Newer management cards can handle both protocols and work fine in older UPS models. They're just not all that cheap...yet.

The AP9605 utilizes a 13 MHz (!!) 80C188 CPU and a National Semiconductor DP8390 NIC on a chip. It consists of two sandwiched together printed circuit boards -- one for the NIC related stuff and the other for the processor and ROMs. Rather surprisingly it is equipped with user-flashable ROMs. While APC no longer mentions the AP9605 on their web site, the latest firmware release is still available on their FTP site. (Amazingly, the AP9603 Token Ring version is still mentioned as of this writing in October 2015. You still won't actually find any downloads or manuals linked from its support page.)

APC indicates that some versions of the AP9603 Token Ring SNMP management card will require a hardware upgrade in order to accept firmware upgrades to version 3.0. It is unclear what this upgrade consists of or how one would get it. Perhaps it is larger ROM chips or flashable ROMs? Every attempt I've made to get an AP9603 has failed.

Placing a SmartSlot Network Management Card in an external module such as the AP9600 makes it quite possible to take the card around to various SmartUPS units and check in on them.

Unlike APC's later management cards, the AP9605 is capable of communicating with any supported device having a SmartSlot. Later management cards require you to change the firmware bundle, which at 2400 bits per second over a serial link, is really rather thoroughly unexciting.

Firmware updates for the AP9605 can be done over serial, telnet (with the aid of TFTP) and seemingly XMODEM as well. Even with TFTP, update speed is still very thoroughly unexciting and you don't get any meaningful status indications as the telnet connection drops almost immediately afterwards.

When installed in or attached to a SmartUPS 750 from February 2008, some of the telnet and SNMP configuration fields displayed protracted strings consisting of extended ASCII garbage. This behavior vanished without the UPS connected. Could it be a partial incompatibility? A conflict between the two cards?

(The former I can't prove, although the displayed garbage disappeared when the card was operating by itself. The latter doesn't seem terribly likely, with the card operating all by its lonesome in a Smart-UPS 1500. One possibility that remains is that of an incompatibility caused by the radical age differences of the UPSes I have as compared to this management card. I also view this as rather unlikely, since the configuration data in question resides in memory on the card itself and not within the UPS. In any case, the card works despite the display of gibberish in these fields. Screenshot to come...when I get around to it.)

Before installing or removing any SmartSlot card, no matter how it happens to be attached, you're supposed to remove power from the UPS completely. Cards that can be hot swapped have longer contact fingers in a few positions as compared to those that don't support hot installation or swapping. To be safe, power down any attached loads safely and turn the UPS off from its front panel controls. Then disconnect the UPS itself from power. At that point you can either wait a few minutes for the UPS to turn itself the rest of the way off, or you may press and hold the OFF (0) button until you hear a click. Lights on the SmartSlot card (if it has any) should go out. At that point you can safely remove the SmartSlot card from your UPS.

If you are planning to reinstall the SmartSlot card, don't be too surprised if the UPS powers itself up upon insertion of the card! I've had a few units that did so, and while I don't think it is in any way harmful if they do, it can certainly be startling. It is rather less amusing when the SmartSlot ribbon cable and its connector break loose from the clips holding them to the SmartSlot "box". Then you have to open up the UPS (being very careful of the hazardous circuitry and making SURE it is very disconnected from all sources of power, including the batteries) and put it back -- or hold it in place while you insert the SmartSlot card.

Should you have one of the newer AP9618 or AP9619 network management cards, you may have noticed that a "humidity" value appears in the Environmental sensing area. APC's bundled temperature probes don't actually report the humidity value because they lack the needed sensor. Yet the AP9618/19 firmware does support collection of humidity data IF you can get a sensor with the humidity module inside it. Thse could be ordered from APC as a separate part back in the day, and they came with the AP9612 Measure-UPS board. Humidity measurement is not possible on the current generation "NMC2" card, model numbers AP9631 or 9632. (Or so I thought inititally: my attention was later brought to this item, which might work to provide a humidity reading on the later APC management cards. Still, it kind of begs the question why APC just didn't include the more capable sensor with the AP9618/19 and AP9631 in the first place.)

If the battery in your AP9617/18/19 or "U" series card runs down, it's a simple matter to replace it with a new CR2032 type. The older AP9605 has no real time clock to worry about (or even a software clock!) and the AP9606 uses "SnapHat" modules that clip over a permanently installed timekeeping IC. Once again, AP963x cards take a significant step back and use a soldered CR2032 (maybe a 2025, it's been a while since I've seen one) battery. As if it weren't bad enough for these NMC2 series cards to have a permanently installed battery, said battery is positioned right over the top of a surface soldered flashable ROM. If the battery ever leaks, there goes your card!

APC employees have stated that the permanently installed battery has to do with product safety regulations. I rather seriously doubt this.

There might be more "odds and ends" to come in time. For now this is quite long and rambly enough, don't you think? (Now that I've added random stuff discussing several generations of product, this has become pretty messy.)

Yet More Diddling Around! (convert an AP9604S into a regular AP9604)

Martin, "aintbigaintclever", from Youtube deduced how to convert an AP9604S enclosure for use with an APC Silcon UPS to a regular AP9604 for use with other APC UPS products. His e-mail is presented below as written:

"I’ve just converted an APC AP9604S (the Silcon variant of the AP9604 Triple Chassis) into an AP9604. Doing this involved working out the schematics for both an AP9604 rev. 6 (borrowed from work) and the AP9604S rev. 7 (from a scrapped Silcon, again from work). This enabled me to work out the schematics for an AP9604 rev. 7 and how to convert one to the other.

Additional components required are:-
2 x 3K3 resistor
4 x 1N4148 diode
1 x 10nF polyester capacitor
1 x KN2907A transistor (or a BC327 fitted back-to-front)
All other components are salvaged from elsewhere on the board where they’re no longer needed.

I stumbled across your website whilst trying to find out about firmware for the AP9604S. If you want the schematics I’ve created, they’re attached in PNG format. Video of the conversion will be uploaded to YouTube later this week."

Martin's schematics are available here, here and here. He retains any and all rights to his work, including copyright. You'll need to ask his permission before using these schematics in your own work, if you plan to make that work publicly accessible. I believe this is his video discussing the conversion process.

As always, your thoughts, comments, suggestions and additions are very welcome! Click here to contact me!

Go Back >

Copyright 2015-2018 by William R. Walsh. Some rights reserved. Review the terms and conditions information linked from the top level page of this server to determine your rights to reproduce and reuse this material.. Last updated with minor changes on 12/17/2018. Preivously updated: 07/01/2016.