The Company Flag: Stupidity Rampant On A Field Of Cluelessness! – How Not To Manage A Critical Control Emergency

The Curmudgeon enjoyed a robust and very intense working career which has now concluded.  He was privileged to work for and with technical specialists with international reputations.  And he was un-privileged to work for and with management at the far other end of the talent spectrum.  Here is his story, told in the first person, about some experiences with the latter folk.

The time was shortly after the beginning of the twenty-first century.  Nearing the end of my professional career and for a second time during that career, I was working as a degreed-engineer in the telecomm section of the power utility industry.  One of my duties that year was the repair and modernization of the company-owned radio sites, a task which fit well with my previous employment experience.

This narrative concerns the Dawson Point radio site, a major company facility.  Dawson was one of the original company radio sites, and it was and still is a very busy radio relay location.  Among other things, it houses the microwave radio repeater stations at the nearer end of a relay chain which stretches out of state to distant electrical generating stations.  From Dawson the arriving microwave traffic from the far end is sent forward to the company’s metropolitan operations locations for further distribution and use.

One of the major items of traffic on that stretch of the company microwave network is the primary control system for the big electrical transmission lines that run to our city from the out-of-state generating stations. This is a “real time” control and monitoring operation for these major lines, with real time data about the transmission network coming into our control center on the ground, and real time switching commands to the transmission network going out from there.

Dawson Point was the last site that I would upgrade in my year-long project.  And one of the upgrades that the site really needed was a new emergency electrical power generator to provide site operating power when/if the regular electrical service to the Point was cut off.  Such cut-offs were a fairly regular occurrence, as there was only a single overhead power distribution circuit running to the Point and it ran through a wilderness area.

The existing Dawson generator had become undersized over the course of several decades as more radio equipment was added to the site, and the generator was also old and tired.  Now when called upon, it could not supply sufficient power to run everything at the site, including the critical air conditioning needed to cool the microwave radios.  From time to time commercial power to the site would drop out, the generator would start and then become overloaded, and shortly thereafter alarms for the microwave radios would begin coming in. [The radios themselves had a battery bank back-up power source from which they operated; this fact will become important later.]

I proposed a new 50 kilowatt (electrical) generator as a replacement, and the proposal was accepted and funded.  This would solve that problem.  Or so I thought.

In the fall of that year, we installed, tested, and commissioned the new generator, which was fueled by propane from a big, high-pressure storage tank on site.  All was well.

I then telephoned our company “Regulatory Compliance” group and routinely asked them to file with the regional environmental authority for an upgraded operating permit for the new, larger generator.  A few hours later I got a telephone call from that group’s manager.  He curtly informed me that their group would file for the certificate, but until they received it the company could not operate the generator.  At all!  I asked how long that permit issuance process would take.  At least six weeks, perhaps longer, was the answer?  I asked whether, during that time, we could keep the generator wired in, and use it solely if an emergency occurred.  Not for routine testing and maintenance, just for emergency use.

Absolutely not was the reply.  The manager shrieked that the environmental authority was just itching to find somebody with “deep pockets” who was violating their agency’s arcane rules and then they would slap heavy financial penalties on the culprit.  And, the manager assured me, that was not going to be our company.  I explained to him that our new generator was on the authority’s “approved equipment list,” that a generator had already been permitted and had operated at the site, that we would certainly be granted a permit, and thus we weren’t operating unauthorized equipment.  And I carefully explained the critical importance that the generator played in our microwave network.

The manager’s reply was terse: no dice!  He wasn’t going to put either his or the company’s butt on the line.  But he would contact one of his contractors and have an authority-approved portable Diesel generator towed to the site and wired in as a temporary substitute.  That way we would at least be covered in the short term.  This he did, and we all sat back to sweat out the permit processing time.

Just six days later, on a Saturday afternoon, one of the biggest natural disasters that would befall our region during that entire decade struck!

It was the next day, Sunday, when the spreading disaster destroyed the commercial power lines running through the wilderness to Dawson Point.  They were gone and I knew that they would not be back for many weeks.  So it was now up to the local generator to carry the load.

Mercifully, the temporary Diesel generator did start and did pick up the load.  I could tell this because at home I was equipped to receive the transmissions from another company radio at Dawson, not powered by the backup batteries. That radio was still talking.  Meanwhile the effects of the disaster spread rapidly, and soon enough there was a critical power emergency over the entire region.

When I awoke early on Monday morning I immediately checked my Dawson radio, and it was no longer transmitting.  I also realized that the company was already in trouble with respect to its normal power distribution responsibilities to the entire region, and so I reported to my office earlier than normal.

There I found the local telecomm managers (who were quite skilled in our technology) already on duty and wearing abundant frowns.  I asked what the problem was, and they explained that during Sunday night/early Monday morning the portable Diesel generator at Dawson had run out of its on-board supply of Diesel fuel and had quit.  As we spoke the microwave radios were running entirely on the batteries, and the batteries were being discharged rapidly.  Within a few hours they would be dead, and the microwave radios would quit.  The company would then lose primary control over the major power transmission lines.  This because some stupid turkey of a non-technical manager was willing to risk loss of control over most of our region’s imported electrical power for the sake of “avoiding” the risk of a potential “fine” that might run to a few thousand dollars if ever imposed.

I said that the solution was obvious: get one of our technicians out to the Point, wire the brand new, unused, propane-powered generator back into the circuit, and get emergency power restored.  (As a management-level employee, I was prohibited by union rules from working on company equipment and thus could not do it on my own initiative—or I would have!)  There was at least a three day supply of propane in the Dawson tank, and we could keep the microwave system (and the other radios at the site) running during this critical time.  Refueling the Diesel generator would get us only another eight to ten hours of operating time.

The telecomm managers said that we couldn’t authorize the restoration ourselves without approval from higher levels within the company, because of the risk of offending the Regulatory Affairs manager by violating his prohibition.  I said that I highly doubted that the environmental authority staff would be very interested in us and our lone generator, located at the end of a long wilderness dirt road, on a day of major disaster in the county.  But I said that I would start a request to operate the generator without a current permit going “up the line” within the company.

I called the Regulatory Affairs manager back and explained the dire need to him.  I secretly hoped that he realized that his clueless “brilliant regulatory compliance scheme” had totally back-fired, but I didn’t utter that thought out loud.  And he took the only positive action that I ever observed from him: he did forward our request up the line.

In fact, the request went to the company’s Emergency Operations Center.  There, fortunately, the entire herd of company Vice Presidents had assembled, and they were intensely involved in “managing” the battle.  Specifically, as I had observed of them on earlier occasions there, they were gathered in a special glassed-in observation room behind and above the actual EOC operating staff, and were very busy performing their traditional VP duties: eating donuts, sipping coffee, and chatting with each other.

They were interrupted with the request, and they solemnly considered it.  Finally they voted: they would accept the “huge financial risk” and would allow the use of the “unauthorized” generator in an emergency to save their microwave network.  How very gutsy of them!

Back in my office, we got the word and dispatched a technician to put the new generator back on line.  Fortunately he got through, reached the Point in time, got the job accomplished, and the microwave network soldiered on.  We also arranged for scheduled propane deliveries to the site (the Dawson Point area was again accessible by road) by a contractor on a three day delivery cycle.

At that point it was then up to the new generator.  And the generator persevered!  It ran continuously at the necessary output power level for thirty-three straight days, save for a one-hour time-out in mid period for an oil and filter change.  During the quick time-out the microwave radios were powered by the (then) fully-charged battery plant.  The generator never faltered, and it finally went silent when the new power lines out to Dawson Point were constructed and activated, a circuit restoration project that had been given a very low priority within the company.

My generator project had saved a major portion of the company during one of its biggest disasters.  The previous generator would never have handled the emergency.  When I now consider the enormous problems that would have been produced had the company lost control over the region’s imported power during the crisis, I still quake slightly!  We were lucky, but we came very close.  The hardware worked perfectly.  The company management didn’t.

And did I receive credit for my work?  No, of course not!  I did receive some continuing criticism for “acting without all the proper consultations” in installing the new generator.  But never any credit.

And then later, after matters had settled down, I learned two important facts about the “regulatory crisis.”

1.  On the first Monday of the disaster the regional environmental authority had announced that it was suspending all of its relevant Rules, and it proclaimed that all organizations could act however they needed to weather the crisis.  This declaration is generally SOP for most regulatory agencies during emergencies.

2.  If our Regulatory Compliance people had merely walked the application for an upgraded generator permit, together with a check in the required amount, over to the front counter of the environmental authority’s local office, the authority’s staff would have delivered the needed authorization immediately back across the counter into our people’s hands!  We would have had the permit with time to spare for the soon-forthcoming emergency, and no half-assed temporary measures would have been needed.  But our staff chose to mail in the application which, after delivery to the agency, was then pigeon-holed by them for weeks without action.

This was by far not the only bone-headed, counter-productive, near-tragic blunder by incompetent kiester-preserving management that I experienced during my tours in the utility industry.  But it was one of the major ones.  And since the industry operates under a “cost plus” return-on-investment basis, with expenses approved by tolerant state Public Service Commissions, there is little cost to the industry for even major blunders.

And there it is.  I did my job as I saw my duty, I was successful, and I have zero apologies for having done it. So SCREW YOU, clueless and witless utility company senior management!  You are a waste of the rate-payers’ dollars and a disgrace to the hard-working staffs in that industry who “keep the lights on” every day.  And, now that I am retired, I’m delighted to be rid of you for the rest of my life!

What do you think?

“Let’s save the universe for RF!”
The old RF Curmudgeon