It was smooth until it wasn’t
In reporting on the Superbowl blackout a CBS News correspondent commented, “It was smooth until it wasn’t.” Which echoes Craig Fugate’s comment, “Our system works really well until it doesn’t.”
Rational, reductionist, predictive, risk-informed, well-tested — and almost always effective — many of our most important modern systems hum along until suddenly they don’t.
From a Tuesday morning front page story in the Times-Picayune:
It’s still unclear exactly what went wrong Sunday. Entergy officials said they are working with the company that built the electrical switchgear, which controls the flow of electricity from the power company to the stadium, to determine if that is to blame.
The equipment, added as part of the upgrades, automatically shuts down when a problem is detected, such as a surge or loss of electricity, potentially signaling — and protecting — against a more protracted power outage.
Ultimately, the equipment worked as it was supposed to. But what caused it to trip Sunday is the central mystery officials are now trying to unravel.
Doug Thornton, senior vice president of SMG, which manages the Superdome, said Monday that the switchgear “sensed an abnormality” and tripped.
“It was a piece of equipment that did its job,” he said. “We don’t know anything beyond that. It’s premature at this point to say what it was or what caused it.”
A cause will be found and a recurrence of that cause will be suppressed. And probably, unknown and unintended, something even worse will be seeded in the fixing.
If you have not, I encourage you to read Bak’s Sand Pile: Strategies for a Catastrophic World by Ted G. Lewis.
