Power outage in main DC

Post Mortem

The outage was caused by a failure in the redundancy control system causing a security shutdown of all power to all racks in the Datacenter cutting out power from both the main feeds and generators.

In an event like this we needed to to site and measure the incoming feeds to see nothing was wrong physically and physically have to push in the trigger buttons to restore the power. The connectors failing are rated at 3x1250AMPS so if this was actually getting faulty power from the gensets then it would create a massive electrical fire.

We are treating this as a catastrophic event as a datacenter of that size and with everything built redundant should never experience an event like that. The electricians are investigating this in every detail and we are doing a full service of all components ahead of schedule to find the root cause.

We have poweroutage in main stockholm DC. We are investigating.