[Service-advisories] telehouse power outage: all servers online, not yet out of the woods

Jamie McClelland jm at mayfirst.org
Sun Apr 29 23:13:38 EDT 2012


Servers affected: albizu allende amaru assata asturias blanco boggs bola bolivar borges bouazizi brown chavez clyde debord debs dolores donoso dorfman dorothy douglass eduardo fred fuller gil hashmi hay jones juana keller ken kiyoshi leslie lima lora lucius lumumba malcolm mandela marcos marquez marx menchu mirabal morales moses negri peltier proudhon puig ranciere robideau roe severo soledad stallman stone toussaint tresca viewsic vilma yser
Period Affected: 2012-04-29 5:30 pm - 10:45 pm America/New_York time
Date: 2012-04-29

All servers are back online as of about 10:45 pm America/New_York time.

According to the electrician on-site, power was lost to a large portion
of lower Manhattan. Additionally, the uninterruptible power supply (UPS)
system in place at Telehouse, which should have seamlessly switched all
Telehouse customers to the on-site generator, failed.

At about 9:00 pm power from the generator was re-routed, by-passing the
failed UPS. We immediately began powering on the servers, only to have
the power go out again 10 minutes later.

Power was quickly restored. However, we were advised not to start our
servers because additional outages were possible. Having power cut to
servers runs the risk of corrupting data. We chose to keep the servers
off for a little longer to avoid that risk.

At 10:30 pm, we were advised to start our servers because the
electricians were sure that we would have a stable power source for at
least a few more hours.

Please keep in mind: Telehouse is still running on generator power,
by-passing the UPS system. Telehouse has not provided us with an
estimate on when the switchover to the UPS will occur. 

Although Telehouse does not expect the switchover to the UPS to cause
another outage, it is a possibility.

Thanks for your patience. We will keep you updated.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 836 bytes
Desc: Digital signature
URL: <http://lists.mayfirst.org/pipermail/service-advisories/attachments/20120429/edbd647e/attachment.pgp>


More information about the Service-advisories mailing list