"Never let a good crisis go to waste." That’s the mantra a former operations vice president I worked with liked to use. She believed that crises themselves weren’t the problem, but rather our failure to learn from them.
At Synaxis, our hearts and minds are with those from Wilmington to Boston, and New York to Detroit, who are struggling in the aftermath of Hurricane Sandy. Indeed, two of our staff members work out of offices in Washington and New York City. But we are also ever mindful of the lessons that organizations should take from this disaster.
No one is immune: With so many companies abandoning the ancient practice of hosting their websites on servers in their air-conditioned closets in favor of placing their sites with hosting centers, it’s easy to forget that these hosting centers are also susceptible to natural disasters. Datagram Inc., a New York City host, was flooded in the wake of Sandy, taking with it the sites for Gawker and Huffington Post, among others. Datagram is a reliable company, but there’s little it can do about water rushing into its basement.
Redundancy isn’t redundant: When establishing a hosting plan for your site, pay attention to redundancy. Make sure that your plan covers you in case of a failure such as that caused by Sandy. You want to make sure that your provider offers “failover” or “failback” service to ensure High Availability, normally defined as a system being up 99.999 percent of the time. Huffington Post used failover to get its site online and running in short order. A small test we did showed that if a server totally failed, failover should have the site back up on the other server in 2-3 minutes. Faster times are possible, but for considerably more money.
Also, keep in mind the idea of geographical diversity. If your host provides failback, but the backup server is in the same geographical area, you are no better off. Increasingly, datacenters are offering geographical diversity, meaning your main server may be in New York, with your failover server in Arizona. This may be overkill for some, but as Sandy has shown, sometimes it’s critical to have your servers spread across the country.
Don’t ignore the DNS: It’s easy to understand how failures can occur in the wake of natural disasters, but DNS (Domain Name System) failures aren’t supposed to occur at all. Sandy should serve as a reminder that they do fail, as recently happened with GoDaddy just last month. So make sure your host offers secondary DNS, so that if your DNS provider fails your site automatically rolls over to another provider and your downtime is minimized.
Synaxis, like most providers, offers a range of hosting options. The cost difference is attributable to the level of security your organization desires.
There is no doubt that things are better today than they were in years past. Overall, most systems faired pretty well this week nationwide. Of the 40 large sites monitored by Keynote, for example, just one—Monster.com—was affected. And while performance has slowed an average of 7 percent, all in all the large sites are weathering Sandy well.
Obviously, most have learned their lesson about protecting their sites. It’s a good time for you to do your own check up and make sure that you have the proper level of redundancy built into your system.