Not long ago I was working with a large technology firm in Silicon Valley, discussing the architecture for a new data warehouse. The project was in a nascent stage, so we were having very high-level talks about the appropriate architecture to install. Part of the decision making was to identify how critical the system was, relative to others. The firm labeled the system "mission critical," of the highest priority, and this designation called for a disaster recovery site to be located in Denver. The rationale was that if a disaster occurred at the headquarters in Silicon Valley, the firm could fail over to a system 1,300 miles away to ensure business continuity.
So, here's a follow-up question: If a 9.0 earthquake completely leveled Silicon Valley, how concerned do you think management would be about the business continuity of the data warehouse? My guess? It wouldn't make the top 10. Oddly enough, however, I see this kind of architecture all the time.
There certainly are management issues with this line of thinking, but from a compliance perspective, there's a bigger problem: Almost everybody's worried about the technology, but only a few are worried about the holistic system the technology fits into. If they were worried, they would easily find themselves agreeing with my reasoning above. Instead, they focus on cool technology that could be used to "save the company from disaster." That reminds me of the famous exchange in The Breakfast Club:
Brian: Did you know, without trigonometry, there'd be no engineering?
Bender: Without lamps, there'd be no light.
Whether you're responsible for complying with such regulations as the Sarbanes-Oxley Act or HIPAA, or if you're trying to help the company enforce good internal governance, your job extends beyond the technology: You must make sure the system's business continuity technology is effective.
Where do contingent controls come in?
Business continuity falls under the contingent category of controls (see the Control Matrix chart below). You are dealing with the impact of a risk event before the event occurs.
Key to getting business continuity correct is the way you profile risk. Like software architects, pure technologists are likely to be concerned primarily that a piece of important technology isn't running. The astute compliance strategist, however, will focus on the bigger impact: the fact that the business isn't running.
From the perspective of the compliance strategist, contingent controls may or may not involve the underlying technologies that drive a system. For instance, consider an order processing system, which certainly can be considered mission critical. A myopic technologist profiling risk for the system might consider the likely risk involved in an uncertain event that causes the technology to fail; the impact of the failure is that the technology is unavailable to the business. A contingent control, in that context, becomes an alternate technology that replicates the primary system, located far away from the master.
From the business systems perspective, you could consider a catastrophic event, for example, an earthquake or a nuclear attack, but how likely are those events? Even if the replicated servers properly failed over in less than a second, would that ensure that business continuity with respect to order processing? Maybe, maybe not.
What's more likely is that one key employee -- for instance, the only one who understands the intricacies of the order processing system -- will decide to leave the company because he feels he's underpaid and underappreciated. It's also possible that your old, unsupported software will fail to integrate with the updated firmware, reducing the entire system to an unusable pile of bits. What happens if the system gets corrupted before it fails over? Now you have a corrupted system at the disaster recovery site, which is still of no use to the business.
Thinking critically about real risk will get you started down the path to an effective foundation for business continuity. Remember, the goal is to keep the business running, not to keep the LEDs flashing on your servers. Start by asking such questions as, "What could prevent my company from processing orders?" From there, start to build contingent controls ,which may employ technology in ways you wouldn't have thought of before. Even if you have remote, redundant servers, your offices would be a pretty dark place without lamps.
This was first published in March 2010