Contingent controls complement business continuity, DR

An important aspect of managing risk is to build in contingent controls alongside your disaster recovery and business continuity implementation.

Will your business keeping running if the lights go out? Implementing contingent controls is an important but sometimes overlooked aspect of ensuring business continuity and disaster recovery.

Not long ago I was working with a large technology firm in Silicon Valley, discussing the architecture for a new data warehouse. The project was in a nascent stage, so we were having very high-level talks about the appropriate architecture to install. Part of the decision making was to identify how critical the system was, relative to others. The firm labeled the system "mission critical," of the highest priority, and this designation called for a disaster recovery site to be located in Denver. The rationale was that if a disaster occurred at the headquarters in Silicon Valley, the firm could fail over to a system 1,300 miles away to ensure business continuity.

More on Business Continuity
Private Sector Preparedness Program provides business continuity options 

Are mandatory business continuity management standards good business?

Comparing how-to guides for business continuity standards

Effective techniques for continuity risk management, measurement
Now, I ask you, what kind of disaster would require a company to locate a shadow system three states and 1,300 miles away from the primary site? We're not talking about a hardware failure, because a redundant system two feet away could handle that. We're not talking about an extended power outage, because you probably could go out 50 miles or so to handle that problem. It seemed that this firm was bracing for some sort of catastrophic disaster in Northern California, perhaps a magnitude 9.0 earthquake or a nuclear attack.

So, here's a follow-up question: If a 9.0 earthquake completely leveled Silicon Valley, how concerned do you think management would be about the business continuity of the data warehouse? My guess? It wouldn't make the top 10. Oddly enough, however, I see this kind of architecture all the time.

There certainly are management issues with this line of thinking, but from a compliance perspective, there's a bigger problem: Almost everybody's worried about the technology, but only a few are worried about the holistic system the technology fits into. If they were worried, they would easily find themselves agreeing with my reasoning above. Instead, they focus on cool technology that could be used to "save the company from disaster." That reminds me of the famous exchange in The Breakfast Club:

Brian: Did you know, without trigonometry, there'd be no engineering?

Bender: Without lamps, there'd be no light.

Whether you're responsible for complying with such regulations as the Sarbanes-Oxley Act or HIPAA, or if you're trying to help the company enforce good internal governance, your job extends beyond the technology: You must make sure the system's business continuity technology is effective.

Where do contingent controls come in?

Business continuity falls under the contingent category of controls (see the Control Matrix chart below). You are dealing with the impact of a risk event before the event occurs.

Key to getting business continuity correct is the way you profile risk. Like software architects, pure technologists are likely to be concerned primarily that a piece of important technology isn't running. The astute compliance strategist, however, will focus on the bigger impact: the fact that the business isn't running.

Control Matrix
 Risk attribute
Root cause
Preventive control
Contingent control

From the perspective of the compliance strategist, contingent controls may or may not involve the underlying technologies that drive a system. For instance, consider an order processing system, which certainly can be considered mission critical. A myopic technologist profiling risk for the system might consider the likely risk involved in an uncertain event that causes the technology to fail; the impact of the failure is that the technology is unavailable to the business. A contingent control, in that context, becomes an alternate technology that replicates the primary system, located far away from the master.

From the business systems perspective, you could consider a catastrophic event, for example, an earthquake or a nuclear attack, but how likely are those events? Even if the replicated servers properly failed over in less than a second, would that ensure that business continuity with respect to order processing? Maybe, maybe not.

What's more likely is that one key employee -- for instance, the only one who understands the intricacies of the order processing system -- will decide to leave the company because he feels he's underpaid and underappreciated. It's also possible that your old, unsupported software will fail to integrate with the updated firmware, reducing the entire system to an unusable pile of bits. What happens if the system gets corrupted before it fails over? Now you have a corrupted system at the disaster recovery site, which is still of no use to the business.

The key to getting business continuity correct lies in how you profile risk.
To set up contingent controls, consider the ways that the overall system -- not just individual technologies --can fail. What will you do in those circumstances to keep the business running? Technology can still play a big part in this, but it doesn't have to be all about redundant systems. How about an education system that disperses the knowledge of the system, displacing the exposure that results from one disgruntled employee carrying the entire intellectual property of the system in his head? Or how about a redundant set of processes to place an order, in case one system gets corrupted?

Thinking critically about real risk will get you started down the path to an effective foundation for business continuity. Remember, the goal is to keep the business running, not to keep the LEDs flashing on your servers. Start by asking such questions as, "What could prevent my company from processing orders?" From there, start to build contingent controls ,which may employ technology in ways you wouldn't have thought of before. Even if you have remote, redundant servers, your offices would be a pretty dark place without lamps.

John Weathington is President and CEO of Excellent Management Systems, Inc., a San Francisco based management.

Let us know what you think about the story; email [email protected]. , Follow @ITCompliance for compliance news throughout the week.

.target="_blank">@ITCompliance for compliance news throughout the week.

Dig Deeper on Disaster recovery and compliance