IT risk mitigation and strategy has become commonplace as threats from hackers, data breaches and even malicious employees make protecting information vital to everyday business operations. What's less common are specific risk-management metrics to determine the success of these efforts: Was the organization just lucky that nobody sought their sensitive data, or was their risk strategy effective in preventing a breach?
In part three of this three-part video, shot at the Marcus Evans Enterprise Risk Management Conference in Chicago earlier this year, SearchCompliance editor Ben Cole sits down with five risk management professionals to discuss how companies can identify potential risk management metrics specific to their organization.
How can companies measure the effectiveness of their organizational risk management processes? Is it just a matter of not having a breach or anything bad occur, or are there actual metrics that can be put in place?
Adi Agrawal (executive director, enterprise risk management, Chicago Mercantile Exchange): I think the question of metrics is interesting because, first, let me clarify: Not having a breach occur is neither feasible nor an appropriate metric. Breaches are going to occur. People make decisions. Some decisions play out appropriately and some don't. That's just been a fact of life forever, and it's going to continue into the future.
In terms of metrics, I look at metrics in two ways. One is qualitative things, things like measuring your capability. Do you as a firm feel that you make good decisions? If you make good decisions, some of the outcomes will not be good, but you still feel you have a good process. The process is well-optimized, communications are good. I think that's one kind of set of metrics, and we've defined metrics like that to measure our own risk programs. We look at how well our process does, how well we communicate with people. Then, in turn, do people get surprised when something happens? That's really the qualitative measure. How often do people in the company who make decisions feel surprised by something?
These measures have to be customized to your program and to your company. We can't just take something off the shelf and slap it in there and hope that it will work.
Adi Agrawal executive director of enterprise risk management, Chicago Mercantile Exchange
The more assertive quantifiable measures are for things like products. What's risk management's product? Our product is the inventory of risks and the inventory of treatment actions and clarification reporting and communication. How do we process that product? Can we have quality measures in it? Can we have timeliness measures in it? Can we have measures for completeness? It builds up the qualitative measure, in a sense. Did we get surprised, and if we got surprised did we then roll that into our universe and treat it?
I think you can have a combination of those. But one thing that's important is that these measures have to be customized to your program and to your company. We can't just take something off the shelf and slap it in there and hope that it will work.
Frank Fiorille (senior director, risk management, Paychex Inc.): I think we've been talking about that a lot at the conference, and how do we kind of quantify that and measure it. One of the things that I think is really interesting is when we talk about crisis management, how quick a company can bounce back. For instance, there's a lot of potential data breaches that companies have gone through, and if you study the stock price of those companies, they take hits immediately but they bounce back real quick. Same thing from a compliance standpoint: If a company has an issue, there's examples out there where the company has had that issue but their stock prices bounced right back very quickly.
That's one kind measurement, and just from an overall performance standpoint, I think [risk-management metrics] will be in your basic business results.
Sean Browning (director, enterprise risk management, Vectren Corp.): I think what you want to do is develop some type of maturity continuum across multiple layers of your cybersecurity or your IT program. You can show progress in terms of what gaps you've identified and how you're closing those gaps. You've got to identify what's the right level for your organization. As you continue to close those gaps, I think you can demonstrate some progress. It certainly doesn't hurt to not have any gaps. Oftentimes in risk management the only real metric is failure. You only really know when you've done something wrong when something bad happens. But that's a juggling act. It's very hard to follow.
I think you do need to show progress and engage management to get the input on that progress, and keep the stakeholders involved.
Victor J. Haddock (senior vice president, internal audit, Magellan Health Services): It's a tough question for all of us in some sort of risk management roles. I think there's an element that when something doesn't happen, was that because of the effectiveness of the risk management activities you had in place or just you were lucky nothing happened? There's a certain element that someone's success that will be measured based on things that are not happening, the company not facing a breach or facing another risk. Certainly, that is something that can be assumed.
I think the real investment comes from are we able to move the needle in some of those areas? Are we able to take advantages of the risk and turn them into opportunities? Those are really more measurable when companies have the tools and set some goals and say, "OK, here's where we're going to be from a security standpoint next year, and did you attain that level?" Did you get the right certification, are you being evaluated by external parties and passed an audit? Those are things that would allow you to see if you getting a return investment on the risk management efforts you put in place.
More on risk-management strategy
Corporate cloud and social media use having major impact on ERM strategy
Cyberthreats, technology influencing modern risk management
I think there are risks that are more subjective, so that's harder to do. But there are certain risks that you can measure. In terms of privacy of data or breaches of data, you can tell how many you had in a year. If you had two releases of data that were unauthorized, or the next year you had none -- there are clearly some measurements associated with the return investment.
Tate Mitchell (director, internal audit, Aegion Corp.): It's kind of somewhat frustrating that it just depends on the type of company you're talking about. My guess is most small and midsized companies have a limitation on resources, so it's more of a reactive-type approach when it comes to risk management or risk mitigation. Then you get to larger companies -- I used to work at Siemens, a much larger company, where the extent of their resources is much more vast. Or when you talk about your typical billion-dollar-a-year revenue type of company, which is a company that I work for right now.
It's a challenge. It's a little bit more of a response-type environment versus prevent and detect. It has a huge impact on how you have to risk manage and risk mitigate. You try to promote the importance of trying to develop a true risk management type of environment to be more preventive and detective. But with a limitation of resources, it sometimes doesn't allow you to do that. So, I think the bigger companies are the ones that are doing it the best just because there's the ability for them to do that.