This content is part of the Essential Guide: A guide to modern records management challenges
Manage Learn to apply best practices and optimize your operations.

Big data era requires new approach to information governance strategy

The big data age is increasing risk and storage costs for businesses, forcing many to reconsider their approach to information governance strategy.

Many organizations have struggled with managing unstructured data -- the often text-heavy, unorganized information that, left unattended, can cause huge risks and unnecessary storage costs.

As the big data era continues, companies will have to reexamine and adapt their information governance strategy. By 2018, 25% of progressive organizations will manage all their unstructured data using information governance and storage management policies, up from less than 1% today, predicts Stamford, Conn.-based consultancy Gartner Inc.

"Once it's created, it's around forever," Gartner Research Director Alan Dayley said of company data at the Gartner Security and Risk Management Summit in National Harbor, Md., in June. "We need to do something with it, and we need to start governing it."

The trouble is, many modern organizations struggle with data governance. The amount of data floating around the average organization -- much of it trivial -- makes determining who owns specific data, how long to keep that data and who is responsible for managing it a difficult proposition. First and foremost, organizations must understand exactly what data they have, and the value of it, Dayley said.

You just can't keep keeping everything forever. It's costly on storage; it's costly just trying to filter through all of it and understand it.
Alan DayleyGartner Inc.

Another cause is data confusion, especially from a regulatory compliance standpoint.

"We're not clear on regulatory and compliance issues," Dayley said. "We don't understand what we're supposed to keep, so we keep everything."

Information governance strategy implementation and deployment requires input from across the organization, Dayley said during his Gartner Summit presentation:

  • Compliance officers should be consulted to interpret regulatory compliance requirements and how long information must be retained according to these regulations. They can also help determine audit schedules.
  • The legal team has responsibility for assessing information risk and determining a defensible deletion policy.
  • Business users must understand the current and historical value of data and need to be included during the information management policy development.

The primary goal of an information governance strategy is to make sure data supports business priorities effectively and efficiently. Any data that presents very little value, such as transient user communications, early working copies of files and old data from legacy applications, should be deleted, Dayley added. "Over time, the value goes way down, whereas the cost to continue to manage it goes way up," he said. "You just can't keep keeping everything forever. It's costly on storage; it's costly just trying to filter through all of it and understand it."

Incorporate big data analytics

If used strategically, analyzing this big data produces huge benefits, said Vice President and Gartner Fellow Neil MacDonald during the summit. MacDonald said that when it comes to information security, organizations often establish baselines of "normal" data behavior and look for meaningful deviations.

Giving information more context through big data analytics allows organizations to establish a better understanding of this "normal" behavior and determine meaningful deviations from the baselines.

"You have to have a very good idea what normal looks like, then look for meaningful variations from that to infer malicious intent," MacDonald said. "How do you find a needle in a haystack if you don't know what the needle looks like?"

Difficulties arise when organizations are required to govern content that they did not create and do not own but may be responsible for -- or find value in. For example, employee-generated social media data creates potential privacy risks but can also be very useful to the business from a marketing standpoint.

The cloud is another concern, because it's part of a trend wherein IT has less and less direct control of the organizational infrastructure, MacDonald said. As more elements of IT infrastructure go mobile, the tech department must offset these security concerns with detailed auditing, logging and monitoring of big data activities.

"You want visibility to compensate for the lack of direct controls," he said.

More on information governance strategy

  • Q&A: Developing a data management policy
  • E-discovery processes provide value to information governance

If properly managed, breaking down and analyzing big data provides huge business benefits, he added.

"You've got the data; why not leverage it?" MacDonald said. "Focus on your objective, which is risk prioritized, actionable insight telling you what to do and what to focus on so that you can have the most impact on protecting the assets of your company."

This won't be easy. The volume of information in the big data era, combined with new file system technologies, repository formats and nascent programming interfaces, mean that more sophisticated and mature archiving, e-discovery and compliance technologies are not yet available, Dayley said.

As a result, organizations will be forced to manage and govern some of this content using manual policies and practices versus automated software while they wait for vendors to catch up, he added. Dayley predicts progressive companies will incorporate policies and products to assist them with automatically governing their unstructured data.

"What these tools do is give you a good visualization of the data and help companies understand what it is," Dayley said.

Let us know what you think about the story; email Ben Cole, associate editor. For more regulatory compliance news and updates throughout the week, follow us on Twitter @ITCompliance.

Dig Deeper on Content management software and compliance