Cost-effectively and efficiently managing ever-growing unstructured data stores is the next frontier that nearly every organization faces in storage management. But as they do so, they are coming to the realization that the IT staff who manage the storage and business owners who generate the data are not sufficiently engaged with one another to achieve these goals. Achieving a more structured, productive engagement between IT and business owners is what today’s Symantec Data Insight 3.0 release enables.
Estimates as to how fast unstructured data is growing vary widely. While analyst estimates typically fall in the 50 – 75% year-over-year (YoY) growth range, some IT executives are also anecdotally sharing that they are seeing unstructured data growth rates in the YoY range of 400 – 800%.
This rapid data growth rate is resulting in similarly huge increases in organization storage needs. At least one analyst firm forecasts that the amount of storage organizations will acquire will quadruple by 2015 to keep up with these unstructured data growth rates.
But what gets overlooked in these stats is that the individuals who own the data are typically not the same ones who own and manage the backend storage. This creates gaps in communication and understanding between them as they seek to establish and/or understand:
- Who owns the data
- How the data is being used
- The best ways to protect the data
Resolving these gaps in communication and understanding between IT and specific lines of business are the specific issues that Data Insight 3.0 addresses. Rather than IT and business owners possessing a more limited, siloed view into an organization’s unstructured data stores, Data Insight gives them a more holistic view into their environment.
More importantly, Data Insight 3.0 enables them to productively engage with one another by giving them the information both groups need to make better data and storage management decisions. This is becoming particular important as growing, unmanaged unstructured data stores create new compliance risks. Absent the ability to understand who owns the data and should act as its custodian, a company cannot adequately or confidently meet internal or external regulatory requirements as they cannot perform thorough reviews of the data.
Three specific enhancements that Data Insight 3.0 offers to facilitate the engagement between these two groups and alleviate these compliance concerns include:
- Data custodian management. Data Insight has always used heuristics, or experience-based techniques, to understand an organization’s specific data usage patterns, permissions on specific files and folders, where data is located, etc. Data Insight 3.0 builds on these capabilities to create organizational maps for the following two reasons.
First, IT is often stuck watching backend storage requirements grow but are unsure who they should turn to on the business side to help them best manage this growth and optimize data placement. Second, individuals on the business side of the house are unaware of the impact that this unstructured data growth is having on the backend storage.
Now using Data Insight 3.0, organizations can discover who the ideal custodian(s) of the data should be on the business side of the house. In this way, IT can quickly identify the right business person in the organization and engage them so data may be placed on the right tier of storage, archived, or deleted to optimize storage capacity and performance. This feature also helps to ensure a company continually remains in compliance with the rules and regulations to which it is subject.
- Extending platform coverage. Data Insight has always integrated with the Windows file sharing CIFS protocol. Data Insight 3.0 expands upon that by now offering support for UNIX servers that run the Veritas file system (VxFS) as well as expanding integration with SharePoint.
This new support also extends to integrating with the UNIX NFS file protocol that is found in high performance verticals such as is found in many NetApp environments. Along with it, Data Insight adds support for new directory services common to NFS/UNIX environments to include LDAP, NIS and NIS+.
Again, the significance of this feature of Data Insight 3.0 as it relates to IT engaging specific lines of business is that IT is can now engage with more business units in the organization. The end game here is to further help drive storage consolidation by better understanding the data is used so they can make more efficient use of the available storage capacity.
- Big data-like architecture. Mapping data to the right owner is only practical if the architecture of the product scales to the tens if not hundreds of terabytes found in enterprises. In response to this requirement, Data Insight 3.0 has employed techniques such as inverted indexing and custom joins that expedite data collection and indexing.
Then to make this process as non-disruptive as possible on the file server, it takes advantage of its advanced integration with NetApp and its own Veritas Storage Foundation platform to ensure that file sharing performance is not impacted while these collection and indexing activities are going on in the background.
Big Data is a Big Challenge in enterprises and, to effectively respond to it, they cannot rely solely on IT staff. Instead, they need both IT and the business sides of the house to work together to make effective data and storage management a reality. However that is only practical if both sides of the house have access to the information they need to understand each other’s concerns.
Data Insight is that tool to make this goal a reality. While Data Insight has always given organizations the ability to understand data ownership, the new abilities in Data Insight 3.0 enable it to create organizational maps of who possesses the data, access data in new data stores, and scale to handle the large amounts of unstructured data that reside in these data repositories.
By putting this tool in place, enterprises have more reasons than ever to ask and expect the IT and business sides of the house to productively engage with the expectation that excuses for not being in compliance with existing rules and regulations to which the company is subject dissipating.