A Deeper Explanation of the Methodology Used in Developing the DCIG Buyer’s Guides

In both the write-up found in all DCIG Buyer’s Guides as well as previous blog posts, DCIG has provided explanations and insight into the methodology it uses to arrive at its conclusions and recommendations found in these Buyer’s Guides. However as these Buyer’s Guide are used more widely by end-users, vendors and value-added resellers (VARs) alike to make technology buying decisions, new questions arise. Questions that DCIG receives occasionally concern the credibility of the product data contained in its Buyer’s Guides and the scope of the products that they cover.
One such question concerns the accuracy of the product data. DCIG takes numerous steps to try to make sure that it accurately and fairly represents the data in its Buyer’s Guides. To do so, DCIG provides each vendor of the products covered in a particular Buyer’s Guide the opportunity to:

  • Complete a survey about its particular product(s)
  • Review a DCIG-completed survey about its product(s)
  • Review the data sheet that DCIG completes before it publicly releases them

Each of these stages is associated with a specific time frame for the vendors to respond. During that time, some do. Some do not.  In cases where DCIG does complete a survey and publish information on behalf of a vendor who does not respond to any of DCIG’s requests for information, DCIG discloses that all of its conclusions are based solely upon publicly available information and the analyst’s knowledge of the product.
This approach may therefore result in imprecise information being published about a particular product. However DCIG maintains that this is a credible and defensible position in that each vendor is given at least two opportunities to respond to requests from DCIG about information regarding its particular product(s).
The reasons as to why a vendor may not respond to DCIG inquiries for information are often valid. They may not understand the value and impact that the DCIG Buyer’s Guides have on end-user buying decisions. They may not want information about their product made publicly available. They may not have time to respond to the surveys and/or review the data sheets that DCIG plans to publish.
While DCIG understands that publishing this information about their products absent a response from the vendor may not fully represent all of the capabilities of their product, it is they who are declining to provide the information necessary for DCIG to accurately portray their product in the market.
That said, another issue that sometimes arises is that even when a vendor does respond or request changes, all, some or none of their requested changes may show up in DCIG’s published content. There are some valid reasons for that.

  1. The feature and/or product are not yet available. DCIG only publishes information about features that are currently shipping and generally available as of the publication date of the Buyer’s Guide.
  2. The response came after the cut-off date. DCIG has put specific dates in place in the development process to ensure Buyer’s Guides are developed on a specific time frame. Responses that come in after that time incur additional analyst time and expense to incorporate into the final published Guide so DCIG may opt not to include them.
  3. DCIG counts the number of features supported differently. Rather than listing every feature that a particular product supports (listing all operating systems supported by a backup software is a good example,) DCIG  recently just started stating the number of operating systems it supports in the data sheets it publishes.

How DCIG calculates this total may, however, differ from how the vendor arrives at a total. For instance, in a survey or response back from the vendor, it may reveal that its product supports Windows Server 2003, Windows Server 2008 and Windows Server 2012. While the vendor may say it supports three (3) operating systems, DCIG would  report in its data sheets that it only supports one (1). For those individuals or organizations who want this level of detail about which specific versions of an operating system that a backup software product supports, DCIG would encourage you to subscribe to its Interactive Buyer’s Guide (IBG) that contains this level of detail.
Another question that also comes up from time to time is, “Why does DCIG cover some products even though the vendor did not respond or provide information?” This is done to maintain the credibility of these Buyer’s Guides. For instance, to generate a Buyer’s Guide on enterprise midrange arrays but to ignore the likes of Dell, EMC, HDS, HP, IBM, NetApp or Oracle (just to name a few) just because they fail to respond would bring into question the validity of the Buyer’s Guide.  Even in cases like this, DCIG does disclose on each product’s respective data sheet if the information displayed is solely based on DCIG’s own research so organizations should view the information in that light.
A fourth challenge or statement that DCIG occasionally hears or receives is that the DCIG Buyer’s Guides are not credible. This comment always leaves me scratching my head. The majority of its analysts are either current or former end-users. Their participation in these Buyer’s Guides makes them more credible than most other analyst reports.
Further, the underpinnings for the methodology used in developing these Buyer’s Guide was used by a Fortune 500 company when they made technology buying decisions. Further, in my conversations with storage and system engineers and architects in these companies, many still use some variation of the methodology that DCIG uses in preparing this Buyer’s Guide to create their own internal Buyer’s Guide. DCIG has simply formalized this process and made the final product more credible and authoritative. As a result, many end-users who do not have time to go through this more formal review process now look forward to the publication of the DCIG Buyer’s Guides to help them in their selection of new technologies.
Some new feedback that DCIG recently received is that, “DCIG charges organizations to access and update information in its online Interactive Buyer’s Guide (IBG) or talk to its analysts.” Well, duh. DCIG is an analyst firm and if you want insight into how DCIG does its analysis or reaches its conclusions, that comes at a cost.
In the same way DCIG expects to be billed when it calls its accountant or attorney for the time we spend meeting with them, so should organizations expect to pay to access and speak with DCIG analysts regarding the results in its published DCIG Buyer’s Guides or in its online IBG. This can be done by purchasing a DCIG Research Services contract.
Finally, some allege that the only way DCIG derives its income is from its Buyer’s Guides. That is incorrect. While DCIG certainly licenses its Buyer’s Guides, DCIG also offers other analyst services based upon these Buyer’s Guides including DCIG Research Services and Special Reports, its new Competitive Advantage report and its forthcoming Product Landscape report. These are in addition to its traditional analyst services such as blogging, customer validations and executive and full length white papers that it offers.
In short, DCIG fully stands behinds the papers it publishes, the research it produces and the methodologies it uses to arrive at its conclusions and plans to continue to do so. This is not to say DCIG is not willing to refine its process and make them better. It is. But as to the credibitility of DCIG’s current data collection and research methods and the research it produces, DCIG sees no reason to change any of its practices and has not been to date presented with any compelling reasons to do so.

Click Here to Signup for the DCIG Newsletter!

Categories

DCIG Newsletter Signup

Thank you for your interest in DCIG research and analysis.

Please sign up for the free DCIG Newsletter to have new analysis delivered to your inbox each week.