Defensible Data is the Goal

Individuals occasionally reach out to DCIG and allege that certain data found in DCIG publications is, from their perspective, “incorrect.” While I appreciate the time and effort that individuals take to review data found in the various DCIG publications and provide feedback on it, viewing any data present in any analyst publication – be it from DCIG or otherwise – as either “right” or “wrong” is the larger premise that one should consider. While DCIG always does its best to follow established, internal processes to ensure that the data it publishes reflects the actual capabilities of the products it covers, DCIG’s broader objective is to publish defensible data.

DCIG is one of the few analyst firms that takes on the task of publishing competitive data. Whether DCIG evaluates multiple products from multiple vendors – such as it does in its Buyer’s Guides – or when it compares two products – such as it does in its Pocket Analyst Reports – these reports inevitably generate some differing opinions and even controversy.

Some of the disagreement stems from DCIG’s practice to rank products or call out when one product has an advantage over another. In the Buyer’s Guides, DCIG ranks products and opines as to whether a product ranks as Recommended, Excellent, or Good. In the Pocket Analyst Reports, DCIG compares two products and deems one vendor or product to have an advantage over the other in terms of a certain feature functionality. In both these publications, the rankings it establishes or the advantages that it declares should be viewed as subjective that reflects DCIG’s opinion – which we believe most people understand and perceive.

However, readers of the DCIG Buyer’s Guides or Pocket Analyst Reports sometimes take issue with the data that DCIG publishes about how individual products support specific features or capabilities.  When they see a check box next to a specific feature indicating support for it or a grey circle next to it indicating no support for it or that DCIG could not determine product support for that feature, they may know from their own experience that the feature should be checked as supported or displayed as unsupported. There is then a proclivity to discount the value of the publication because DCIG evaluated a feature in a way does not align with their experience or knowledge.

If you have had that experience, one should keep two principles in mind when evaluating the data DCIG publishes regarding support for product features:

  1. All data published represents DCIG’s opinion. DCIG does its best to ensure the accuracy of all data it publishes. It reviews product data sheets, administrator guides, user guides, and reaches out to vendors to solicit their input. However, there are any number of reasons the data we publish may not accurately reflect the product’s actual capabilities. The product admin or user guides may be incorrect or out of date. Incorrect feedback may have been provided. The data may have not been transcribed correctly at some point during the layout process. The product may have added (or removed) support for certain features. It is for these reasons and others that DCIG treats all data it publishes as its opinion and not as fact and readers of DCIG’s publications should do likewise.
  2. Vendors do not disclose all information about their products. This came as a surprise even to DCIG. It was our expectation that if a vendor supported a feature that they would want to share that information. Not true, as we have learned. Just because a product supports a feature does not mean that vendors necessarily want that information known publicly. This is due, in part, to the fact that enterprise environments are very complicated and the feature, while it is offered and supported by a product, may only work in certain environments under specific conditions. In those circumstances, vendors prefer not to publicly disclose that they support a feature since then their current and potential customers may hold them accountable for delivering on that feature in their environment.

It is for these reasons and others that DCIG’s goal in its publications is to publish defensible data. People may and likely will disagree with some of DCIG’s conclusions and observations, even those that such as feature support that organizations may view as more objective than subjective. However, DCIG has learned over the many years that it has published its Buyer’s Guides and Pocket Analyst Reports that all data on technology topics is more subjective than objective in nature that many may realize or even prefer and should be treated as such.

Jerome M. Wendt

About Jerome M. Wendt

President & Founder of DCIG, LLC Jerome Wendt is the President and Founder of DCIG, LLC., an independent storage analyst and consulting firm. Mr. Wendt founded the company in November 2007.

Leave a Reply