Services for Organizations

Using our research, best practices and expertise, we help you understand how to optimize your business processes using applications, information and technology. We provide advisory, education, and assessment services to rapidly identify and prioritize areas for improvement and perform vendor selection

Consulting & Strategy Sessions

Ventana On Demand

    Services for Investment Firms

    We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

    Consulting & Strategy Sessions

    Ventana On Demand

      Services for Technology Vendors

      We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

      Analyst Relations

      Demand Generation

      Product Marketing

      Market Coverage

      Request a Briefing

        David Menninger's Analyst Perspectives

        << Back to Blog Index

        Evaluating Analytics and BI Software Vendors’ Capabilities

        Ventana Research has been evaluating analytics and business intelligence (BI) software for a long time—almost 20 years. Our methodology for these assessments is referred to as a Value Index. We use weightings derived from our benchmark research about how you, as buyers of these technologies, value and evaluate vendors. You can view our 2019 Value Index results here. I am in the process of completing the 2020 evaluation now.

        As we consider analytics vendors, we look at over 1,500 criteria for each vendor. One-third of those criteria are related to capabilities of the vendors’ products. My goal here is to give you a perspective on why we evaluate such a broad range of capabilities, and to help your organization understand how it should consider a more robust evaluation of what you need from analytics and BI.

        Let’s work backwards through the analytics process to understand the categories we evaluate. Analytics should lead to action in an organization. To drive action, you must not only make decisions, but you must communicate those decisions and the actions they require to the appropriate portions of your organization. Where possible, you should also be able to automate those tasks, activities and processes, including some of the actions resulting from the decisions.

        There are many aspects to communication in analytics processes. People consume information on different devices and collaborate across multiple channels. Each type of device should support rich, personalized interactions that are appropriate to the individual and the device. For example, you don’t necessarily need to prepare a completely new analysis from scratch on a mobile phone, but you should be able to ask questions and get answers using natural language processing. We assert that by 2022, Ventana_Research_2020_Assertion_Conversational_Computingone-half of organizations will be using conversational computing in voice and chat methods to simplify interactions with business applications. Organizations also need to communicate their plans, strategy and even vision to make sure everyone is aligned around the same goals and objectives. Without this context, individual analyses may be made in a vacuum and may not support the organization’s mission.

        Making decisions requires several steps that go beyond merely looking at historical data in a visualization. History can be informative, but analyses should also include predictive and comparative analyses of alternative scenarios for dealing with what has been observed. One solution for combatting declining sales may be found in a recommendation to increase future advertising and marketing campaigns. An alternative recommendation may be to hire additional inside sales reps to make more outbound calls. A complete evaluation of the alternatives using analytics would compare the projected costs and benefits of each. If increased marketing campaigns are selected, for example, predicting which offers to make to which customer segments will help maximize the effectiveness of the spend.

        Not everyone in your organization will understand how to perform these different analytical steps. Guidance is both helpful and necessary. Simple guidance might include a series of related analyses sometimes referred to as a “story” or “presentation.” More sophisticated guidance includes natural language explanations of relationships in the data, and suggested next steps to consider. Providing guidance also ensures consistency throughout the organization.

        Analyses should be supported by multiple presentations and ways to interact with the information. In some cases, a simple list or report is enough. In other cases, a dashboard including a variety of metrics may be more appropriate. And in others, interactive visualization and exploration of the data is necessary. With the volume of presentations and information available in organizations, employees need to be able to easily and efficiently search through all the material to find the right information and analyses.

        Underlying all of this is the data behind the analyses. There are several key Ventana_Research_Benchmark_Research_Data_Prep17_13_Time_Spent_200205-2requirements around data processing. First, which sources can be accessed, and how easy is it to access those sources? Second, how does the product support data preparation? These two tasks, connecting to data sources and preparing data for analysis, are the most time-consuming tasks in the analytical process for many organizations as was found in our Data Preparation Benchmark Research.

        We also evaluate the richness of the data modeling capabilities. Line-of-business personnel should be able to view the data organized in a way that is familiar to them, including hierarchies and other associations in the data. It should also be easy to create derived metrics as part of the data model since these are generally the basis for key performance indicators in an organization. Throughout all these data processes we also examine the data quality and governance capabilities.

        Our criteria may not align entirely with every organization. And no vendor has ever fully met all the criteria we evaluate. In your organization’s evaluations, determine what importance to assign these different categories of capabilities. If you choose to use multiple products to meet your needs, make sure to consider how well the products integrate with one another. Fortunately, vendors are starting to include more embedding and integration capabilities to products, opening up a wider range of options for an organization.

        Hopefully, this analyst perspective will help your organization evaluate analytics and BI software vendors capabilities. Organizations should consider the full breadth of criteria highlighted above to insure they can maximize the value of their analytic processes.


        David Menninger


        David Menninger
        Executive Director, Technology Research

        David Menninger leads technology software research and advisory for Ventana Research, now part of ISG. Building on over three decades of enterprise software leadership experience, he guides the team responsible for a wide range of technology-focused data and analytics topics, including AI for IT and AI-infused software.


        Analyst Perspective Policy

        • Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

          Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to

        View Policy

        Subscribe to Email Updates

        Posts by Month

        see all

        Posts by Topic

        see all

        Analyst Perspectives Archive

        See All