David Menninger's Analyst Perspectives

I have written previously that the world of data and analytics will become more and more centered around real-time, streaming data. Data is created constantly and increasingly is being collected simultaneously. Technology advances now enable organizations to process and analyze information as it is being collected to respond in real time to opportunities and threats. Not all use cases require real-time analysis and response, but many do, including multiple use cases that can improve customer experiences. For example, best-in-class e-commerce interactions should provide real-time updates on inventory status to avoid stock-out or back-order situations. Customer service interactions should provide real-time recommendations that minimize the time to resolution. Location-based offers should be targeted at the customer’s current location, not their location several minutes ago. Another domain where real-time analyses are critical is internet of things (IoT) applications. Additionally, use cases like predictive maintenance require timely information to prevent equipment failures that help avoid additional costs and damage.

For years, maybe decades, we have heard about the struggles between IT and line-of-business functions. In this perspective, we will look at some of the data from our Analytics and Data Benchmark Research about the roles of IT and line-of-business teams in analytics and data processes. We will also look at some of the disconnects between these two groups. And, by looking at how organizations are operating today and the results they are achieving, we can discern some of the best practices for improving the outcomes of analytics and data processes.

Organizations face various challenges with analytics and business intelligence processes, including data curation and modeling across disparate sources and data warehouses, maintaining data quality and ensuring security and governance. Traditional processes are slow when transforming large and diverse datasets into something which is easily consumable in BI. And, it can take days or weeks to create reports and dashboards — maybe longer if processes change and new data sources are introduced. Our Analytics and Data Benchmark Research shows that the most time-consuming processes are preparing data, reviewing it for quality issues and preparing reports for presentation and distribution.

Today, organizations understand the importance of good external data that can be integrated with internal data to train machine learning models. Our Machine Learning Dynamic Insights research showed that external data adds a significant value in gaining competitive advantage, improving customer experience and increasing sales. But getting the right external data for a particular requirement is not always easy. Internal data is usually not enough to train different models because of its narrow scope of usage and lack of relevance. Manual data acquisition methods are resource-intensive and can take weeks or months to get the data ready to feed into models.

Natural language processing (NLP) is a field that combines artificial intelligence (AI), data science and linguistics that enables computers to understand, interpret and manipulate text or spoken words. NLP includes generating narratives based on a set of data values, using text or speech as inputs to access information, and analysing text or speech, for instance, to determine its sentiment. There are various techniques for interpreting human language, ranging from statistical and machine learning (ML) methods to rules-based and algorithmic approaches. In this perspective, we will focus on two aspects of NLP: natural language query (NLQ), which offers the ability to use natural language expressions to discover and understand data, and natural language generation (NLG), which uses AI to produce written or spoken narratives from a dataset. NLQ and NLG enable business personnel to communicate information needs with business intelligence (BI) systems more easily.