The data governance landscape is growing rapidly. Organizations handling vast amounts of data face multiple challenges as more regulations are added to govern sensitive information. Adoption of multi-cloud strategies increases governance concerns with new data sources that are accessed in real time. Our Data Governance Benchmark Research shows that organizations face multiple challenges when deploying data governance. Three-quarters (73%) of organizations report disparate data sources as the biggest challenge, and half of the organizations report creating, modifying, managing and enforcing governance policies as the second biggest challenge.
Organizations have been using data virtualization to collect and integrate data from various sources, and in different formats, to create a single source of truth without redundancy or overlap, thus improving and accelerating decision-making giving them a competitive advantage in the market. Our research shows that data virtualization is popular in the big data world. One-quarter (27%) of participants in our Data Lake Dynamic Insights Research reported they were currently using data virtualization, and another two-quarters (46%) planned to include data virtualization in the future. Even more interesting, those who are using data virtualization reported higher rates of satisfaction (79%) with their data lake than those who are not (36%). Our Analytics and Data Benchmark Research shows more than one-third of organizations (37%) are using data virtualization in that context. Here, too, those using data virtualization reported higher levels of satisfaction (88%) than those that are not (66%).
I have written previously that the world of data and analytics will become more and more centered around real-time, streaming data. Data is created constantly and increasingly is being collected simultaneously. Technology advances now enable organizations to process and analyze information as it is being collected to respond in real time to opportunities and threats. Not all use cases require real-time analysis and response, but many do, including multiple use cases that can improve customer experiences. For example, best-in-class e-commerce interactions should provide real-time updates on inventory status to avoid stock-out or back-order situations. Customer service interactions should provide real-time recommendations that minimize the time to resolution. Location-based offers should be targeted at the customer’s current location, not their location several minutes ago. Another domain where real-time analyses are critical is internet of things (IoT) applications. Additionally, use cases like predictive maintenance require timely information to prevent equipment failures that help avoid additional costs and damage.
For years, maybe decades, we have heard about the struggles between IT and line-of-business functions. In this perspective, we will look at some of the data from our Analytics and Data Benchmark Research about the roles of IT and line-of-business teams in analytics and data processes. We will also look at some of the disconnects between these two groups. And, by looking at how organizations are operating today and the results they are achieving, we can discern some of the best practices for improving the outcomes of analytics and data processes.
Organizations face various challenges with analytics and business intelligence processes, including data curation and modeling across disparate sources and data warehouses, maintaining data quality and ensuring security and governance. Traditional processes are slow when transforming large and diverse datasets into something which is easily consumable in BI. And, it can take days or weeks to create reports and dashboards — maybe longer if processes change and new data sources are introduced. Our Analytics and Data Benchmark Research shows that the most time-consuming processes are preparing data, reviewing it for quality issues and preparing reports for presentation and distribution.
Today, organizations understand the importance of good external data that can be integrated with internal data to train machine learning models. Our Machine Learning Dynamic Insights research showed that external data adds a significant value in gaining competitive advantage, improving customer experience and increasing sales. But getting the right external data for a particular requirement is not always easy. Internal data is usually not enough to train different models because of its narrow scope of usage and lack of relevance. Manual data acquisition methods are resource-intensive and can take weeks or months to get the data ready to feed into models.
Natural language processing (NLP) is a field that combines artificial intelligence (AI), data science and linguistics that enables computers to understand, interpret and manipulate text or spoken words. NLP includes generating narratives based on a set of data values, using text or speech as inputs to access information, and analysing text or speech, for instance, to determine its sentiment. There are various techniques for interpreting human language, ranging from statistical and machine learning (ML) methods to rules-based and algorithmic approaches. In this perspective, we will focus on two aspects of NLP: natural language query (NLQ), which offers the ability to use natural language expressions to discover and understand data, and natural language generation (NLG), which uses AI to produce written or spoken narratives from a dataset. NLQ and NLG enable business personnel to communicate information needs with business intelligence (BI) systems more easily.
Despite all the advances organizations have made with respect to analytics, our most recent research shows the majority of the workforce in the majority of organizations are not using analytics and business intelligence (BI). Less than one-quarter (23%) report that one-half or more of their workforce is using analytics and BI. This is a problem. It means organizations are not enabling their workforce to perform at peak efficiency and effectiveness. It means the workforce in many organizations does not have access to the same information by which they are being measured. It means organizations must find other ways to communicate with, and manage, the workforce.
Topics: Sales, embedded analytics, Analytics, Business Intelligence, Data, Sales Performance Management, Digital Technology, Digital Commerce, natural language processing, subscription management, partner management, sales engagement, revenue management, Collaborative & Conversational Computing
Many organizations invest in data governance out of concern over misuse of data or potential data breaches. These are important considerations and valid aspects of data governance programs. However, good data governance also has positive impacts on organizations. For example, I have previously written about the valuable connection between the use of data catalogs and satisfaction with an organization’s data lake. Our most recent Analytics and Data Benchmark Research demonstrates some of the beneficial links between data governance and analytics. In this Perspective, I’ll share some of the correlations identified in our research.
Organizations of all sizes are dealing with exponentially increasing data volume and data sources, which creates challenges such as siloed information, increased technical complexities across various systems and slow reporting of important business metrics. Migrating to the cloud does not solve the problems associated with performing analytics and business intelligence on data stored in disparate systems. Also, the computing power needed to process large volumes of data consists of clusters of servers with hundreds or thousands of nodes that can be difficult to administer. Our Analytics and Data Benchmark Research shows that organizations have concerns about current analytics and BI technology. Findings include difficulty integrating data with other business processes, systems that are not flexible enough to scale operations and trouble accessing data from various data sources.