Markets have been more volatile than ever. It creates a need for decision makers to utilize technologies such as artificial intelligence and machine learning (AI/ML) to better understand the external factors that impact their business. By identifying these factors, organizations can better plan for changing market environments and seize market opportunities. However, manual modeling is a time-consuming process and results in a limited number of models and tests. Also, updating those models is slow and laborious. With the addition of market volatility, it creates multiple challenges for CFOs, managers and financial planning specialists. With limited exposure to external drivers of demand and delivery, the process becomes very costly. Developing accurate forecasts requires integrating exogenous data with the internal performance data, but it’s challenging to find quality external data and then get that raw data clean enough to input into any model. My colleague, Robert Kugel, recently shared his perspective on using external data for forecasting, budgeting and planning to enhance predictive capabilities.
Organizations conduct data analysis in many ways. The process can include multiple spreadsheets, applications, desktop tools, disparate data systems, data warehouses and analytics solutions. This creates difficulties for management to provide and maintain updated information across multiple departments. Our Analytics and Data Benchmark Research shows that organizations face a variety of challenges with analytics and business intelligence. One-third of participants find it difficult to integrate analytics and BI with other business processes. Participants also find that not all software is flexible enough for the constantly changing business environment, and that it is hard to access all data sources.
In previous perspectives in this series, I’ve discussed some of the realities of cloud computing including costs, hybrid and multi-cloud configurations and business continuity. This perspective examines the realities of security and regulatory concerns associated with cloud computing. These issues are often cited by our research participants as reasons they are not embracing the cloud. To be fair, the majority of our research participants are embracing the cloud. However, among those that have not yet made the transition to the cloud, security and regulatory concerns are among the most common issues cited across the various studies we have conducted.
Recently, I suggested you need to “mind the gap” between data and analytics. This perspective addresses another gap — the gap in skills between business intelligence (BI) and artificial intelligence/machine learning (AI/ML).
Embedded business intelligence (BI) continues to transform the business landscape, enabling organizations to quickly interpret data and convert it into actionable insights. It allows organizations to extract information in real time and answer wide-ranging business questions. Embedding analytics helps tackle the issue of extracting information from data which is a time-consuming process. Our research shows organizations spend more time cleaning and optimizing data for analysis rather than creating insights. On top of that, they are adding more data sources and information systems which in turn introduces more complexity. Our Analytics and Data Benchmark Research shows that organizations face various challenges with analytics and BI. More than one-third of participants (35%) responded that they find it hard to integrate analytics and BI with business processes and connect to multiple data sources. By embedding analytics and BI into business processes and workflows, organizations can enable users to make critical decisions fast, enhancing overall business agility.
Artificial intelligence and machine learning are valuable to data and analytics activities. Our research shows that organizations using AI/ML report gaining competitive advantage, improving customer experiences, responding faster to opportunities and threats and improving the bottom line with increased sales and lower costs. No wonder nearly 9 in 10 (87%) research participants report using AI/ML or planning to do so.
As I recently pointed out, process mining has emerged as a pivotal technology for data-driven organizations to discover, monitor and improve processes through use of real-time event data, transactional data and log files. With recent advancements, process mining has become more efficient at discovering insights in complex processes using algorithms and visualizations. Organizations use it to better understand the current state of systems and business processes. It is also used to enable business process intelligence and improvement in any function or industry using events and activity models for data-driven decision-making. We assert that through 2024, 1 in 4 organizations will look to streamline their operations by exploring process mining to optimize workflow and business processes.
Process mining is defined as the analysis of application telemetry including log files, transaction data and other instrumentation to understand and improve operational processes. Log data provides an abundance of information about what operations are occurring, the sequences involved in the processes, how long the processes are taking and whether or not the processes are completed successfully. As computing power has increased and storage costs have decreased, the economics of collecting and analyzing large amounts of log data have become much more attractive.
Organizations are collecting data from multiple data sources and a variety of systems to enrich their analytics and business intelligence (BI). But collecting data is only half of the equation. As the data grows, it becomes challenging to find the right data at the right time. Many organizations can’t take full advantage of their data lakes because they don’t know what data actually exists. Also, there are more regulations and compliance requirements than ever before. It is critical for organizations to understand the kind of data they have, who is handling it, what it is being used for and how it needs to be protected. They also have to avoid putting too many layers and wrappers around the data as it can make the data difficult to access. These challenges create a need for more automated ways to discover, track, research and govern the data.
Business intelligence has evolved. It now includes a spectrum of analytics, one of the most promising of which has been described as augmented intelligence. Some organizations have used the term to describe the practical reality that artificial intelligence with machine learning is not replacing human intelligence, but augmenting it. The term also represents the application of AI/ML to make business intelligence and analytics tools more powerful and easier to use. It’s this latter usage that I prefer and I’d like to explore in this perspective.