David Menninger's Analyst Perspectives

I have written previously that the world of data and analytics will become more and more centered around real-time, streaming data. Data is created constantly and increasingly is being collected simultaneously. Technology advances now enable organizations to process and analyze information as it is being collected to respond in real time to opportunities and threats. Not all use cases require real-time analysis and response, but many do, including multiple use cases that can improve customer experiences. For example, best-in-class e-commerce interactions should provide real-time updates on inventory status to avoid stock-out or back-order situations. Customer service interactions should provide real-time recommendations that minimize the time to resolution. Location-based offers should be targeted at the customer’s current location, not their location several minutes ago. Another domain where real-time analyses are critical is internet of things (IoT) applications. Additionally, use cases like predictive maintenance require timely information to prevent equipment failures that help avoid additional costs and damage.

Our Analytics and Data Benchmark Research shows almost one-quarter (22%) of participants conduct real-time analyses of their data. Another 10% are analyzing data at least hourly. The research also shows a correlation between the frequency of analysis and an organization’s satisfaction with their analytics. Organizations performing real-time analyses report the highest levels of satisfaction. Those performing weekly or monthly analyses report the lowest levels of satisfaction. This correlation is not surprising. Individuals need timely analysis to inform their decision-making processes. We assert that by 2024, one-half of organizations will incorporate streaming analytics into their business processes, enabling them to respond to opportunities and threats faster.

The research also explores whether organizations have adequate technologies to support different types of analyses. Half as many organizations (19%) report they have completely adequate technologies for real-time analysis compared with those that have completely adequate technologies for reports (38%) and dashboards (38%). Real-time analyses require a different approach. Having someone sit around and watch a dashboard waiting for information to change is not a very effective way to monitor changes. The data may be changing so frequently that it is hard to even see what is changing. Even if someone were to identify and understand the changes, would they be able to determine the correct response before it was too late?

A proactive rather than reactive approach is needed. At a minimum, alerts and notifications can be used to draw attention to issues as they arise rather than expecting someone to watch a dashboard. Unfortunately, the technologies for alerts and notifications are not considered to be much better than the technologies for real-time analysis. It only solves one part of the problem — identifying an issue. Determining and implementing the correct response in real time is the other part of the challenge. In simple cases, evaluating and selecting the correct response can be done with heuristics or straightforward logic. However, it is more likely that artificial intelligence and machine learning (AI/ML) will be needed to effectively evaluate and select an appropriate response. Once an appropriate response is selected, it may need to be reviewed and approved (perhaps via an alert) and then implemented. The implementation may be a recommendation on a customer service screen, it may be updating the inventory status on an e-commerce screen, it may be placing a replenishment order, or it may be a combination of several actions.

As organizations become more proactive and oriented around real-time processes, they need to adopt an event-driven approach to their operations. Nearly one-half (48%) are using streaming data in their operational processes, yet only 15% report their event-streaming technologies are completely adequate. Real-time and streaming require a focus on event-driven architectures to support responsiveness and are on the path to what we call a digital business.

Analytics vendors must recognize the needs organizations have for real-time operations. They will not be solved by reports and dashboards. Organizations need better alert and notification capabilities. They also require improved AI/ML capabilities to evaluate the appropriate actions to take. And they need enhanced agent technologies to connect the various applications — analytical and operational — involved in taking action. In the meantime, organizations should consider third-party technologies for some of these capabilities. We continue to study the issues associated with real-time analyses and event-driven architectures in our Streaming Data Dynamic Insights Research. Please contribute to this research with input from your organization.

Regards,

David Menninger