David Menninger's Analyst Perspectives

Artificial intelligence and machine learning are valuable to data and analytics activities. Our research shows that organizations using AI/ML report gaining competitive advantage, improving customer experiences, responding faster to opportunities and threats and improving the bottom line with increased sales and lower costs. No wonder nearly 9 in 10 (87%) research participants report using AI/ML or planning to do so. However, using AI/ML can be challenging. The development of accurate models requires significant amounts of data and highly skilled resources. Only one-quarter (23%) of organizations report they have the AI/ML skills needed. Additionally, less than one-third (31%) report that AI/ML technologies are adequate. Putting AI/ML into production requires many steps — it’s more than just developing a model. Once an initial model is developed, it must be deployed into an operational application to capture its benefits. It must also be maintained to ensure that it remains accurate as data and market conditions change. Each of these tasks are noted by one-quarter of our research participants (26%) as their most significant AI/ML operational challenge. AI/ML projects must integrate with an organization’s IT and applications infrastructure, and require planning from the outset for the deployment of models developed by the data science teams across the enterprise. The discipline required in the deployment of these models relies upon close interaction between data scientists and an organization’s IT development operations team to manage frequent updates to applications. MLOps, short for machine learning operations, can help organizations better manage AI/ML projects. It is the data science complement to DevOps, or development operations. As with DataOps and AnalyticOps, too many of these processes have, in the past, involved ad hoc and manual activities. MLOps is the discipline of making AI/ML activities repeatable and automated. It includes the collection of artifacts and orchestration of processes necessary to deploy and maintain AI/ML models. These include the data pipelines that feed the models, as well as the models themselves. MLOps also incorporates the ongoing evaluation of the accuracy of models, then retraining and redeploying models as necessary. Another aspect of MLOps – perhaps a byproduct of imposing discipline around the processes – is better governance of AI/ML. Since models are no longer deployed using ad hoc processes, the models go through the necessary steps to be approved first. For example, automated tests of accuracy as well as tests to detect bias can be run as part of the process. In addition, because the artifacts are collected, they can be submitted as part of compliance reporting or available for review by regulatory bodies. Many organizations are adopting continuous integration and continuous deployment processes (CI/CD) for the various applications they create. Given the focus on responsiveness and the need to be competitive, adopting a CI/CD approach increases organizational agility. If it takes months or years to modify, test and deploy new software capabilities, how can an organization be responsive? While the discipline of CI/CD requires some new ways of thinking and doing things, it has significant benefit to the organization. Data and analytics activities, including AI/ML, need to fit into these processes. Software vendors recognize this need and offer MLOps capabilities, either as part of AI/ML platforms or as separate offerings. If your organization is pursuing AI/ML — as the majority are — you should consider and evaluate options for MLOps. Without a process for MLOps, AI/ML models may never get deployed, or may languish with mediocrity as they decline in accuracy. Regards, David Menninger

David Menninger's Analyst Perspectives

MLOps: A Disciplined Approach That Increases Organizational Agility

Celonis Improves Business Processes with Process Mining

Process Mining Improves Business Processes

Our Analyst Perspective Policy

Our Analysts

David Menninger

Jeff Orr

Keith Dawson

Mark Smith

Matt Aslett

Quincy Valencia

Robert Kugel

Stephen Hurrell

Ventana Research

Subscribe to Email Updates

Posts by Month

Posts by Topic

Analyst Perspectives Archive

About Us

Links