Services for Organizations

Using our research, best practices and expertise, we help you understand how to optimize your business processes using applications, information and technology. We provide advisory, education, and assessment services to rapidly identify and prioritize areas for improvement and perform vendor selection

Consulting & Strategy Sessions

Ventana On Demand

    Services for Investment Firms

    We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

    Consulting & Strategy Sessions

    Ventana On Demand

      Services for Technology Vendors

      We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

      Analyst Relations

      Demand Generation

      Product Marketing

      Market Coverage

      Request a Briefing



        David Menninger's Analyst Perspectives

        << Back to Blog Index

        MLOps: A Disciplined Approach That Increases Organizational Agility

        Artificial intelligence and machine learning are valuable to data and analytics activities. Our research shows that organizations using AI/ML report gaining competitive advantage, improving customer experiences, responding faster to opportunities and threats and improving the bottom line with increased sales and lower costs. No wonder nearly 9 in 10 (87%) research participants report using AI/ML or planning to do so.

        Ventana_Research_Analytics_and_Data_Benchmark_Research_AI_AdoptionHowever, using AI/ML can be challenging. The development of accurate models requires significant amounts of data and highly skilled resources. Only one-quarter (23%) of organizations report they have the AI/ML skills needed. Additionally, less than one-third (31%) report that AI/ML technologies are adequate.

        Putting AI/ML into production requires many steps — it’s more than just developing a model. Once an initial model is developed, it must be deployed into an operational application to capture its benefits. It must also be maintained to ensure that it remains accurate as data and market conditions change. Each of these tasks are noted by one-quarter of our research participants (26%) as their most significant AI/ML operational challenge.

        AI/ML projects must integrate with an organization’s IT and applications infrastructure, and require planning from the outset for the deployment of models developed by the data science teams across the enterprise. The discipline required in the deployment of these models relies upon close interaction between data scientists and an organization’s IT development operations team to manage frequent updates to applications.

        MLOps, short for machine learning operations, can help organizations better manage AI/ML projects. It is the data science complement to DevOps, or development operations. As with DataOps and AnalyticOps, too many of these processes have, in the past, involved ad hoc and manual activities. MLOps is the discipline of making AI/ML activities repeatable and automated. It includes the collection of artifacts and orchestration of processes necessary to deploy and maintain AI/ML models. These include the data pipelines that feed the models, as well as the models themselves. MLOps also incorporates the ongoing evaluation of the accuracy of models, then retraining and redeploying models as necessary.

        Another aspect of MLOps – perhaps a byproduct of imposing discipline around the processes – is better governance of AI/ML. Since models are no longer deployed using ad hoc processes, the models go through the necessary steps to be approved first. For example, automated tests of accuracy as well as tests to detect bias can be run as part of the process. In addition, because the artifacts are collected, they can be submitted as part of compliance reporting or available for review by regulatory bodies.

        Many organizations are adopting continuous integration and continuous deployment processes (CI/CD) for the various applications they create. Given the focus on responsiveness and the need to be competitive, adopting a CI/CD approach increases organizational agility. If it takes months or years to modify, test and deploy new software capabilities, how can an organization be responsive? While the discipline of CI/CD requires some new ways of thinking and doing things, it has significant benefit to the organization.

        Data and analytics activities, including AI/ML, need to fit into these processes. Software vendors recognize this need and offer MLOps capabilities, either as part of AI/ML platforms or as separate offerings. If your organization is pursuing AI/ML — as the majority are — you should consider and evaluate options for MLOps. Without a process for MLOps, AI/ML models may never get deployed, or may languish with mediocrity as they decline in accuracy.

        Regards,

        David Menninger

        Authors:

        David Menninger
        Executive Director, Technology Research

        David Menninger leads technology software research and advisory for Ventana Research, now part of ISG. Building on over three decades of enterprise software leadership experience, he guides the team responsible for a wide range of technology-focused data and analytics topics, including AI for IT and AI-infused software.

        JOIN OUR COMMUNITY

        Analyst Perspective Policy

        • Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

          Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to ChiefResearchOfficer@ventanaresearch.com

        View Policy

        Our Analysts

        Subscribe to Email Updates

        Posts by Month

        see all

        Posts by Topic

        see all


        Analyst Perspectives Archive

        See All