Services for Organizations

Using our research, best practices and expertise, we help you understand how to optimize your business processes using applications, information and technology. We provide advisory, education, and assessment services to rapidly identify and prioritize areas for improvement and perform vendor selection

Consulting & Strategy Sessions

Ventana On Demand

    Services for Investment Firms

    We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

    Consulting & Strategy Sessions

    Ventana On Demand

      Services for Technology Vendors

      We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

      Analyst Relations

      Demand Generation

      Product Marketing

      Market Coverage

      Request a Briefing



        David Menninger's Analyst Perspectives

        << Back to Blog Index

        Reversing the ETL Process Improves Data Operations

        At one point, analytics and business intelligence were considered non-mission critical activities. One of the primary concerns in designing analytics systems was to ensure they didn’t interfere with or draw computing resources away from operational systems. But today, analytical systems are integral to many aspects of operations. More than 9 in 10 participants in our Analytics and Data Benchmark Research reported analytics had improved activities and processes. However, most analytics and BI software products are still designed as read-only systems, and the process of using their output is either a manual process or involves custom coding or scripting to provide inputs to the operational systems. The most common complaint about analytics and BI technologies is that they are hard to integrate with business processes. In this perspective, we’ll look at how the recent trend of “reverse ETL” helps organizations utilize the output of analytic processes more effectively in operations.  

        When data warehouses emerged, ETL – extract, transform and load – became the pattern for data integration. Source data is extracted from operational systems, transformed in flight and loaded into the data warehouse. Ventana_Research_BR_Analytics_and_Data_05_complaints_2023As data lakes arrived on the scene, ELT – extract, load and transform – became another process where data is extracted from source systems, loaded into a highly scalable data platform and processed there rather than in a separate system.

        Now, several solutions, including a new software category called reverse ETL, are emerging for moving information from analytical data warehouses and BI tools into the operational systems they are supporting. Reverse ETL applications are based on the premise that information stored in data warehouses is valuable for operational activities and some of it should be moved into those systems. For example, data warehouses often enrich customer data with metrics such as segmentation scores, propensity to respond to different offers and external data such as demographic information. If this data is available within operational systems, it can be used directly in customer service interactions or marketing campaigns.

        Over the past few years, I’ve noticed analytics vendors starting to recognize the need to move information into operational systems. Most of the solutions are limited, but others are more systematic. The more limited solutions invoke custom scripts from the analytics systems to move information. The more systematic solutions involve two-way connectors with the source systems. Many BI tools have connectors to gather data from source systems. Several vendors now offer two-way connectors to either pull information from source systems or push information to those systems. Simultaneously, a new category of vendors is developing reverse ETL tools independent of the analytics and BI systems. As the name implies, reverse ETL systems extract data from the data warehouse, perhaps perform light transformation and load it into operational systems.

        There are some challenges with each approach. Custom scripts or application programming interface calls may be useful for moving small amounts of information, but often, large volumes of data must be moved into the source systems. Two-way connectors associated with BI tools can move larger amounts of information but require careful governance so data from the operational system is not corrupted. Reverse ETL systems are primarily designed to work with data from the data warehouse. Often, valuable information exists in the analytics and BI tools that is materialized via calculations. That information wouldn’t be available to the reverse ETL tools.

        I welcome the developments in this area. BI needs to be actionable, but actions are rarely taken from the BI tools. They are implemented in systems for marketing, customer service, human resources, supply chain and other operational systems. Keep an eye on this area as it develops. Explore the options associated with your information architecture. As applications become available to replace custom coding or scripts, consider switching as a way to scale the value of analytics and BI investments.

        Regards,

        David Menninger

        Authors:

        David Menninger
        Executive Director, Technology Research

        David Menninger leads technology software research and advisory for Ventana Research, now part of ISG. Building on over three decades of enterprise software leadership experience, he guides the team responsible for a wide range of technology-focused data and analytics topics, including AI for IT and AI-infused software.

        JOIN OUR COMMUNITY

        Our Analyst Perspective Policy

        • Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

          Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to ChiefResearchOfficer@ventanaresearch.com

        View Policy

        Our Analysts

        Subscribe to Email Updates

        Posts by Month

        see all

        Posts by Topic

        see all


        Analyst Perspectives Archive

        See All