Services for Organizations

Using our research, best practices and expertise, we help you understand how to optimize your business processes using applications, information and technology. We provide advisory, education, and assessment services to rapidly identify and prioritize areas for improvement and perform vendor selection

Consulting & Strategy Sessions

Ventana On Demand

    Services for Investment Firms

    We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

    Consulting & Strategy Sessions

    Ventana On Demand

      Services for Technology Vendors

      We provide guidance using our market research and expertise to significantly improve your marketing, sales and product efforts. We offer a portfolio of advisory, research, thought leadership and digital education services to help optimize market strategy, planning and execution.

      Analyst Relations

      Demand Generation

      Product Marketing

      Market Coverage

      Request a Briefing



        David Menninger's Analyst Perspectives

        << Back to Blog Index

        The World of Big Data Gets Even Bigger at Hadoop World

        Cloudera’s recent Hadoop World 2011 event confirmed that the world of big data is getting even bigger. As I wrote of last year’s event, Hadoop, the open source large-scale data processing technology, has gone mainstream. And while 75% of the audience attended this year for the first time and so may not have realized the breadth of Hadoop’s acceptance, statistics announced in the opening keynote show widespread use of it. Mike Olson, Cloudera CEO, reported that the event was sold out, with 1,400 attendees from 580 organizations and 27 countries. In independent confirmation, our benchmark research shows that 54% of organizations are either using or evaluating Hadoop for their big-data needs.

        Before or during Hadoop World, several vendors made announcements that further reinforced the growth of the market. Cloudera announced it has raised an additional $40 million to expand its operations. Oracle, a sponsor of the event, introduced its Big Data Appliance, which includes Hadoop. NetApp announced a partnership with Cloudera to provide a preconfigured, appliance-like solution called NetApp Open Solution for Hadoop. Hortonworks, a Cloudera competitor, unveiled its own distribution of Hadoop called Hortonworks Data Platform.

        Despite these announcements, one of the main impressions I took away from the event is that this emperor has no clothes. If you recall the story, the fact that the emperor had no clothes was not a metaphor of his authority but a question, literally and figuratively, about whom and what he had surrounded himself with. By many accounts, Hadoop is in a similar situation. It is a powerful but immature technology that has grown popular despite its shortcomings. In his keynote, Olson acknowledged this, saying that it’s not enough to provide a platform for Java developers. Doug Cutting, Cloudera’s architect and creator of Hadoop, described it as the kernel of a distributed operating system for big data. Not many end users are prepared to work directly with the kernel of an operating system.

        Those users need the technology to be easier to handle and more broadly accessible. This issue shows up in our research finding that staffing and training are the two biggest obstacles to analyzing large-scale data sets with Hadoop. MapR, another Cloudera competitor with its own Hadoop distribution, is trying to capitalize on the need to overcome this obstacle by offering free training resources. And some vendors recognize the need to surround Hadoop with “better clothes.” Karmasphere and Datameer provide tools that make Hadoop easier to use in the analytics process. Informatica recently announced HParser, which makes it easier to parse the unstructured data often collected and analyzed with Hadoop. I spoke with other vendors at the event that are still in stealth mode, but we can expect continued development in the Hadoop ecosystem. Giving this trend momentum, venture capitalist Accel Partners announced a $100 million fund to invest in big-data companies.

        This market is evolving with many moving parts. Hadoop is not one thing, but a collection of multiple projects. Hadoop’s distributed file system (HDFS) and MapReduce have been the cornerstones of Hadoop adoption; the majority of organizations use those two components, our research confirms. HBase, a columnar database built on HDFS, received a lot of attention at the event and was the subject of several presentations, including one about Facebook using HBase for real-time data access. Attendees were offered a free copy of HBase: The Definitive Guide, and several commented that perhaps the event should have been called HBase World.

        So while the emperor has no clothes, there are plenty of tailors making suits. I expect increasing competition among different distributions of Hadoop and among existing and new tool vendors trying to make Hadoop easier to use. The advantage of all this interest in Hadoop is that the open source community is aware of many of the platform’s issues and is working to resolve them. The disadvantage is that with all the separate components and so many competitors that it will continue to be a confusing landscape until the market matures further.

        Regards,

        David Menninger – VP & Research Director

        Authors:

        David Menninger
        Executive Director, Technology Research

        David Menninger leads technology software research and advisory for Ventana Research, now part of ISG. Building on over three decades of enterprise software leadership experience, he guides the team responsible for a wide range of technology-focused data and analytics topics, including AI for IT and AI-infused software.

        JOIN OUR COMMUNITY

        Our Analyst Perspective Policy

        • Ventana Research’s Analyst Perspectives are fact-based analysis and guidance on business, industry and technology vendor trends. Each Analyst Perspective presents the view of the analyst who is an established subject matter expert on new developments, business and technology trends, findings from our research, or best practice insights.

          Each is prepared and reviewed in accordance with Ventana Research’s strict standards for accuracy and objectivity and reviewed to ensure it delivers reliable and actionable insights. It is reviewed and edited by research management and is approved by the Chief Research Officer; no individual or organization outside of Ventana Research reviews any Analyst Perspective before it is published. If you have any issue with an Analyst Perspective, please email them to ChiefResearchOfficer@ventanaresearch.com

        View Policy

        Our Analysts

        Subscribe to Email Updates

        Posts by Month

        see all

        Posts by Topic

        see all


        Analyst Perspectives Archive

        See All