It has been more than five years since James Dixon of Pentaho coined the term “data lake.” His original post suggests, “If you think of a data mart as a store of bottled water – cleansed and packaged and structured for easy consumption – the data lake is a large body of water in a more natural state.” The analogy is a simple one, but in my experience talking with many end users there is still mystery surrounding the concept. In this post I’d like to clarify what a data lake is, review the reasons an organization might consider using one and the challenges they present, and outline some developments in software tools that support data lakes.
Topics: Big Data, Data Science, Predictive Analytics, Social Media, Business Analytics, Business Intelligence, Data Governance, Data Lake, Governance, Risk & Compliance (GRC), Information Management, Uncategorized, Strata+Hadoop
On Monday, March 21, Informatica, a vendor of information management software, announced Big Data Management version 10.1. My colleague Mark Smith covered the introduction of v. 10.0 late last year, along with Informatica’s expansion from data integration to broader data management. Informatica’s Big Data Management 10.1 release offers new capabilities, including for the hot topic of self-service data preparation for Hadoop, which Informatica is calling Intelligent Data Lake. The term “data lake” describes large collections of detailed data from across an organization, often stored in Hadoop. With this release Informatica seeks to add more enterprise capabilities to data lake implementations.
I recently attended the SAS Analyst Summit in Steamboat Springs, Colo. (Twitter Hashtag #SASSB) The event offers an occasion for the company to discuss its direction and to assess its strengths and potential weaknesses. SAS is privately held, so customers and prospects cannot subject its performance to the same level of scrutiny as public companies, and thus events like this one provide a valuable source of additional information.
Topics: Big Data, Predictive Analytics, SAS, Analytics, Business Analytics, Business Collaboration, Business Intelligence, Business Performance, Information Applications, Information Management, Uncategorized, Visualization
Last week I attended Spark Summit East 2016 at the New York Hilton Midtown. It revealed several ways in which Spark technology might impact the big data market.
The big data market continues to expand and enable new types of analyses, new business models and new revenues streams for organizations that implement these capabilities. Following our previous research into big data and information optimization, we’ll investigate the technology trends affecting both of these domains as part of our 2016 research agenda.
Some followers of Ventana Research may recall my work here several years ago. Here and elsewhere I have spent most of my career in the data and analytics markets matching user requirements with technologies to meet those needs. I’m happy to be returning to Ventana Research to resume investigating ways in which organizations can make the most of their data to improve their business processes; for a first look, please see our 2016 research agenda on Big Data and Information Optimization. I relish the opportunity to conduct primary market research in the form of Ventana’s well-known benchmark research and to help end users and vendors apply the information collected in those studies.
Topics: Big Data, Predictive Analytics, Analytics, Business Analytics, Business Intelligence, Information Management, Internet of Things, Operational Intelligence, Uncategorized, Unicorns, IT Performance Management (ITPM)
Throughout the course of our research in 2016, we’ll be exploring ways in which organizations can maximize the value of their data. Ventana Research believes that analytics is the engine and data is the fuel to power better business decisions. Several themes emerged from our benchmark research on incorporating data and analytics into organizational processes, and we will follow them in our 2016 Business Analytics Research Agenda:
Topics: Big Data, Predictive Analytics, Analytics, Business Analytics, Business Collaboration, Business Intelligence, Cloud Computing, Information Applications, Information Management, Operational Intelligence
In our definition, information management encompasses the acquisition, organization, dissemination and use of information by organizations to create and enhance business value. Effective information management ensures optimal access, relevance, timeliness, quality and security of this data with the aim to improve organizational performance. This goal is not easily met, especially as organizations acquire ever more data at an ever faster pace. In our business analytics benchmark research of more than 2,600 organizations, almost half (45%) have to integrate six or more types of data in their analyses. More than two-thirds reported that they spend more time preparing data than analyzing it. To assist in dealing with these sorts of issues and others, we’ve laid out an ambitious information management research agenda for 2012.
Topics: Data Quality, Master Data Management, Social Media, Analytics, Business Analytics, Business Intelligence, Cloud Computing, Complex Event Processing, Data Governance, Data Integration, Information Applications, Information Life Cycle Management, Information Management, Operational Intelligence, IT Performance Management (ITPM)
For most people involved with business intelligence (BI), these are exciting times. Using BI to improve business processes continues to motivate organizations to invest in BI. The focus on BI also empowers business analytics and can be rented in the cloud computing model of accessing software. New technologies are adding dimensions to BI and creating both excitement and confusion for enterprises implementing them. We offer a variety of accomplished research that can help organizations overcome the hype and understand how to use these technologies to improve business decision-making, and we’re planning new research in 2012 on these topics.
Topics: Mobile Business Intelligence, Predictive Analytics, Sales Performance, Social Media, Supply Chain Performance, Business Analytics, Business Intelligence, Business Performance, Collaboration, Customer & Contact Center, Financial Performance, Information Management, Operational Intelligence, Workforce Performance
My colleague Mark Smith and I recently attended data integration vendor Informatica’s annual industry analyst event. The company offered some impressive numbers regarding growth and profitability over the years, with 30 consecutive quarters of growth even during the recent recession. Through acquisition and its own research and development activities Informatica now has a broad portfolio of products. It includes data integration and supporting migration, replication and synchronization needs, master data management, complex event processing and other elements of the information management spectrum. As at last year’s event, the company retains a sharp focus on the data integration related portfolio, and its product roadmap addresses four key themes impacting that market: big data, cloud computing, social media and mobile technology. We also see these themes as significant technology trends, and our approach is outlined in our 2012 research agendas for information management and in the larger business technology innovation agenda. Thus it was interesting to hear Informatica’s take on them.
Topics: Big Data, Sales Performance, Social Media, Supply Chain Performance, Business Analytics, Business Intelligence, Business Performance, Cloud Computing, Customer & Contact Center, Data Integration, Financial Performance, Informatica, Information Management, Workforce Performance