Organizations are dealing with exponentially increasing data that ranges broadly from customer-generated information, financial transactions, edge-generated data and even operational IT server logs. A combination of complex data lake and data warehouse capabilities are required to leverage this data. Our research shows that nearly three-quarters of organizations deploy both data lakes and data warehouses but are using a variety of approaches which can be cumbersome. A single platform that can provide both capabilities will help address organizations’ requirements.
Businesses are transforming their organizations, building a data culture and deploying sophisticated analytics more broadly than ever. However, the process of using data and analytics is not always easy. The necessary tools are often separate, but our research shows organizations prefer an integrated environment. In our Data Preparation Benchmark Research, we found that 41% of participants use Analytics and Business Intelligence tools for data preparation.
Topics: embedded analytics, Analytics, Business Intelligence, Collaboration, Data Preparation, Information Management, Internet of Things, Data, Digital Technology, natural language processing, Conversational Computing, AI and Machine Learning
Traditional on-premises data processing solutions have led to a hugely complex and expensive set of data silos where IT spends more time managing the infrastructure than extracting value from the data. Big data architectures have attempted to solve the problem with large pools of cost-effective storage, but in doing so have often created on-premises management and administration challenges. These challenges of acquiring, installing and maintaining large clusters of computing resources gave rise to cloud-based implementations as an alternative. Public cloud is becoming the new center for data as organizations migrate from static on-premises IT architectures to global, dynamic and multi-cloud architectures.
Organizations are always looking to improve their ability to use data and AI to gain meaningful and actionable insights into their operations, services and customer needs. But unlocking value from data requires multiple analytics workloads, data science tools and machine learning algorithms to run against the same diverse data sets. Organizations still struggle with limited data visibility and insufficient insights, which are often caused by a multitude of reasons such as analytic workloads running independently, data spread across multiple data centers, data governance, etc. In our ongoing benchmark research project, we are researching the ways in which organizations work with big data and the challenges they face.
A data lake is a centralized repository designed to house big data in structured, semi-structured and unstructured form. I have been covering the data lake topic for several years and encourage you to check out an earlier perspective called Data Lakes: Safe Way to Swim in Big Data? for background. Our data lake research has uncovered some points to consider in your efforts, and I’d like to offer a deeper dive into our findings.
Every organization performing analytics with multiple employees needs to collaborate. They should be collaborating in the analytics process and in communicating the results of those analyses. As I continue my evaluation of analytics and data vendors, I have to admit some disappointment at the level of collaborative capabilities some analytics vendors provide. To be fair, the level of capabilities vary widely, but I expected collaborative capabilities to be more uniformly available as a standard feature in analytics technologies by now. I had anticipated that three-quarters of analytics vendors would include collaboration capabilities. More than half the vendors I have evaluated support some comments and discussion in their products, only a few have incorporated social recognition and wall posting as part of their collaborative capabilities. So, what impact does a lack of analytics collaboration have on organizations undergoing digital transformation?
For decades, data integration was a rigid process. Data was processed in batches once a month, once a week or once a day. Organizations needed to make sure those processes were completed successfully—and reliably—so they had the data necessary to make informed business decisions. The result was battle-tested integrations that could withstand the test of time.
Ventana Research has been evaluating analytics and business intelligence (BI) software for a long time—almost 20 years. Our methodology for these assessments is referred to as a Value Index. We use weightings derived from our benchmark research about how you, as buyers of these technologies, value and evaluate vendors. You can view our 2019 Value Index results here. I am in the process of completing the 2020 evaluation now.
Topics: business intelligence, embedded analytics, Analytics, Collaboration, Data Governance, Data Preparation, Information Management (IM), natural language processing, Conversational Computing, AI and Machine Learning, collaborative computing, software evaluation
In this analyst perspective, Dave Menninger takes a look at data lakes. He explains the term “data lake,” describes common use cases and shares his views on some of the latest market trends. He explores the relationship between data warehouses and data lakes and share some of Ventana Research’s findings on the subject. He also provides an assessment of the risks organizations face in working with data lakes and offers recommendations for maximizing the potential of data.
Effectively managing data privacy and security is a high-stakes matter. When an organization doesn’t get it right, it often becomes front-page news and occasionally becomes a subject of litigation. Yet organizations face an equally challenging imperative to ensure that business users have easy access to the data they need. Depending on how they are implemented, data governance policies can inhibit access to data, making it harder to find and utilize the data assets of an organization.