Big Data Starts to Strain Data Warehouse Platforms

Big Data Starts to Strain Data Warehouse Platforms

Big Data Starts to Strain Data Warehouse Platforms

As data warehouses strain under heavy big data workloads, data warehousing, BI software and advanced analytics will present opportunities for solution providers.

Data Warehouse Platforms in Use

61% said their organizations used a traditional warehousing vendor, while 50% used a hybrid approach that blends legacy technology with big data platforms, such as Hadoop and Spark.

Top Data Warehouse Issues

67% were satisfied with their data warehouses. But 31% said the biggest issue with current data warehousing solutions is that they are not designed for modern analytics on big data, Internet of things and other new types of data, and 29% said it is expensive to maintain as data needs and volumes grow exponentially (29%).

Top BI Issues

58% are satisfied with BI tools. The biggest complaint (32%) was the inability to address big data. Slowness and performance were tied at 21% each as the second-biggest complaints with current BI solutions.

Analytics Challenge

As data volumes increase, execs are struggling to glean meaningful insights from their data (60%), and getting access to and analyzing the right data at the right time is a problem (56%). About a quarter (24%) discard or sample data rather than analyze all of it. Nearly half said they still rely on SQL analytics to help them make sense of big data.

The Evolving Data Warehouse

56% reported that their legacy data warehousing platform is handling the new workloads but experiencing the strain, and 42% said the data warehouse keeps breaking under the weight of ballooning analytic workloads.

Data Warehouse Confidence

50% were very or completely confident their infrastructure could handle a surge in incoming data. In a rare difference of opinion between CEOs and IT execs, 32% of CEOs were “completely confident” in their existing data infrastructure’s ability to handle additional data, compared with just 17% of IT execs.

Top Data Warehouse Concerns

The most important challenges for traditional platforms and BI tools were cost (47%), traditional technologies not being architected to handle modern workloads (46%) and an explosion in new data types (41%).

Data Warehouse Goals

48% report that they need to protect their existing IT investments to avoid a costly “rip and replace.”

Meeting Challenges

To address issues with their companies’ traditional data platforms, 62% of execs said they would add tools that can help them maximize their investments and 45% said they would move excess data into storage to deal with later.

Reasons to Change Data-Management Systems

48% said the inability to scale would prompt them to change data-management systems, and 45% said the systems being too slow would be a reason to switch. Tighter IT budgets that leave little room to scale were referenced by 21% of CEOs and IT pros. Only 27% said their data architectures from five years ago would work well five years from now.