Channel Insider content and product recommendations are editorially independent. We may make money when you click on links to our partners. Learn More.

Big Data challenges

1 - Too Much Big Data VarietyToo Much Big Data Variety

The variety of big data may not be the spice of life after all. In fact, 71% of the data scientists surveyed said new types of data make developing analytics apps more difficult.

2 - Biggest Big Data ChallengeBiggest Big Data Challenge

Big data analytics is more complex than most appreciate: 40% said they struggle with new types of data while 36% said it takes too long to get the answer they are seeking.

3 - Types of Data Being Used in Next YearTypes of Data Being Used in Next Year

Data comes in many structured and unstructured forms. Time series and business transaction data are tied, at 66%, followed by geospatial data, at 55%.

4 - Plans for Big Data AnalyticsPlans for Big Data Analytics

The use of big data analytics applications is already pervasive: 59% have already deployed big data analytics and 31% plan to do so in the next two years.

5 - Data Movement IssuesData Movement Issues

Big data tends to stay where it gets generated. One of the bigger issues for 36% of data scientists is that the amount of data that needs to be analyzed is too big for their organization to move into an application.

6 - Limitations of Hadoop/SparkLimitations of Hadoop/Spark

Hadoop is only one element of the big data equation: 39% said Hadoop is too difficult to program while 37% said it’s too slow for ad hoc queries and 30% said it’s too slow for real-time analytics.

7 - Hadoop/Spark AbandonmentHadoop/Spark Abandonment

Failure rates involving Hadoop are still pretty high: 37% of the data scientists who have tried Hadoop/Spark have abandoned it.

8 - The Role of the Relational DatabaseThe Role of the Relational Database

Many organizations are outgrowing traditional relational databases. Just under half, or 49%, said they are having trouble fitting their data in a relational database.

9 - Big Data Stress FactorsBig Data Stress Factors

Much is asked when much is given. The growth of big data has made the job of being a data scientist more stressful for 39% of data scientists.

10 - The Big Data QuestionThe Big Data Question

A knowledge of science does not always translate into business-savvy insight. Almost a quarter, 24%, said they don’t know what questions to ask when they gain access to all that data.