Apache Spark Adoption Fuels Big Data Analytics Growth

As one of the most vibrant and widely adopted open-source projects, Apache Spark in-memory clusters are driving new opportunities for application development as well as increased consumption of IT infrastructure. Apache Spark is now the most active Apache project, with more than 600 contributions being made in last 12 months by more than 200 organizations. A new survey of 1,417 IT professionals working with Apache Spark conducted by Databricks, a provider of Apache Spark as a cloud service, finds that high-performance analytics applications that can work with massive amounts of big data are driving most of that demand. Apache Spark is now being used to aggregate multiple types of data in-memory versus only pulling data from Hadoop. For solution providers, Apache Spark is significant because it's one of the core technologies used to modernize data warehouses, a huge segment of the IT industry that accounts for multiple billions in revenue. As is often the case with any emerging technology, the expertise needed to transform data warehouses using Apache Spark is presently hard to come by, which may signal a major opportunity for the channel.