Big Data Opportunities Loom Large for the Channel

By Michael Vizard
big data analytics opportunities and challenges

Big data analytics opportunities appear to be in the eye of the beholder. While the hype surrounding big data has been generally high, the ability of most organizations to implement an analytics strategy that incorporates most of their data has been fairly limited.

Data fragmentation poses a major challenge. In a new survey of 402 business and IT professionals conducted by the industry IT association CompTIA, 45 percent said that a high degree of their data is fragmented, with another 42 percent conceding their data fragmentation issues are at least moderate.

Overall, the CompTIA study finds that just over half the organizations surveyed have at least one big data project in play, with another 36 percent reporting they are in the planning stages.

Given that adoption rate, big data analytics represents a major challenge and opportunity for solution providers. Although most of the big data focus to date has been on platforms such as Hadoop, in practice, big data analytics opportunities span a panoply of products and services that solution providers need to master.

Grasping Big Data Analytics Opportunities and Challenges

It's important to understand the nuances of big data analytics opportunities and challenges.

"Big data encompasses a range of products across the spectrum," said Seth Robinson, senior director for technology analysis at CompTIA. "There are a whole range of issues involving, for example, storage and security."

The CompTIA survey makes it clear that businesses are already deriving value from their big data investments. A full 72 percent of the organizations that have started a big data project report that their projects have exceeded expectations.

A similar survey conducted by Teradata in partnership with McKinsey Consulting showed that roughly a quarter of decision-makers report that they are starting to see significant returns on their big data investments in the form of increased revenue and reduced costs.

However, much work remains to be done. Approximately three-quarters of organizations CompTIA surveyed said their businesses would be stronger if they could harness all their data, while 73 percent said they need better real-time analysis.

That latter requirement is driving many organizations to wrap a variety of complementary technologies around Hadoop, ranging from the Apache Spark in-memory clustering software to databases from vendors such as SAP and MarkLogic. In fact, one of the things driving so much interest in Apache Spark as an alternative to MapReduce as a programming tool is that it can be applied directly against multiple data sources

MarkLogic CEO Gary Bloom said what most companies still don't fully appreciate about big data is what it takes to operationalize that data inside their businesses. Most organizations, Bloom said, can make use of Hadoop today to analyze what's already occurred in their business in a batch mode process. However, it requires a database that can process transactions and analytics at the same time to operationalize data in a way that allows business to act on it in real time, for example, to curtail fraud, Bloom said.

"The MarkLogic database is designed to ingest and index data in real time," Bloom said. "Hadoop is really just about consolidating data."

Big Data Analytics Tools and Techniques

However, Bloom isn't suggesting that organizations shouldn't continue to invest in Hadoop. It's just that many organizations will soon discover that Hadoop as a vehicle for addressing data fragmentation is only a piece of the much bigger data management puzzle that is made up of a variety of big data analytics tools and techniques, Bloom contends.

In fact, Mike Flannagan, vice president of data and analytics for Cisco, noted that  the processing of big data will require a more nuanced approach to computing as data winds up getting processed in different stages. For example, big data analytics in the context of Internet of things (IoT) applications is really a distributed application that involves processing at the end point, the gateway and in the network before anything makes its way back to the data center. As such, many organizations still need to figure out what data needs to be processed exactly when, where and how, Flannagan said.

"When it comes to big data, everybody talks about volume, velocity and variety," Flannagan said. "But they may want to start thinking more in terms of fast data and the right data."

Regardless of the approach, one thing that is for certain is that when it comes to big data analytics skills is that many organizations will remain fairly challenged. Roughly half the organizations CompTIA polled admitted that they see skills gaps in areas such as real-time analytics, relational databases and data security.

Throw in the cloud, security, storage systems, increased network bandwidth requirements, as well as the challenges associated with building big data applications and the scarcity of data scientists needed to make sense of it all, and it quickly becomes evident that as far as the IT channel is concerned big data is becoming a proverbial gift that could very well keep on giving all through the rest of this decade and beyond.

Michael Vizard has been covering IT issues in the enterprise for more than 25 years as an editor and columnist for publications such as InfoWorld, eWEEK, Baseline, CRN, ComputerWorld and Digital Review.

This article was originally published on 2015-12-23