Managing Storage in a Software-Defined World

By Michael Vizard
software-defined storage

While just about everybody would tend to agree that storage needs to be simpler to manage, very few people seem to agree what's the best way to go about making that happen.

As a result, it seems like every vendor these days is bandying about the term software-defined storage (SDS), but what that means is actually open to interpretation.

For some vendors, it means something as relatively simple as a storage operating system that can be deployed on top of any storage system or, for that matter, magnetic or flash drive. Others, however, say that SDS needs to be able to manage storage at a high enough level of abstraction that it enables the entire system to be programmatically addressed via APIs. In that context, both the OpenStack cloud storage framework and the S3 cloud storage service from Amazon Web Services (AWS) are considered the two leading examples of how APIs are transforming storage management in the cloud era.

One thing that almost everyone in storage circles can agree on is that we've reached a demarcation point. Instead of manually managing volumes of storage, the management of those systems is increasingly going to become automated. Faced with eventually having to manage petabytes of data, the average IT organization simply can't afford to keep hiring additional storage administrators to manage all that data.

In fact, one thing that differentiates so-called Web scale companies such as Facebook and Google from the average IT organization is that they have found a way to allow storage administrators to manage petabytes of data. Recognizing that it's only a matter of time before the current rate of data growth will create a similar set of challenges for the average IT organization, vendors have been racing to develop systems that automate the management of large amounts of data at scale.

Many emerging storage vendors see this shift as an opportunity to usurp leading storage vendors that include EMC, NetApp, Hitachi Data Systems, IBM, Dell and Hewlett-Packard.

Leading examples of vendors trying to make an SDS name for themselves include Coraid, Caringo, Cleversafe, Exablox and DataCore. In the case of Coraid, SDS comes in the form of a unified storage system that attaches directly to raw Ethernet.

"Our definition of SDS would be that that the system needs to automatically create a storage profile for the application," said Robert Przykucki, senior director of product management for Coraid. "The system then uses the profile to put the application on the right container."

DataCore, meanwhile, argues that storage management in the context of SDS needs to be heterogeneous almost by definition.

"You can't manage a bunch of isolated storage islands," said DataCore CEO George Teixeira. “Storage needs to be a shared resource.”

Elsewhere, Exablox is leveraging SDS to create storage appliances that scale out in a way that allow solution providers and their customers to choose whatever drives they want.

"Rather than selling legacy RAID technology, we've developed an object-based file system," said Exablox CEO Doug Brockett. "It allows our partners to let customers bring their own drives at price points that are at 5 to 10 times lower than NetApp, EMC, HP or Dell."

Also employing SDS to manage data at scale is Cleversafe, which makes use of software to slice data up in a way that improves scalability and makes that data more secure.

"Innovation now takes place in the software using commodity hardware," said Cleversafe CEO John Morris. "You have to be able to have an abstraction of the policy layer."

Caringo, meanwhile, has developed an approach to scaling an SDS platform that reduces costs by eliminating the need for a controller.

"We rely on standard x86 systems running Linux," said Adrian Herrera, senior director of marketing for Caringo. "Multiple nodes then make up what we call a Swarm."

Meanwhile, incumbent storage vendors such as EMC are keenly aware of the significance of SDS, but view the rise of SDS as more evolutionary than revolutionary.

"The intelligence is clearly going to be in the software," said Sal DeSimone, CTO of the Advanced Storage Division of EMC. "It will happen in stages."

In the case of EMC, that first stage is ViPR, a next-generation controller that creates a data plane that isolates the management of data from the underlying physical hardware.

NetApp, meanwhile, argues that the shift to SDS has already been under way for years. NetApp CTO Jay Kidd said what is really changing is the degree to which SDS can be applied to manage storage at scale. As such, Kidd contends that data management in the age of the cloud will come down to the resiliency of the software being used to manage it.

"Almost anybody these days can build a software-defined system," said Kidd. "The challenge is to be able to build resiliency into the software in a way that manages data at scale."

To that particular end, Kidd said that NetApp has been invested heavily in developing Clustered Data OnTap, which is designed to manage data at scale across private and public cloud computing environments.

At the same time, vendors such as HP said they see the shift to SDS as an opportunity to gain market share at the expense of established rivals. HP created a data mobility capability, called 3Par Online Import, that reduces storage switching costs by making it simpler to migrate data from one storage system to another.

"Data mobility is about making the upgrade process non-disruptive," said Vish Mulchand, director of solutions marketing for HP Storage. "It's now easier to upgrade from the existing versions of EMC's VNX system to HP 3Par StoreServ storage than it is to upgrade to the newest version of VNX."

For solution providers in the channel, technology transformations on this scale eventually create new opportunities. The challenge now is figuring out when those opportunities are going to be ripe.

The good news is that not only are customers struggling with more data than ever, they are keenly interested in having a more agile IT capability. In fact, the quest for that agility is driving not only much of the interest in SDS, but also all other things software-defined across the enterprise.

Of course, the degree to which SDS or any other form of software-defined anything will hold up as a separate distinct category of anything remains to be seen.

The one thing that is for certain is that as SDS continues to evolve, the opportunities for the channel surrounding storage have never been greater.

Michael Vizard has been covering IT issues in the enterprise for 25 years as an editor and columnist for publications such as InfoWorld, eWEEK, Baseline, CRN, ComputerWorld and Digital Review.

This article was originally published on 2014-05-30