Four years ago, the intrusion prevention system market consisted of a few next-generation intrusion detection system appliances with elementary blocking capabilities. Most vendors and analysts at the time said IPSes would remain a minor offshoot of the IDS segment, mainly because administrators were loath to run appliances that could block network traffic actively.

Those predictions, however, are proving false. The IPS sector has grown steadily and drawn the interest—and the deep pockets—of heavyweights such as Cisco Systems Inc. and 3Com Corp. The IPS market now encompasses a variety of in-line host and network solutions as well as large-scale network monitoring systems capable of making real-time changes in routers, switches and other devices to fend off attacks.

Click here to read eWEEK Labs' review of Top Layer Networks' IPS.

Some vendors, such as Sana Security Inc., have even moved the IPS concept to the desktop.

All this activity comes even as many experts say IPSes are still in their infancy, with much room left to mature. A key factor in that maturation will be the convergence of IPS with other security technologies, including IDS and firewalls and perhaps even anti-virus software, experts say.

"The threats are getting faster, and we're seeing more polymorphic code. The new appliances you'll see down the road will be able to look directly at the behavior of malware and not the signature," said Steven Hofmyer, founder and chief scientist at Sana, based in San Mateo, Calif. "You will get more systems that use behavioral heuristics. If you can change the game so that you only need signatures about 10 percent of the time, that's a big change."

Today, most IPSes—like their IDS forebears—rely on signatures to identify attack traffic. A few use a system that models normal traffic on a protected host or network to help identify anomalies. Both approaches have their strengths and weaknesses, but Hofmyer said he believes that in the near future, most enterprise IPS solutions will incorporate a combination of the two.

"I think you'll see IDS incorporated into IPS and anomaly detection; signatures and the option of prevention or just detection mode will all be part of it," Hofmyer said. "Still, not everyone will want to run it in prevention mode 100 percent of the time."

Other vendors also see convergence on the horizon and say enterprise customers now depend on IPS solutions to such an extent that they are considered part of the network infrastructure, much like switches or firewalls. That's a far cry from the days when administrators would keep the IPS in listen-only mode for months for fear it might block legitimate traffic.

"What's really important to customers now is that the products have the same level of maturity as other network security gear," said John Parker, director of product management at McAfee Inc., based in Santa Clara, Calif. "The IPS can't go down, but addressing redundancy and failover is not trivial. We're looking at redundant management now because what if there's a failure, and the next big outbreak occurs at that point?"

There are other challenges ahead for IPS as well. For example, how will the systems handle emerging technologies such as VOIP (voice over IP), which is becoming a mission-critical enterprise application?

To help IT managers develop a request for proposal for prospective IPS vendors, eWEEK Labs has put together a series of questions that can serve as a starting point. Click here for the sample RFP.

"There's a challenge there in terms of recognizing and decoding packets for VOIP," said Jason Anderson, product manager at Lancope Inc., based in Atlanta. "Not everybody can do it. IPS is not going to solve all of your problems. There's an important and necessary position for IPS in the enterprise, but it's still only a piece. It's great for eliminating a certain amount of noise, but you still have to cover the traffic that gets through.

"IPS is more broadly accepted for prevention now, but it's still typically turned on for a small subset of traffic where it can be highly accurate," Anderson said.

Check out eWEEK.com's for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEK.com Security Center Editor Larry Seltzer's Weblog.