The two companies are arguing, mostly in European papers in order to influence antitrust policy in the European Union, that security design decisions in Windows Vista are unfair to them and actually weaken system security. Their arguments are, for the most part, transparently false and self-serving.

There are two main issues being bandied about, kernel patching and the Windows Security Center.

I'll focus on kernel patching here and on the Security Center in another column, although the big issues are largely the same: Microsoft has made design changes in Windows Vista that make the system more secure, but that limit the technical options for third-party products. Those third parties are unhappy.

Click here to read Microsoft's Ben Fathi's response to questions about these issues.

When I say "Windows Vista" I should be more specific: Kernel patch protection is implemented only in the 64-bit version of Windows Vista, just as it has been in the 64-bit versions of Windows XP and Windows Server 2003.

Resource Library:

"Kernel patching" refers to the hooking, at run-time, of operating system calls. A good explanation of this process may be found in the descriptions of its most infamous use: the Sony DRM rootkit affair of about one year ago. As described by Mark Russinovich, now a Microsoft employee, the Sony rootkit used kernel patching to do its dirty work:

A common way to intercept kernel-mode application APIs is to patch the kernel's system service table, a technique that I pioneered with Bryce for Windows back in 1996 when we wrote the first version of Regmon. Every kernel service that's exported for use by Windows applications has a pointer in a table that's indexed with the internal service number Windows assigns to the API. If a driver replaces an entry in the table with a pointer to its own function then the kernel invokes the driver function any time an application executes the API and the driver can control the behavior of the API.

Old-time DOS programmers like me are reminded of TSR-hooking in order to implement popup programs and similar facilities.

Sony's example demonstrates what is obvious, that kernel patching can be used for nefarious purposes, and especially for rootkits. This is why Microsoft decided, in the 64-bit versions of Windows, to put checks in the operating system to disallow patching of the system service table. Why not the 32-bit versions as well? Basically out of consideration for vendors like McAfee that had come to rely on it for their own purposes.

Next page: McAfee and Symantec's claims.

However, if you listened to McAfee you'd get the impression that all versions of Vista are affected by kernel patch protection. I asked McAfee if the company planned to correct this dishonest letter from its Chairman and CEO (here as a PDF), but got no response.

Symantec is more honest, from what I can see, in its public postings on the matter, such as this blog entry and this paper (PDF).

Symantec implies that Microsoft can bypass PatchGuard with its own products:

If Microsoft wants to make Vista more secure, it should provide equal access to the platform that its own developers have to ensure that security vendors can continue to innovate on the platform, and to ensure that consumers and OEMs can continue to choose the best security solutions for the platform. This has always been the case with prior operating systems.

Microsoft's response to this is to say that its own aftermarket products, such as its OneCare security service, have no more access to kernel patching than anyone else. The Vista developers themselves of course have more access to the operating system kernel, but that's just the way it is. Incidentally, Microsoft has some good blogs on the subject: The Windows Vista Security Blog: An Introduction to Kernel Patch Protection and Jeff Jones: Symantec's Plea : Protect Our Protection Racket.

And just because companies don't have the same access to the kernel doesn't mean they don't have sufficient access to the kernel. With kernel patch protection you can still write a kernel-level driver, monitor process creations and DLL imports, then rewrite the binary code before it's copied to executable memory. You just can't patch the kernel as it's running.

Microsoft has unveiled tools to fight Vista piracy. Click here to read more.

For the more bread-and-butter aspects of malware protection, the right way to go is through a filter driver, a driver type that has existed since time immemorial, at least in Windows terms. Filter drivers are a built-in method in Windows for monitoring and processing I/O, whether it's file I/O or network I/O or whatever.

The answer from McAfee's and Symantec's point of view is basically to keep Windows insecure. They have discussed whitelisting their products in the kernel, but that really stinks of new problems and hacks and I don't blame Microsoft for not considering it. Given that, McAfee thinks we're all better off with a Windows more vulnerable to rootkits and other similar attacks. That way we'll need more of its products.

All this does put more of a burden and responsibility on Microsoft to respond quickly if, or rather when, security problems develop in the kernel. That's the way it has to be, and only time will tell if the company lives up to its obligations. I like its incentives in this regard better than McAfee's.

Security Center Editor Larry Seltzer has worked in and written about the computer industry since 1983. He can be reached at larryseltzer@ziffdavis.com.

Check out eWEEK.com's for the latest security news, reviews and analysis. And for insights on security coverage around the Web, take a look at eWEEK.com Security Center Editor Larry Seltzer's Weblog.