GrSecurity: how much do I lose by disabling PaX?

I've been working on a hardened virtual machine host, using a GrSec stable kernel and the latest Virtualbox. Unfortunately PaX does not like Virtualbox at all. Starting a virtual machine results in a PaX log message about a "suspicious general protection fault," and immediate banning of the guilty user account.

I'm trying a rebuilt kernel with the PaX protections disabled. I have to ask though, how much is theoretically lost in doing this? In particular, it looks like most kernel space memory protection is part of PaX (as opposed to userspace stuff like enhanced ASLR). Since most Virtualbox VM escapes would probably be via holes in Virtualbox kernel module, it seems prudent to me to keep as much hardening as possible for the host kernel... Can anyone give me a summary of the most significant features I'd lose by disabling PaX?