I've run into some very difficult-to-troubleshoot CSS problems that were ultimately traced to a missing semicolon, extra bracket, or other syntax problem in a CSS file. If your CSS file can be validated, it's easy to find these issues; if your CSS file contains a lot of garbage that isn't expected to validate, it's harder.

Put the IE-specific stuff in a separate file. (I know you said you don't want to do that, but it really does have its benefits.)