Well, if you want the industry standard "professional" video suite, you'll need to switch to Avid. I've never used it, so I can't comment on how well it works, but it is still the de facto industry standard.
Sony Vegas is a bit of a mixed metaphor. It does editing and compositing, but it just doesn't work quite the way you expect it to work. The work-flow is just qa bit off from what you'll be used to coming from another editor. You can still create video with it, though. But as for its' compositing abilities, it is nowhere close to AfterEffects.
As for Premiere... I produced a cable television program for 4 years with Premiere 5.1, and while it had its' issues, it worked.
My overall experience with video editing software/systems is that you need to put a lot more time/effort/consideration into creating a stable software+hardware environment. Which means paying a lot more attention to hardware compatibility issues, giving yourself a lot of hardware headroom, choosing the best video resolutions & formats, tuning your operating system for best performance, using a solid performing video converter, etc. The more "professional" the desired outcome, the more important it is to consider these factors.
As for dealing with your video formats, Premiere may be choking on having mixed formats on a time line. If you are creating long-form videos, I would find a format that Premiere prefers that meets your requirements, and convert to that format during the capture process. Run the video through a converter, or use a decent video camera that does conversion input/output. If you are doing short-form videos, such as television commercials, do the same for captured video, and use AfterEffects to transcode shorter clips to your needed format. And while AE is a compositor, for a short enough final result, theres no reason that you couldn't edit with it. The ability to layer video and set in/out points on a time line pretty much sums up the fundamental ability of both Premiere & AE.
