H.264 is a distribution codec, with inter-frame compression. That means that one frame often depends on others. iMovie converts all video to Apple Intermediate Codec, which compresses each frame individually. This means you can remove or reorder any frames you want. As you see, it is also much larger. It will then render the final product back into H.264 (which is also relatively time-consuming).
MPEG-2 (used in DVDs) has the same problem. There are "cuts-only" editors that allow you to remove each GOP (Group of Pictures) -- the set of frames that depend on each other -- but each GOP is about a half-second long, so you edits are not frame-accurate. When you're done, it will then simply copy the file, skipping over the GOPs you don't want, which is faster than re-rending/compressing (but hardly instantaneous).
A quick search did not reveal any cuts-only editors for H.264 that don't involve the "install a bunch of stuff and hope it works" routine on Windows. But if you really can't spare the disk space, you might consider it.