I'm using iterm2, and I'm finding very nice functionality using vim 7.3 in the terminal with the following options:
set mouse=niv #or set mouse=a
EDIT: set mouse=a also seems cool to use as indicated by the other suggestion.
What this does is automatically switches you into either visual mode from normal while selecting text, or into the mode that says -- (insert) VISUAL -- from insert mode. This also has the handy advantage over not setting these modes at all in that when you've got line numbers or relative line numbers, it'll go ahead and copy those numbers you likely don't want to copy. When it drops you into mouse-enabled visual mode this way, it avoids copying those line numbers, as it works to control the visual mode selection rather than the native terminal's selection (which gets suppressed). Scrolling works just fine throughout, keeping in mind that it selects everything from where you had started the selection to the bottom of the screen. Forget about ctrl+c/ctrl+v in vim - You use the vim yankypasta commands (y, yy, p, etc) to copy to the system clipboard:
Again, iterm2 -- osx's built in terminal is trash. I don't really see your beef with line-by-line scrolling, as I'm happy with how this simply functions as vim should, but maybe it's for similar reasons to why I can't watch other people play video games out of motion sickness or why flourescent light and CRT monitor flicker messes with people's heads to the point of migraine. Try adjusting the refresh rate on your brain, you can feel the knob if you press into your temple hard enough and turn.