As in previous answers (+1 for both) the trick is to use -type f predicate.
Note, that instead of -exec rm '{}' you can also use -delete predicate. But don't do that. With -exec rm '{}' you can (and should) first do -exec echo rm '{}' to verify that this is really what do you want. After that rerun the command without the echo.
Using -delete is faster (no extra fork() and execve() for each file), but this is risky because -delete works also as a condition, so:
# delete *.tmp files
find . -type f -name '*.tmp' -delete
but if you ONLY swap arguments:
# delete ALL files
find . -type f -name '*.tmp' -delete
If you ever need find and rm to work faster for tons of files, check out the find ... | xargs ... rm UNIX idiom.