I chose a 1T external drive for time machine and checked the "encrypt backups" option. I thought it will only encrypt the backups, but as it seems it is now encrypting the whole drive which has quite some data on it and is not SSD ;). Is there a way to cancel the encryption process? I turned off Time Machine and chose another drive for it.
Maybe try highlighting the drive in disk utility, go to "File > Turn Off Encryption". If that isn't possible while the encryption is in progress, there probably is no 'nice' way of stopping it. In that case your best bet would be to wait it out, or risk making a mess - Like Thomas said.
Yes, in 10.9 you can roll it back while its busy encrypting
You will then be promted with the OS X password unlock/decrypt dialog. After that it starts decrypting.
Check the status with
Look for "Conversion Progress" and check if "Conversion Direction:" is set to "backward"