I am encrypting a volume with TrueCrypt. I have decided to use
AES as the encryption algorithm, seeing as it is the government standard.
But which hash algorithm is better (more secure)?
I am using Mac.
Both SHA-512 and Whirlpool are 512 bit hashes, while RIPEMD-160 is 160 bits. TrueCrypt uses these hashes with PBKDF2 to derive keys. When creating file volumes, Truecrypt uses 1,000 rounds for both SHA-512 and Whirlpool, but 2,000 rounds for RIPEMD-160. I have some software called TCHead that decrypts TrueCrypt headers. It's very simple and straight-forward and demonstrates how and where these hashes are used.
To answer your question, in my opinion, either SHA-512 or Whirlpool would be fine. I would use either and not worry about that too much. Hope this helps.
You may find the Cryptanalysis section of the Wikipedia page comparing hash algorithms useful to make your decision.