I understand that VNC is not encrypted, which is fine if I tunnel with SSH. But what about setting a password? Is it really helpful? Or is it really a joke?
Well, the manual says
vncpasswd['s] ... default behavior is to prompt for a VNC password and then store an obfuscated version of this password to a file. ... Note that the stored password is not encrypted securely - anyone who has access to this file can trivially find out the plain-text password, so vncpasswd always sets appropriate permissions (read and write only by the owner.) However, when accessing a VNC desktop, a challenge-response mechanism is used over the wire making it hard for anyone to crack the password simply by snooping on the network.
This is enough to keep out the idly curious, or the other user on your machine who has forgotten their display number and tried to connect to your display by mistake, but it sounds like not enough to keep out a determined attacker.
Note especially that the password file is "obfuscated," rather than encrypted; anyone who can read your read-only files can trivially determine the password.