Lets start from the beginning, how encryption works.
AES needs a key of a specific length, for AES256 that 256bits. Because your password or file will not be exactly 256bits long, Truecrypt uses a hashing algorithm (eg RIPEMD-160) to generate the correct length.
For example, a header key for the AES-256 cipher is always 256 bits long even if HMAC-RIPEMD-160 is used
Using only a password has the disadvantage that the pool of characters is somewhat limited to the symbols on your keyboard. Using a keyfile, especially one with a high level of randomness will be more secure, just because it has more possible combinations. The disadvantage, of course, is that someone might get ahold of this file.
If the keyfile is somehow destroyed it will be nearly impossible to recreate it, unless it is a file made up of know content (eg the UN-Charta), still you have to ensure that the content is exactly the same, especially for files with meta data (eg creation date, owner name) this will be a hard task. But a plain text file will just be as limited to its number of symbols as your keyboard is, and thus is a very bad idea to start with.
Forgetting a password is as bad as this, writing it down is as bad as having a keyfile on something like an USB-stick. You should always make a backup of your keyfile in case of data-corruption or if your drive dies.
Basically it comes down what to you prefer and what level of security you want to archieve. A very long password that no one can force out of your brain or a keyfile for which you can ensure no one has access to. Next possibility is to combine password and keyfile, this way you might use a shorter password. But for this solution you still have to ensure that a third person cannot aquire the keyfile (eg by storing it inside a container or drive encrypted by a long password).