The old logic about 64 bit code was based on code size, well actually pointer/data size. Both pointers and long integers are now twice as big (8 bytes vs 4 bytes). All this expanded memory usage uses more memory bandwidth, may make you page to disk more, may make less efficient use of caching, etc.
Even saying that, even in the beginning of the x86 move to 64 bit, there were advantages to going to 64 bit. The 32 bit ia32 architecture was always resource constrained, and had a very small number of registers (the places where things are added and moved around from). When AMD invented its 64 bit mode, they added a lot of registers (since they didn't have to be backwards compatible) so this may have increased speed in some circumstances, or at least offset some of the speed loss from the increased code size.
BTW: Intel didn't come up with the 64 bit extensions; it was betting hard on Itanium, which was a brand new architecture with a radical new design. What helped Intel before (x86 backwards compatibility) hurt Intel this time (Itanium x86 compatibility sucked at the beginning) and now Itanium is a niche product. Intel was able to copy the AMD64 extensions because of previous licensing agreements, rebadged them EM64t, and here we go.
Now, every chip has 64 bit, so if you want to get a 32 bit only chip, you're using old tech. And a lot of new OSes need 64 bit to use all the RAM in systems.
TL;DR version: The warning against 64 bit mode is based on looking at one thing only (code size inflation) and ignores other advances (better instruction set). You won't see much of a difference, feel free to worry about other things