The biggest single issue is going to be the availability of plugins. Since they're code that need loaded into the same process as the host application, they have to have the same architecture as the host. If the plugins you need aren't available in 64-bit, you'll need the 32-bit Photoshop in order to use them.
Beyond that, I'm not sure that there's much of a point to the 32-bit version.
Also, as @MikeFitzpatrick pointed out, TWAIN divers are another major issue if the scanner vendor doesn't have a 64-bit driver available. As a practical matter, this really only applies to older hardware -- 64-bit drivers are a requirement for Vista's WHQL certification. That said, I know that high-end scanners can be kept in service for a very long time, partially (or even primarily) because they're dreadfully expensive.