I have a few cabinets that I keep old computer harware (monitors, power supplies, etc). I checked one of the cabinets and smelled strongly of electrical components. Sure enough, one of the power supplies had gone bad. My intuition tells me that it was probably already on it's way out, but I really have no way of verifying that in this case. So, that begs the question, can computer components go bad when not in use?
Yes, electrolytic capacitors can contain corrosive materials for their electrolyte. Those materials can deteriorate the ability of the capacitor with time if not in use. This can sometimes be fixed by reforming the capacitor. See the Wikipedia page in the first link for more ways the capacitor can fail.
In addition to the capacitors mentioned by Scott, various metals oxydize - solder connections included. Solder isn't even a stable state compound. It changes state internally for a very long time.
Some metals even form growths that look like mold which can cause shorts. http://en.wikipedia.org/wiki/Whisker_%28metallurgy%29
Plastics and other materials can develop cross linkages or other problems that cause them to become brittle. I have a couple of loudspeakers with an outer rubbery rim on the cones that turned to goo after many years.
Dust, humidity, vibration, thermal expansion and contraction with ambient temperature changes, and any corrosive chemicals in the air (smog, cleaning solvents, etc.) can also damage equipment over time. Residues of some of the chemicals used to manufacture circuits (acid washes, fluxes, etc.) may also remain on circuit boards and cause problems over time.
Even cosmic rays can eventually break the tiny components of integrated circuits. It just occurs at very low probabilities (which accumulate over time) unless the circuits are in satellites or nuclear reactors.
Just to show how weird things really are, glass is a supercooled liquid. If you look at really old window panes, you can sometimes see that the glass has slightly flowed downward toward the bottom.
It comes under the general heading of "This too shall pass" (or entropy, if you want to get fancy).