Your mainboard has some voltage on it, if your PC is shutdown but the PSU still switched on.
This is to support features like "wake on LAN" and other stuff.
I have often exchanged parts in my computer with the PSU on and nothing happened, but it is always a better idea to switch it completely off, because you can't know which parts are under current while you are removing them. (and this still does not mean that they have to break, but they can. its like defusing a bomb, green cable first)
You can look up what the beep codes tell you, they are normaly a sequence of some long and short beeps that help indicate what is wrong. (bad memory, bad vga etc)
Here is one site, be aware that different BIOS' have different codes,but there are not that many BIOS Vendors and the codes are mostly very similar.