I recently remove the CPU and fan from my motherboard overnight. When I went to reinstall them, I noticed the thermal compound looked kind of dried up. Do I absolutely need to go out and buy some more thermal compound for my CPU to work OK? Will it overheat if the thermal compound is like this?
Yes, if you remove the heatsink from the CPU, you need to clean off the CPU (you can buy dedicated cleaners for this, or use some isopropyl alcohol) and apply more thermal paste.
In general, thermal paste should be reapplied every time the heatsink/fan assembly is removed from the CPU. The old thermal paste may dry up and will not conduct heat as effectively as before. The CPU may overheat and shut down as a result; even if it does not, temperatures will be higher than they should be.
This applies regardless of the processor type. Remove any dried-up paste before you add new thermal paste, as described by user55325's answer.
You can moisten the thermal paste with a tiny amount of isopropyl alcohol (or, if all else fails, a microscopic amount of water) and remount your CPU with the existing paste. You won't get full performance, as the CPU will throttle and won't turbo boost if it gets too hot, but it's not supposed to be possible to damage anything on a CPU like yours with an integral heat spreader.
The CPU is specifically intended to run at maximum temperature when it's under full load. Under normal conditions, it will boost its clock rate until it reaches maximum temperature.