How come, for example, a 2.66 GHz dual-core i5 will be faster than a 2.66 GHz Core 2 Duo (also dual-core)?
Is this because of newer instructions that can process information in less clock cycles? Other architecture changes?
It's not because of newer instructions usually. It's just because the processor requires fewer instruction cycles to execute the same instructions. This can be for a large number of reasons:
And so on.
The absolute definitive reference is the Intel 64 and IA-32 Architectures Software Developer Manuals. They detail the changes between architectures, and they're a great resource to understand the x86 architecture.
I would recommend that you download the combined volumes 1 through 3C (first download link on that page). Volume 1 Chapter 2.2 has the information you want.
Some general differences I see listed in that chapter, going from the Core to the Nehalem/Sandy Bridge microarchitectures are:
The complete list can be found in the link provided above (Vol. 1, Ch. 2.2).