CPU usage is a measure of how much resource the CPU has available, but there are many different types of instruction that can be processed and they all have different processing and memory requirements.
A task that is memory intensive may cause the CPU to stall while it fetches data from memory and so reduce the effective instruction throughput while still having the CPU "in use".
Also there are many different parts of the CPU that may be saturated differently.
From Wikichips Sandy Bridge uArch:
You can see we have an initial instruction decoder frontend, which for complex and diverse instruction streams might struggle to keep the rest of the pipeline full.
If you only have integer additions then you will be able to use 3 of the core execution units, as the CPU has 3 INT ALU units. If you only have floating point multiplications then you may only use the single FPU MUL (multiply) unit.
The CPU also operates as a pipeline, and while one unit is in use in an execution unit, you may be able to schedule an operation in the next cycle. This means a diverse instruction stream can make better use of resource as a unit not in use can be scheduled in the same EU, but with a different instruction type.
Different instructions will also have different execution times and have larger or smaller set of associated circuitry to execute. A simple addition may take one or two clock cycles, while a floating point instruction might take longer and have a larger amount of circuitry involved. Taking longer might mean it uses more power, as might the larger area of circuitry. Alternatively the instruction taking longer might mean that the front end scheduling circuitry pauses and briefly uses less power while it waits for available execution units, while the smaller faster instructions use more overall circuitry if you include other parts of the CPU.
As a result to make full use of the CPU you need a diverse instruction stream, and what may exercise one CPU might not fully exercise another due to different arrangement and number of execution units and their capabilities.
Execution units can go "low power" with modern power gating method and as a result not contribute to the heat output of the device, or contribute a lot less.
Caches also contribute to power consumption. Using the cache will mean that instructions and data can be fetched and, as a result, executed faster than a routine with a data set in memory that is too large for a cache.
As a result different programs or instruction streams may cause different peak power usage and so different temperatures.
Architectural differences across processor generations, and even in the same generation where cache sizes, processor options and different instruction availability may have an effect.