performance - Why are newer generations of processors faster at the same clock speed?

Why, for example, would a 2.66 GHz dual-core Core i5 be faster than a 2.66 GHz Core 2 Duo, which is also dual-core?

Is this because of newer instructions that can process information in fewer clock cycles? What other architectural changes are involved?

This question comes up often and the answers are usually the same. This post is meant to provide a definitive, canonical answer for this question. Feel free to edit the answers to add additional details.

Answer

It's not because of newer instructions usually. It's just because the processor requires fewer instruction cycles to execute the same instructions. This can be for a large number of reasons:

1. 
   

   Large caches mean less time wasted waiting for memory.

   
   


2. 
   

   More execution units means less time waiting to start operating on an instruction.

   
   


3. 
   

   Better branch prediction means less time wasted speculatively executing instructions that never actually need to be executed.

   
   


4. 
   

   Execution unit improvements mean less time waiting for instructions to complete.

   
   


5. 
   

   Shorter pipelines means pipelines fill up faster.

   
   

And so on.