multi processor - Why are multiple levels of caches used in modern CPUs?

I recently read about processors (mainly multi-processor) and I came across the concept of multi-level caches. These designs have several levels of cache in this case to increase the performance.

I could not figure out, however, how a additional caches increases performance in the processor? Why not just increase the size of existing caches instead?

Answer

The use of multiple cache levels is partially a mechanism to coordinate multi-core processors and partially a compromise between price and performance.

In a processor with multiple cores, each core has its own L1 cache. This allows the core to read and write from and to the cache without worrying about interfering with other cores. The cores need shared storage, though, so that they can exchange certain information easily. The L2 cache is shared by all cores, so it's used as a sort of communal storage space where information is available for all threads.

The difference between the L2 and L3 caches is the compromise part. Caches are made of static RAM, or SRAM. This is different from the Dynamic RAM (DRAM) that makes up your main memory. Dynamic RAM needs to be "refreshed" periodically, that is, over time DRAM cells lose their value unless they are read and then re-written. Your memory controller does this automatically, but every time the memory controller has to do this (thousands of times a second) it is unable to read/write values for the processor until it is done. This slows down DRAM. SRAM does not have this limitation, SRAM will retain its value indefinitely as long as it has operating power, making it much faster. So, your caches (both L2 and L3) are made of SRAM. The trouble is that SRAM is very expensive; while 4GB of high-speed DRAM is a bit pricey but affordable, 4GB of SRAM is well beyond your price range.

So, whenever a processor manufacturer decides to add more SRAM to the design, it gets noticeably more expensive. SRAM comes in multiple speeds, and as you might expect faster SRAM is more expensive than slower SRAM. So, your processor's manufacturer has tried to optimize it for both speed and cost by using both a higher speed SRAM and a lower speed SRAM. The processor is then designed such that it will put the values it needs most in the faster cache (L2) and the information that it needs less in a slower cache (L3). By carefully managing this memory in the processor's microcode, this creates an even compromise: there is more cache, and some of the cache (hopefully the cache that the processor needs most) is very fast.

So, to sum it up, processors have multi-level caches in order to increase the capacity of the processor cache without also dramatically increasing the price of the processor. This careful mixture allows for processors that are faster and cheaper.