"And, in fact, a small cache is usually as good as a big one. The jump in hit rates is usually pretty slender when you move to a larger cache." That's not true at all. It it were true, then secondary caches would make no difference, since of course there is a primary cache present on chip for all these machines. Yet, we know that the presence of a secondary cache is critical for performance.