Handling Cache Misses

The figure shows the miss rate versus block size. Each line represents a cache of different size. Increasing the block size usually decreases the miss rate because larger blocks exploit spatial locality to lower miss rates.

Note that the miss rate actually goes up if the block size is too large relative to the cache size. A more serious problem associated with just increasing the block size is that the cost of a miss increases. The steps to be taken on an instruction cache miss are
  1. Cache sends a miss signal to stall the processor.

  2. Send the original PC value (current PC – 4) to the memory.

  3. Instruct main memory to perform a read and wait for the memory to complete its access.

  4. Decide which cache block to allocate/replace.

  5. If the block to be replaced is modified, then move the block into a write buffer, which will be discussed later. Otherwise, the block to be replaced can be simply discarded.

  6. Write the cache entry by

    1. putting the data from memory in the data portion of the entry,
    2. writing the upper bits of the address (from the ALU) into the tag field, and
    3. turning the valid bit on.

  7. Restart the instruction execution at the first step, which will refetch the instruction, this time finding it in the cache.