Memory Hierarchy

In a perfect world, a computer would have a processor and a very large amount of very fast memory, as fast as the processor. But the problem is, high performance parts are ungodly expensive and nobody can afford a computer unless compromises are made. Cue the memory hierarchy, which is different "levels" of memory that have different performance rates, but all serve a specific purpose. A lot of research lately has been done to reduce the amount of levels or make accessing the slower one less painful.

Memory can be generalized into five hierarchies based upon intended use and speed. If what the processor needs isn't in one level, it moves on to the next, to look for what it needs. The first three hierarchies, registers, cache, and main memory, are currently volatile memory. This means once the power is cut from them, they lose their data. The last two are not volatile and are considered "permanent storage".


Typical access time: One clock cycle.

Registers are typically Static RAM in the processor that hold a data word, which on modern processors is typically 64 or 128 bits. The most important register, found in all processors, is the program counter. This tells where the next instruction is at. Most processors have a status word register which is used for decision making (notably MIPS processors don't have one) and the accumulator, which is used to store the result of a math operation. Complex instruction set computers usually have around a dozen or so registers since they're geared more for accessing main memory. Reduced instruction set computers have a lot more registers.


Typical access time: Tens to hundreds of clock cycles.

Cache is also usually found within the processor, but occasionally it may be another chip. Divided into levels, cache holds frequently used chunks of data from main memory. The more times that piece of data is used, its latency when accessing it approaches cache speed. Single core processors rarely have more than two levels of cache. Modern multi-core processors will have three, two levels for each core, one level that's shared.

Main Memory

Typical access time: Hundreds of clock cycles.

Most commonly called RAM. Main memory is relatively fast and holds most of the data and instructions that are needed by currently running programs. This used to be a precious resource up until the mid to late 2000s. Today, memory is so plentiful that modern-day operating systems will use unused portions of main memory to store the core parts of programs that are used a lot (which is also called caching) so that when opening that program, it appears to load a lot faster.

Secondary Memory

Typical access time: Millions of clock cycles.

Secondary memory is where data can be permanently stored, usually a hard disk drive (HDD) or solid state disk (SSD). The unfortunate thing is that secondary memory has a huge gap of latency, and there's a lot of work done to close this. Aside from bandwidth performance (2GB/s vs. 25.6GB/s on DDR3-1600 RAM), latency is still a bit of an issue (~100,000ns versus 10ns).

Removable memory

Typical access time: Tens of millions of clock cycles

Data that's intended to move around resides on removable memory. Examples include floppy disks, CDs and DVDs, and USB thumb disks. The biggest drawback of course is that they're horrendously slow. However, external hard drives can retain near native speeds with eSATA or USB3.0.


A way to comprehend all of this more easily is to pretend you have an open notes physics exam. You also decided to bring a cheat sheet.

  • A register would hold the current formula you're using right now. You'll promptly toss this out of your head when you're done. note 
  • Cache would be holding the formulas you memorized. Occasionally you'll have a brain fart. note 
  • Main memory is the cheat sheet you have. It holds all the pertinent information you could use. But you might forget something minor or you're not quite sure how it worked... note 
  • Secondary memory is your notes. This may have a condensed version of what's in your book, sort of like how installing a program doesn't exactly copy everything to the hard drive.note 
  • Removable memory would be your books. Since you couldn't take your books, you could lack the necessary "program" to solve your problem.