History Analysis / CentralProcessingUnit

Back to UsefulNotes/CentralProcessingUnit

Superscalar execution is when the execution units of a processor are duplicated. This allows it to run more instructions at once for the same program. For instance, if there are a number of operations that don't depend on the results of each other, then they can all execute at once. Strictly speaking though, this isn't an implementation of a UsefulNotes/MultiCoreProcessor though, as the instructions must come from the same unit of execution of a program (typically a thread).

* Certain types of processors, such as the MediaNotes/GraphicsProcessingUnit, are designed as [[UsefulNotes/FlynnsTaxonomy single-instruction, multiple data (SIMD) processors]], meaning they run the same instruction for a large data set. Branching, while possible, means that only some of the data is processed which reduces the efficiency of the processor.

A semi-related note, if one looks into the specs of the CPU, they'll likely find that the CPU has two types of L1 cache: one for instructions and one for data. This might seem odd since everything above this in the [[UsefulNotes/MemoryHierarchy Memory Hierarchy]] allows for both instructions and data to exist in it. This separation mostly has to do with two schools of thought with regards to how [=CPUs=] access memory. The first is the von Neumann architecture, which is where data and instructions can live in the same memory pool. The second is the Harvard architecture, which separates data and instructions in their own memory pools. von Neumann architecture's main benefit is cost (only need one pool of memory and one bus to access it), while Harvard architecture's main benefit is performance (CPU can fetch both instructions and data independently and doesn't have to figure out which one is which). Modern [=CPUs=] use the so-called modified Harvard architecture, where the execution core itself is Harvard, but everything else is von Neumann. In addition, modern memory controllers can also emulate the Harvard architecture with von Neumann implementations by marking sections of memory space as "no execute," making it a data-only space.

* Certain types of processors, such as the UsefulNotes/GraphicsProcessingUnit, are designed as [[UsefulNotes/FlynnsTaxonomy single-instruction, multiple data (SIMD) processors]], meaning they run the same instruction for a large data set. Branching, while possible, means that only some of the data is processed which reduces the efficiency of the processor.

There was just one problem. While computation speeds were increasing quite nicely, memory speeds and capacity were not. As [[UsefulNotes/ProgrammingLanguage higher level programming languages]] were developing and taking off, newer processors were built to support these directly to make the execution code as compact as possible. This happened until about the mid [=1980s=] when computer scientists began to figure out that performing simpler tasks in sequence could be done much quicker, often one instruction per clock cycle. Processors were built upon this new design paradigm and given the name Reduced Instruction Set Computing (RISC). Reduced in this case means the time it takes to complete an instruction is reduced. Retroactively, processors made before then were given the name Complex Instruction Set Computing. RISC proved to be such an efficient method of execution that many manufacturers by the late [=1990s=] had or were making RISC processors. x86 is the only, if not one of the only, CISC [=ISAs=] still in widespread use. However, modern x86 processors are designed such that the instructions are decoded into micro-ops and performed in a RISC-like manner.

In the early days of [[UsefulNotes/ProgrammingLanguage high level programming]], memory was expensive, small in capacity, and the speed gap between it and the CPU was growing. The idea of CISC was to cram several operations into a single instruction (such as "Compare and Exchange 8 bytes") and to allow multiple ways to access the data (encoded as part of the instruction, in a register, or in memory). As the name implies though, there are complexities implementing CISC [=ISAs=] because operations can have variable length instructions and their behavior depends on the parameters passed into it.

Examples: Intel x86, AMD x64

In the early days of [[UsefulNotes/ProgrammingLanguage high level programming]], memory was expensive, small in capacity, and the speed gap between it and the CPU was growing. The idea of CISC was to cram several operations into a single instruction (such as "Compare and Exchange 8 bytes") and to allow multiple ways to access the data (encoded as part of the instruction, in a register, or in memory). As the name implies though, there are complexities implementing CISC [=ISAs=] because instructions are variable length and their behavior depends on the parameters passed into it.

* Supporting integers and floating point only and not special types like Binary Coded Decimal or Strings
* Having a lot of general-purpose registers rather than a smaller set of register with more special-purpose registers. In the beginning, x86 only had 4 out of 14 registers meant for general purpose operations, compared to ARM which has 8-12 registers for general purpose operations. Though most special-purpose registers can be read from/written to by software without restriction, so they gradually became special-purpose in name only (though it helps compilers out when it comes to knowing which registers to use for a certain purpose)

By the 2000s, RISC had largely taken over. While x86 remains in widespread use, most implementations decode and translate it into something more RISC-like.

Examples: Intel IA-64, Transmeta Crusoe, MCST Elbrus

An evolution of VLIW, the idea of NISC is to instead of compiling software into instructions and let the processor figure out how issue those instructions on its resources, the compiler can figure out where the data being fed into the processor goes and the instructions are only telling the processor how this data flow should work. That is, if you think of a processor's execution units (add, shifting, etc) as modules, you only need to say "the input of this module reads from here and the output of this module goes to there." While technically needing instructions to direct the data flow, these instructions direct data flow, whereas traditional CPU instructions direct control flow.

Pipelining in a processor is a way of emulating an assembly line in order to increase throughput. To copy from Website/TheOtherWiki:

Pipelining does have its problems though. What kills it is branching such as due to if-statements and loops. If there has to be any branching, then everything that preceded the branch has to be thrown away. Taking the car assembly line example, what if suddenly the insane CEO decides trucks are to be made and cease production on all cars?

!!! Out of Order execution
A type of instruction reordering where some instructions queued for execution in the future can cut in line if they're not dependent on results from an earlier instruction. This is to prevent cases where an earlier instruction can stall the processor if it's waiting for something but later instructions can run right away. The big issue it has is it requires complex hardware to ensure that the output retains the logical ordering, often eating into die space and power consumption. Historically it was kept out of processors meant for small electronics, but has crept its way back in because performance boost it offers now outweighs its drawbacks.

Superscalar execution is when the execution units of a processor are duplicated. This allows it to run more instructions in a program. For instance, if there are a few math operations that don't depend on each other and there's enough execution units the do them all, they can all run at once. Strictly speaking though, this isn't an implementation of a UsefulNotes/MultiCoreProcessor though, as the CPU core still contains a single program's state.

However, some processor manufacturers found a way to simulate a multicore processor via a technique called simultaneous multithreading (SMT). By allowing multiple program states to be loaded on a CPU core, if there's enough free resources after scheduling work for one state, another state can run on whatever's left. You can think of this like two children sharing a box of crayons or technicians sharing a toolbox rather than everyone having their own.

!!! Microcode, Micro-operations, and Micro-instructions

In simpler, traditional [=CPUs=], the control unit, which handles how instructions are executed and how data is directed, used to act directly on the instructions from the ISA. However, as [=ISAs=] get more complicated, along with innovative ways to execute instructions get created, having the control unit be directly controlled by the ISA was starting to prove to be a limiting factor. In addition, since the instruction decoder is often hard-wired, any change in how the ISA works or if there's a bug in how the instructions are decoded means an expensive redesign of that part. This is especially true with bugs, as this means older processors will still have a problem.

This is where the micro-code/operation/instruction comes in. Rather than handle the ISA directly, the control unit and execution unit work on its own unique sort of ISA typically called micro-instructions. These map to micro-operations that the execution unit handles. The ISA is translated via microcode, which in its simplest implementation, is a ROM containing a mapping of which ISA instructions map to which micro-instructions. The main benefit of this is that if there's an issue with how the ISA is translated via microcode, it can be patched using an update. Though in most modern [=CPUs=] that employ this, it's stored in SRAM inside the CPU, so any microcode updates will have to be applied again on boot, mostly as a means to make sure the CPU isn't made worse by a bad microcode update or if the user wants to decide whether or not the benefit is worth it (as was the case with Spectre/Meltdown updates).

An evolution of VLIW, the idea of NISC is to instead of compiling software into instructions and let the processor figure out how issue those instructions on its resources, the compiler can figure out where the data being fed into the processor goes and the instructions are only telling the processor how this data flow should work. That is, if you think of a processor's execution units (add, shifting, etc) as modules, you only need to say "the input of this module reads from here and the output of this module goes to there." While technically needing instructions to direct the data flow, this is not the same as the processor needing to figure out an instruction is an ADD instruction.

!!! Hybrid core design
An approach that started with mobile devices, hybrid core designs pair fewer, larger high performance CPU cores with more, smaller high efficiency cores. The idea is that the high performance cores (typically called P-cores) can handle tasks that where faster performance is preferable or where low response times are needed. The high efficiency cores (typically called E-cores) can handle background or time-insensitive tasks or help boost the performance of the P-cores. Some hybrid designs may include 3 or more performance layers, but usually only 2 are used.

As processors got more complex, these hardware bugs become more costly to fix because it requires another revision of the hardware. Also by the time the bug is discovered and characterized, it's likely thousands and thousands of units were sold and so you're left with people who have the issue. Often the company must have a free-replacement program in order to not be sued. Software can mitigate the problem by not running the offending instructions or working around them, but as long as the hardware bug exists, there's always the potential of the computer crashing or worse.

Then CPU manufacturers started designing [=ISAs=] to translate instructions into microcodes to run micro-ops. These [=CPUs=] were also designed so the microcode can be updated. Since most of the core operations are handled by microcodes, bugs or problems with the microcode can be fixed with an update. However, microcodes only apply to the execution side of the CPU, not the instruction decoding and scheduling side, so there's still the potential of hardware bugs if they exist there.

* Having a lot of general-purpose registers rather than a smaller set of register with more special-purpose registers. In the beginning, x86 only had 4 out of 14 registers meant for general purpose operations, compared to ARM which has 8-12 registers for general purpose operations. Though most special-purpose registers can be read from/written to by software without restriction, so they evolved to special-purpose in name (though it helps compilers out when it comes to knowing which registers to use for a certain purpose)

* Making uniform instruction sizes. While using fixed instruction sizes for the entire ISA is often the goal, some [=ISAs=] may allow for variable length instructions, but there are mechanisms in place to limit the variability. For instance, ARM has 16, 32, and 64-bit instructions, but it can only execute them in a specific CPU mode. MIPS introduced 16, 32, and 48-bit instruction sizes, the same operation (such as a 16-bit ADD instruction vs a 48-bit one) is represented by different opcodes. This is unlike x86, where instruction sizes can be up to 15 bytes and there must be some decoding beforehand to determine this.

* Making most registers general purpose rather than have special-purpose registers. In the beginning, x86 only had 4 out of 14 registers meant for general purpose operations, compared to ARM which has 8-12 registers for general purpose operations.

* Making uniform instruction sizes. Note that while ARM in its entirely technically doesn't have this, as it has 16, 32, and 64-bit instructions, they can only be executed in appropriate CPU modes. This is unlike x86, where its instructions can be up to 15 bytes long and be executed in any mode as long as it supports that instruction.

Pipelining in a processor is a way of emulating an assembly line in order to increase throughput. To copy from Wiki/TheOtherWiki:

Developed in the 70s, the principle of RISC is to reduce the time it takes to complete each instruction. It was found even on a CISC machine, that doing simpler instructions were faster than doing one complex instruction. Soon after processors were designed with the aim of reducing the amount of time it takes to complete an instruction, with goal being one instruction per clock. To do this, RISC architectures typically simplify the instruction decoding process by making uniform instruction sizes, reducing memory access types that an instruction can do, supporting fewer data types (e.g., integers and floating point only, no special types like Binary Coded Decimal or Strings), and most registers can be used for anything, rather than having registers for specific purposes. Ultimately to achieve this simplicity, RISC operations that modify data can only perform said operations within its registers, using load/store instructions to move data around. Appropriately, this is called a [[ExactlyWhatItSaysOnTheTin load/store architecture]]

By the 2000s, RISC had largely taken over. While the x86 remains in widespread use, most implementations of it by then used an instruction translator to convert it to RISC type instructions before being processed.

An evolution of VLIW, the idea of NISC is to instead of compiling software into instructions and let the processor figure out how issue those instructions on its resources, the compiler can figure out where the data being fed into the processor goes and the instructions are only telling the processor how this data flow should work. That is, if you think of a processor's execution units (add, shifting, etc) as modules, you only need to say "the input of this module reads from here and the output of this module goes to there."