Computer architecture: how processors and memory work

Explore the interface between software and hardware—from instruction sets and CPU datapaths to pipelines, caches, virtual memory, and measured performance.

What computer architecture describes

Computer architecture describes the organization and behavior of a computing system, especially the interface visible to software and the hardware that implements it. The architecture shapes what instructions programs can use, how data is represented, and how the processor, memory, and devices cooperate.

The instruction set architecture (ISA) is the software-visible contract: instructions, registers, data types, and memory behavior. A microarchitecture is a particular hardware design that implements that contract. Different processors can implement the same ISA with different pipelines, cache sizes, and performance characteristics.

• ISA: what machine instructions mean to software.
• Microarchitecture: how a processor carries out those instructions internally.
• System architecture: how processors, memory, storage, and devices are connected.

This separation makes software portability possible at the instruction level: a program compiled for a compatible ISA can run on different microarchitectures, although performance may vary substantially.

Instructions and machine code

A processor executes machine instructions encoded as bits. An instruction might add register values, load data from memory, store a result, or branch to another instruction. Assembly language gives human-readable names to these operations and registers.

LOAD  r1, [address]
ADD   r3, r1, r2
STORE [result], r3

This pseudocode loads a value into register r1, adds it to the value in r2, and stores the result. Actual ISA syntax and instructions differ. High-level statements may compile to several instructions, and a compiler may reorder or combine them while preserving the program's defined behavior.

Registers and data representation

Registers are small storage locations inside a CPU core. They hold operands, addresses, and intermediate results. Architectures define register widths and uses. Wider registers can represent larger integers or process more bits per instruction, but do not automatically make every program faster.

Memory stores bytes at addresses. Endianness describes the order in which the bytes of a multi-byte value are laid out. Data alignment can affect performance and, on some systems, whether an access is supported. Compilers and language runtimes usually manage these details, but systems programmers may need to understand them at interfaces.

Instruction styles

Some instruction sets use a relatively small set of regular operations; others include more complex instructions. RISC and CISC are broad historical categories, but modern processors blur the distinction. An implementation may translate complex instructions into simpler internal operations called micro-operations.

The CPU datapath

A simplified CPU contains a control unit, registers, an arithmetic logic unit, and paths that move values between them. The control unit fetches and decodes an instruction, selects operands, and directs the required operation. A program counter tracks the address of the next instruction, subject to branches and exceptions.

1. Fetch: read the instruction at the program counter.
2. Decode: determine the operation and required operands.
3. Read operands: obtain values from registers or memory.
4. Execute: perform arithmetic, logic, address calculation, or a branch decision.
5. Write back: store the result in a register or memory and update control state.

Modern processors execute many instructions in overlapping stages, and may begin work speculatively before knowing whether a branch is taken. These techniques improve throughput but require hardware to preserve the program's architectural behavior and recover from mispredictions or exceptions.

Pipelining and hazards

A pipeline splits instruction execution into stages so different instructions can occupy different stages at the same time. Pipelining can increase instruction throughput; it does not necessarily reduce the time required for one instruction to travel through all stages.

Pipeline hazards prevent an instruction from proceeding as scheduled:

• Data hazard: an instruction needs a result that an earlier instruction has not produced.
• Control hazard: the next instruction depends on a branch decision.
• Structural hazard: two operations need the same hardware resource at once.

Processors use forwarding, stalls, branch prediction, and other techniques to manage hazards. A stall inserts a delay; a misprediction can discard speculative work. The exact cost depends on the microarchitecture and code behavior.

Compiler optimizations can sometimes rearrange independent operations to keep a pipeline busy. Programmers should first choose clear algorithms and data layouts; low-level tuning should be guided by measurements on target hardware.

The memory hierarchy: registers, caches, and RAM

Processor cores can execute operations faster than main memory can deliver arbitrary data. Computers use a hierarchy: registers are smallest and fastest; caches hold recently or nearby data; RAM is larger but slower; persistent storage is much larger and retains data without power.

Caches rely on locality. Temporal locality means recently used data is likely to be used again; spatial locality means nearby addresses are likely to be accessed. Sequentially scanning an array often benefits from both, while following scattered pointers may cause more cache misses.

Level	Typical role	Trade-off
Registers	Current operands and results	Very fast, very limited capacity
L1/L2/L3 cache	Keep useful instructions and data close to cores	Small relative to RAM; cache misses cost time
RAM	Active programs and data	Large and volatile; slower than caches
SSD or disk	Persistent files and program storage	Much slower than RAM; nonvolatile

A cache hit serves data from a nearby cache; a miss fetches it from a lower level. Cache coherence mechanisms coordinate copies when multiple cores access shared memory. False sharing can cause extra coherence traffic when independent variables share a cache line.

Memory bandwidth and latency

Latency is the time to complete an individual access; bandwidth is the rate of data transfer. A workload can be limited by either. Repeated dependent pointer accesses may be latency-bound, while streaming through large arrays may be bandwidth-bound.

Virtual memory and protection

Programs normally use virtual addresses. Hardware and the operating system translate them to physical memory through page tables and a translation lookaside buffer (TLB), which caches recent translations. Virtual memory provides each process with an address-space abstraction and helps isolate processes from each other.

Memory is divided into pages. Accessing a page without a current mapping may trigger a page fault. The OS checks the access and may load or create the page, or report an invalid access. Frequent faults that require storage can severely reduce performance.

Page tables also encode permissions such as readable, writable, or executable. These protections help prevent one process from accidentally or maliciously overwriting another's memory. They are part of a defense system, not a guarantee against every exploit or hardware fault.

Multiple cores, accelerators, and consistency

A multicore processor contains multiple CPU cores that can execute instructions simultaneously. Cores may share caches or memory controllers, so adding threads can increase contention. Software must coordinate access to shared data, and the language memory model defines which ordering guarantees programs can rely on.

Graphics processors and other accelerators provide specialized hardware for particular patterns of parallel work. GPUs can perform many similar operations efficiently, but transferring data to the device and coordinating results has a cost. A specialized processor helps only when the workload matches its capabilities.

Architecture also affects power consumption and heat. Mobile devices may reduce clock speed under sustained load; servers balance throughput, energy, and cooling. The fastest peak benchmark may not represent sustained performance in a constrained environment.

Measure performance and reason about bottlenecks

Execution time depends on instruction count, cycles per instruction, and clock rate, but these factors interact. A useful simplified relation is:

CPU time ≈ instruction count × average cycles per instruction × seconds per cycle

Reducing instruction count may increase memory stalls; raising clock rate may increase power; an optimization for one processor may hurt another. Benchmark with representative input and measure the complete operation, including setup and data movement.

1. Establish a correct baseline and record the workload.
2. Profile to find whether the bottleneck is computation, memory, I/O, or synchronization.
3. Change one factor and repeat the same measurements.
4. Check correctness and memory use, not only speed.
5. Report processor, compiler, settings, and input so results can be interpreted.

Microbenchmarks can measure tiny operations, but they are sensitive to warm-up, compiler optimization, and measurement overhead. Do not infer whole-application performance from one isolated instruction or synthetic loop.

A compact glossary

ISA

The instruction-set contract visible to software.

Microarchitecture

A hardware implementation of an instruction set.

Pipeline

An execution design that overlaps stages of multiple instructions.

Cache line

A block of memory transferred and managed together by a cache.

TLB

A cache of recent virtual-to-physical address translations.

Memory bandwidth

The rate at which data can be transferred to or from memory.

References and further reading

1. David A. Patterson and John L. Hennessy. Computer Organization and Design RISC-V Edition, 2nd edition. Morgan Kaufmann, 2020. Publisher information.
2. David A. Patterson and John L. Hennessy. Computer Architecture: A Quantitative Approach, 6th edition. Morgan Kaufmann, 2017. Publisher information.
3. RISC-V International. RISC-V Specifications. Architecture specifications.
4. Agner Fog. Software optimization resources. Performance reference materials.

Try it yourself

1. Explain the difference between an ISA and a microarchitecture.
2. Why can pipelining improve throughput without making one instruction finish sooner?
3. Give an example of temporal locality and one of spatial locality.
4. Estimate CPU time if a program executes 2 billion instructions at an average of 1.5 cycles each on a 3 GHz processor.
5. Describe one reason adding more CPU cores may not improve a program's speed.

Remember: software performance emerges from instruction execution, data movement, parallelism, and the specific hardware implementation.