Multithreading (computer architecture)

Short description: Ability of a CPU to provide multiple threads of execution concurrently

A process with two threads of execution, running on a single processor

In computer architecture, multithreading is the ability of a central processing unit (CPU) (or a single core in a multi-core processor) to provide multiple threads of execution concurrently, supported by the operating system. This approach differs from multiprocessing. In a multithreaded application, the threads share the resources of a single or multiple cores, which include the computing units, the CPU caches, and the translation lookaside buffer (TLB).

Where multiprocessing systems include multiple complete processing units in one or more cores, multithreading aims to increase utilization of a single core by using thread-level parallelism, as well as instruction-level parallelism. As the two techniques are complementary, they are combined in nearly all modern systems architectures with multiple multithreading CPUs and with CPUs with multiple multithreading cores.

## Contents

* 1 Overview
* 1.1 Advantages
* 1.2 Disadvantages
* 2 Types of multithreading
* 2.1 Interleaved/Temporal multithreading
* 2.1.1 Coarse-grained multithreading
* 2.1.2 Fine grained multithreading
* 2.2 Simultaneous multithreading
* 3 Implementation specifics
* 4 See also
* 5 References
* 6 External links

## Overview

The multithreading paradigm has become more popular as efforts to further exploit instruction-level parallelism have stalled since the late 1990s. This allowed the concept of throughput computing to re-emerge from the more specialized field of transaction processing. Even though it is very difficult to further speed up a single thread or single program, most computer systems are actually multitasking among multiple threads or programs. Thus, techniques that improve the throughput of all tasks result in overall performance gains.

Two major techniques for throughput computing are multithreading and multiprocessing.

### Advantages

If a thread gets a lot of cache misses, the other threads can continue taking advantage of the unused computing resources, which may lead to faster overall execution, as these resources would have been idle if only a single thread were executed. Also, if a thread cannot use all the computing resources of the CPU (because instructions depend on each other's result), running another thread may prevent those resources from becoming idle.

### Disadvantages

Multiple threads can interfere with each other when sharing hardware resources such as caches or translation lookaside buffers (TLBs). As a result, execution times of a single thread are not improved and can be degraded, even when only one thread is executing, due to lower frequencies or additional pipeline stages that are necessary to accommodate thread-switching hardware.

Overall efficiency varies; Intel claims up to 30% improvement with its Hyper-Threading Technology,[1] while a synthetic program just performing a loop of non-optimized dependent floating-point operations actually gains a 100% speed improvement when run in parallel. On the other hand, hand-tuned assembly language programs using MMX or AltiVec extensions and performing data prefetches (as a good video encoder might) do not suffer from cache misses or idle computing resources. Such programs therefore do not benefit from hardware multithreading and can indeed see degraded performance due to contention for shared resources.

From the software standpoint, hardware support for multithreading is more visible to software, requiring more changes to both application programs and operating systems than multiprocessing. Hardware techniques used to support multithreading often parallel the software techniques used for computer multitasking. Thread scheduling is also a major problem in multithreading.

## Types of multithreading

### Interleaved/Temporal multithreading

Main article: Temporal multithreading

#### Coarse-grained multithreading

The simplest type of multithreading occurs when one thread runs until it is blocked by an event that normally would create a long-latency stall. Such a stall might be a cache miss that has to access off-chip memory, which might take hundreds of CPU cycles for the data to return. Instead of waiting for the stall to resolve, a threaded processor would switch execution to another thread that was ready to run. Only when the data for the previous thread had arrived, would the previous thread be placed back on the list of ready-to-run threads.

For example:

1. Cycle i: instruction j from thread A is issued.
2. Cycle i \+ 1: instruction j \+ 1 from thread A is issued.
3. Cycle i \+ 2: instruction j \+ 2 from thread A is issued, which is a load instruction that misses in all caches.
4. Cycle i \+ 3: thread scheduler invoked, switches to thread B.
5. Cycle i \+ 4: instruction k from thread B is issued.
6. Cycle i \+ 5: instruction k \+ 1 from thread B is issued.

Conceptually, it is similar to cooperative multi-tasking used in real-time operating systems, in which tasks voluntarily give up execution time when they need to wait upon some type of the event. This type of multithreading is known as block, cooperative or coarse-grained multithreading.

The goal of multithreading hardware support is to allow quick switching between a blocked thread and another thread ready to run. Switching from one thread to another means the hardware switches from using one register set to another. To achieve this goal, the hardware for the program visible registers, as well as some processor control registers (such as the program counter), is replicated. For example, to quickly switch between two threads, the processor is built with two sets of registers.

Additional hardware support for multithreading allows thread switching to be done in one CPU cycle, bringing performance improvements. Also, additional hardware allows each thread to behave as if it were executing alone and not sharing any hardware resources with other threads, minimizing the amount of software changes needed within the application and the operating system to support multithreading.

Many families of microcontrollers and embedded processors have multiple register banks to allow quick context switching for interrupts. Such schemes can be considered a type of block multithreading among the user program thread and the interrupt threads.[citation needed]

#### Fine grained multithreading

Main article: Barrel processor

The purpose of Fine grained multithreading is to remove all data dependency stalls from the execution pipeline. Since one thread is relatively independent from other threads, there is less chance of one instruction in one pipelining stage needing an output from an older instruction in the pipeline. Conceptually, it is similar to preemptive multitasking used in operating systems; an analogy would be that the time slice given to each active thread is one CPU cycle.

For example:

1. Cycle i \+ 1: an instruction from thread B is issued.
2. Cycle i \+ 2: an instruction from thread C is issued.

This type of multithreading was first called barrel processing, in which the staves of a barrel represent the pipeline stages and their executing threads. Interleaved, preemptive, fine-grained or time-sliced multithreading are more modern terminology.

In addition to the hardware costs discussed in the block type of multithreading, interleaved multithreading has an additional cost of each pipeline stage tracking the thread ID of the instruction it is processing. Also, since there are more threads being executed concurrently in the pipeline, shared resources such as caches and TLBs need to be larger to avoid thrashing between the different threads.

### Simultaneous multithreading

Main article: Simultaneous multithreading

The most advanced type of multithreading applies to superscalar processors. Whereas a normal superscalar processor issues multiple instructions from a single thread every CPU cycle, in simultaneous multithreading (SMT) a superscalar processor can issue instructions from multiple threads every CPU cycle. Recognizing that any single thread has a limited amount of instruction-level parallelism, this type of multithreading tries to exploit parallelism available across multiple threads to decrease the waste associated with unused issue slots.

For example:

1. Cycle i: instructions j and j \+ 1 from thread A and instruction k from thread B are simultaneously issued.
2. Cycle i \+ 1: instruction j \+ 2 from thread A, instruction k \+ 1 from thread B, and instruction m from thread C are all simultaneously issued.
3. Cycle i \+ 2: instruction j \+ 3 from thread A and instructions m \+ 1 and m \+ 2 from thread C are all simultaneously issued.

To distinguish the other types of multithreading from SMT, the term "temporal multithreading" is used to denote when instructions from only one thread can be issued at a time.

In addition to the hardware costs discussed for interleaved multithreading, SMT has the additional cost of each pipeline stage tracking the thread ID of each instruction being processed. Again, shared resources such as caches and TLBs have to be sized for the large number of active threads being processed.

Implementations include DEC (later Compaq) EV8 (not completed), Intel Hyper-Threading Technology, IBM POWER5/POWER6/POWER7/POWER8/POWER9, IBM z13/z14/z15, Sun Microsystems UltraSPARC T2, Cray XMT, and AMD Bulldozer and Zen microarchitectures.

## Implementation specifics

A major area of research is the thread scheduler that must quickly choose from among the list of ready-to-run threads to execute next, as well as maintain the ready-to-run and stalled thread lists. An important subtopic is the different thread priority schemes that can be used by the scheduler. The thread scheduler might be implemented totally in software, totally in hardware, or as a hardware/software combination.

Another area of research is what type of events should cause a thread switch: cache misses, inter-thread communication, DMA completion, etc.

If the multithreading scheme replicates all of the software-visible state, including privileged control registers and TLBs, then it enables virtual machines to be created for each thread. This allows each thread to run its own operating system on the same processor. On the other hand, if only user-mode state is saved, then less hardware is required, which would allow more threads to be active at one time for the same die area or cost.

## See also

* Async/await
* Super-threading
* Speculative multithreading

## References

1. ↑ "Intel Hyper-Threading Technology, Technical User's Guide". p. 13. http://cache-www.intel.com/cd/00/00/01/77/17705_htt_user_guide.pdf.

## External links

* A Survey of Processors with Explicit Multithreading, ACM, March 2003, by Theo Ungerer, Borut Robi and Jurij Silc
* Operating System | Difference between Multitasking, Multithreading and Multiprocessing GeeksforGeeks, 6 Sept. 2018.

* v
* t
* e

Computer science

Note: This template roughly follows the 2012 ACM Computing Classification System.

Hardware|

* Printed circuit board
* Peripheral
* Integrated circuit
* Very Large Scale Integration
* Systems on Chip (SoCs)
* Energy consumption (Green computing)
* Electronic design automation
* Hardware acceleration

Computer systems
organization|

* Computer architecture
* Embedded system
* Real-time computing
* Dependability

Networks|

* Network architecture
* Network protocol
* Network components
* Network scheduler
* Network performance evaluation
* Network service

Software organization|

* Interpreter
* Middleware
* Virtual machine
* Operating system
* Software quality

Software notations
and tools|

* Programming paradigm
* Programming language
* Compiler
* Domain-specific language
* Modeling language
* Software framework
* Integrated development environment
* Software configuration management
* Software library
* Software repository

Software development|

* Software development process
* Requirements analysis
* Software design
* Software construction
* Software deployment
* Software maintenance
* Programming team
* Open-source model

Theory of computation|

* Model of computation
* Formal language
* Automata theory
* Computational complexity theory
* Logic
* Semantics

Algorithms|

* Algorithm design
* Analysis of algorithms
* Algorithmic efficiency
* Randomized algorithm
* Computational geometry

Mathematics
of computing|

* Discrete mathematics
* Probability
* Statistics
* Mathematical software
* Information theory
* Mathematical analysis
* Numerical analysis

Information
systems|

* Database management system
* Information storage systems
* Enterprise information system
* Social information systems
* Geographic information system
* Decision support system
* Process control system
* Multimedia information system
* Data mining
* Digital library
* Computing platform
* Digital marketing
* World Wide Web
* Information retrieval

Security|

* Cryptography
* Formal methods
* Security services
* Intrusion detection system
* Hardware security
* Network security
* Information security
* Application security

Human–computer
interaction|

* Interaction design
* Social computing
* Ubiquitous computing
* Visualization
* Accessibility

Concurrency|

* Concurrent computing
* Parallel computing
* Distributed computing
* Multithreading
* Multiprocessing

Artificial
intelligence|

* Natural language processing
* Knowledge representation and reasoning
* Computer vision
* Automated planning and scheduling
* Search methodology
* Control method
* Philosophy of artificial intelligence
* Distributed artificial intelligence

Machine learning|

* Supervised learning
* Unsupervised learning
* Reinforcement learning
* Multi-task learning
* Cross-validation

Graphics|

* Animation
* Rendering
* Image manipulation
* Graphics processing unit
* Mixed reality
* Virtual reality
* Image compression
* Solid modeling

Applied
computing|

* E-commerce
* Enterprise software
* Computational mathematics
* Computational physics
* Computational chemistry
* Computational biology
* Computational social science
* Computational engineering
* Computational healthcare
* Digital art
* Electronic publishing
* Cyberwarfare
* Electronic voting
* Video games
* Word processing
* Operations research
* Educational technology
* Document management

* Book
* Category
* Outline
* WikiProject
* Commons

* v
* t
* e

Processor technologies

Models|

* Turing machine
* Universal
* Post–Turing
* Quantum
* Belt machine
* Stack machine
* Finite-state machine
* with datapath
* Hierarchical
* Queue automaton
* Register machines
* Counter
* Pointer
* Random-access
* Random-access stored program

Architecture|

* Microarchitecture
* Von Neumann
* Harvard
* modified
* Dataflow
* Transport-triggered
* Cellular
* Endianness
* Memory access
* NUMA
* HUMA
* Load/store
* Register/memory
* Cache hierarchy
* Memory hierarchy
* Virtual memory
* Secondary storage
* Heterogeneous
* Fabric
* Multiprocessing
* Cognitive
* Neuromorphic

Instruction set
architectures|

| Types|

* CISC
* RISC
* Application-specific
* EDGE
* TRIPS
* VLIW
* EPIC
* MISC
* OISC
* NISC
* ZISC
* comparison
* addressing modes

|

* x86
* ARM
* MIPS
* Power ISA
* SPARC
* Itanium
* Unicore
* MicroBlaze
* RISC-V
* others

Execution|

| Instruction pipelining|

* Pipeline stall
* Operand forwarding
* Classic RISC pipeline

|

Hazards|

* Data dependency
* Structural
* Control
* False sharing

Out-of-order|

* Tomasulo algorithm
* Reservation station
* Re-order buffer
* Register renaming

Speculative|

* Branch prediction
* Memory dependence prediction

Parallelism|

| Level|

* Bit
* Bit-serial
* Word
* Instruction
* Pipelining
* Scalar
* Superscalar
* Task
* Thread
* Process
* Data
* Vector
* Memory
* Distributed

|

Multithreading|

* Temporal
* Simultaneous
* Hyperthreading
* Speculative
* Preemptive
* Cooperative

Flynn's taxonomy|

* SISD
* SIMD
* SWAR
* SIMT
* MISD
* MIMD
* SPMD

Processor
performance|

* Transistor count
* Instructions per cycle (IPC)
* Cycles per instruction (CPI)
* Instructions per second (IPS)
* Floating-point operations per second (FLOPS)
* Transactions per second (TPS)
* Synaptic updates per second (SUPS)
* Performance per watt (PPW)
* Cache performance metrics
* Computer performance by orders of magnitude

Types|

* Central processing unit (CPU)
* Graphics processing unit (GPU)
* GPGPU
* Vector
* Barrel
* Stream
* Coprocessor
* ASIC
* FPGA
* CPLD
* Multi-chip module (MCM)
* System in package (SiP)

| By application|

* Microprocessor
* Microcontroller
* Mobile
* Notebook
* Ultra-low-voltage
* ASIP

|

Systems
on chip|

* System on a chip (SoC)
* Multiprocessor (MPSoC)
* Programmable (PSoC)
* Network on a chip (NoC)

Hardware
accelerators|

* AI accelerator
* Vision processing unit (VPU)
* Physics processing unit (PPU)
* Digital signal processor (DSP)
* Tensor processing unit (TPU)
* Secure cryptoprocessor
* Network processor
* Baseband processor

Word size|

* 1-bit
* 2-bit
* 4-bit
* 8-bit
* 16-bit
* 32-bit
* 48-bit
* 64-bit
* 128-bit
* 256-bit
* 512-bit
* others
* variable

Core count|

* Single-core
* Multi-core
* Manycore
* Heterogeneous architecture

Components|

* Core
* Cache
* CPU cache
* replacement policies
* coherence
* Bus
* Clock rate
* Clock signal
* FIFO

| Functional units|

* Arithmetic logic unit (ALU)
* Address generation unit (AGU)
* Floating-point unit (FPU)
* Memory management unit (MMU)
* Load–store unit
* Translation lookaside buffer (TLB)
* Integrated memory controller (IMC)

|

Logic|

* Combinational
* Sequential
* Glue
* Logic gate
* Quantum
* Array

Registers|

* Processor register
* Status register
* Stack register
* Register file
* Memory buffer
* Program counter

Control unit|

* Instruction unit
* Data buffer
* Write buffer
* Microcode ROM
* Counter

Datapath|

* Multiplexer
* Demultiplexer
* Adder
* Multiplier
* CPU
* Binary decoder
* Address decoder
* Sum addressed decoder
* Barrel shifter

Circuitry|

* Integrated circuit
* 3D
* Mixed-signal
* Power management
* Boolean
* Digital
* Analog
* Quantum
* Switch

Power
management|

* PMU
* APM
* ACPI
* Dynamic frequency scaling
* Dynamic voltage scaling
* Clock gating
* Performance per watt (PPW)
* Race to sleep

Related|

* History of general-purpose CPUs
* Microprocessor chronology
* Processor design
* Digital electronics
* Hardware security module
* Semiconductor device fabrication
* Tick–tock model

* v
* t
* e

Parallel computing

General|

* Distributed computing
* Parallel computing
* Massively parallel
* Cloud computing
* High-performance computing
* Multiprocessing
* Manycore processor
* GPGPU
* Computer network
* Systolic array

Levels|

* Bit
* Instruction
* Thread
* Task
* Data
* Memory
* Loop
* Pipeline

Multithreading|

* Temporal
* Simultaneous (SMT)
* Speculative (SpMT)
* Preemptive
* Cooperative
* Clustered Multi-Thread (CMT)
* Hardware scout

Theory|

* PRAM model
* PEM Model
* Analysis of parallel algorithms
* Amdahl's law
* Gustafson's law
* Cost efficiency
* Karp–Flatt metric
* Slowdown
* Speedup

Elements|

* Process
* Thread
* Fiber
* Instruction window
* Array data structure

Coordination|

* Multiprocessing
* Memory coherency
* Cache coherency
* Cache invalidation
* Barrier
* Synchronization
* Application checkpointing

Programming|

* Stream processing
* Dataflow programming
* Models
* Implicit parallelism
* Explicit parallelism
* Concurrency
* Non-blocking algorithm

Hardware|

* Flynn's taxonomy
* SISD
* SIMD
* SIMT
* MISD
* MIMD
* Dataflow architecture
* Pipelined processor
* Superscalar processor
* Vector processor
* Multiprocessor
* symmetric
* asymmetric
* Memory
* shared
* distributed
* distributed shared
* UMA
* NUMA
* COMA
* Massively parallel computer
* Computer cluster
* Grid computer
* Hardware acceleration

APIs|

* Ateji PX
* Boost.Thread
* Chapel
* HPX
* Charm++
* Cilk
* Coarray Fortran
* CUDA
* HIP
* Dryad
* C++ AMP
* Global Arrays
* MPI
* OpenMP
* OpenCL
* OpenHMPP
* OpenACC
* TPL
* PLINQ
* PVM
* POSIX Threads
* RaftLib
* UPC
* TBB
* ZPL

Problems|

* Automatic parallelization
* Deadlock
* Livelock
* Deterministic algorithm
* Embarrassingly parallel
* Parallel slowdown
* Race condition
* Software lockout
* Scalability
* Starvation

* Category: parallel computing
* Media related to Parallel computing at Wikimedia Commons

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Multithreading (computer architecture). Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Multithreading_(computer_architecture)&oldid=187485"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template