Massively parallel

Short description: Use of many processors to perform simultaneous operations

Massively parallel is the term for using a large number of computer processors (or separate computers) to simultaneously perform a set of coordinated computations in parallel. GPUs are massively parallel architecture with tens of thousands of threads.

One approach is grid computing, where the processing power of many computers in distributed, diverse administrative domains is opportunistically used whenever a computer is available.[1] An example is BOINC, a volunteer-based, opportunistic grid system, whereby the grid provides power only on a best effort basis.[2]

Another approach is grouping many processors in close proximity to each other, as in a computer cluster. In such a centralized system the speed and flexibility of the interconnect becomes very important, and modern supercomputers have used various approaches ranging from enhanced InfiniBand systems to three-dimensional torus interconnects.[3]

The term also applies to massively parallel processor arrays (MPPAs), a type of integrated circuit with an array of hundreds or thousands of central processing units (CPUs) and random-access memory (RAM) banks. These processors pass work to one another through a reconfigurable interconnect of channels. By harnessing many processors working in parallel, an MPPA chip can accomplish more demanding tasks than conventional chips.[citation needed] MPPAs are based on a software parallel programming model for developing high-performance embedded system applications.

Goodyear MPP was an early implementation of a massively parallel computer architecture. MPP architectures are the second most common supercomputer implementations after clusters, as of November 2013.[4]

Data warehouse appliances such as Teradata, Netezza or Microsoft's PDW commonly implement an MPP architecture to handle the processing of very large amounts of data in parallel.

## See also

* Multiprocessing
* Embarrassingly parallel
* Parallel computing
* Process-oriented programming
* Shared-nothing architecture (SN)
* Symmetric multiprocessing (SMP)
* Connection Machine
* Cellular automaton
* CUDA framework
* Manycore processor
* Vector processor

## References

1. ↑ Grid computing: experiment management, tool integration, and scientific workflows by Radu Prodan, Thomas Fahringer 2007 ISBN 3-540-69261-4 pages 1–4
2. ↑ Parallel and Distributed Computational Intelligence by Francisco Fernández de Vega 2010 ISBN 3-642-10674-9 pages 65–68
3. ↑ Knight, Will: "IBM creates world's most powerful computer", NewScientist.com news service, June 2007
4. ↑ "Archived copy". http://s.top500.org/static/lists/2013/11/TOP500_201311_Poster.png.

* v
* t
* e

Parallel computing

General|

* Distributed computing
* Parallel computing
* Massively parallel
* Cloud computing
* High-performance computing
* Multiprocessing
* Manycore processor
* GPGPU
* Computer network
* Systolic array

Levels|

* Bit
* Instruction
* Thread
* Task
* Data
* Memory
* Loop
* Pipeline

Multithreading|

* Temporal
* Simultaneous (SMT)
* Speculative (SpMT)
* Preemptive
* Cooperative
* Clustered Multi-Thread (CMT)
* Hardware scout

Theory|

* PRAM model
* PEM Model
* Analysis of parallel algorithms
* Amdahl's law
* Gustafson's law
* Cost efficiency
* Karp–Flatt metric
* Slowdown
* Speedup

Elements|

* Process
* Thread
* Fiber
* Instruction window
* Array data structure

Coordination|

* Multiprocessing
* Memory coherency
* Cache coherency
* Cache invalidation
* Barrier
* Synchronization
* Application checkpointing

Programming|

* Stream processing
* Dataflow programming
* Models
* Implicit parallelism
* Explicit parallelism
* Concurrency
* Non-blocking algorithm

Hardware|

* Flynn's taxonomy
* SISD
* SIMD
* SIMT
* MISD
* MIMD
* Dataflow architecture
* Pipelined processor
* Superscalar processor
* Vector processor
* Multiprocessor
* symmetric
* asymmetric
* Memory
* shared
* distributed
* distributed shared
* UMA
* NUMA
* COMA
* Massively parallel computer
* Computer cluster
* Grid computer
* Hardware acceleration

APIs|

* Ateji PX
* Boost.Thread
* Chapel
* HPX
* Charm++
* Cilk
* Coarray Fortran
* CUDA
* HIP
* Dryad
* C++ AMP
* Global Arrays
* MPI
* OpenMP
* OpenCL
* OpenHMPP
* OpenACC
* TPL
* PLINQ
* PVM
* POSIX Threads
* RaftLib
* UPC
* TBB
* ZPL

Problems|

* Automatic parallelization
* Deadlock
* Livelock
* Deterministic algorithm
* Embarrassingly parallel
* Parallel slowdown
* Race condition
* Software lockout
* Scalability
* Starvation

* Category: parallel computing
* Media related to Parallel computing at Wikimedia Commons

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Massively parallel. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Massively_parallel&oldid=185763"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template