Storm (event processor)

Apache StormApache Storm's Logo

Distributed and fault-tolerant realtime computation

Developer(s)| Backtype, Twitter
Stable release|

2.1.0 / 6 September 2019; 6 years ago (2019-09-06)

Repository| Storm Repository
Written in| Clojure & Java
Operating system| Cross-platform
Type| Distributed stream processing
License| Apache License 2.0
Website| storm.apache.org

Apache Storm is a distributed stream processing computation framework written predominantly in the Clojure programming language. Originally created by Nathan Marz[1] and team at BackType,[2] the project was open sourced after being acquired by Twitter.[3] It uses custom created "spouts" and "bolts" to define information sources and manipulations to allow batch, distributed processing of streaming data. The initial release was on 17 September 2011.[4]

A Storm application is designed as a "topology" in the shape of a directed acyclic graph (DAG) with spouts and bolts acting as the graph vertices. Edges on the graph are named streams and direct data from one node to another. Together, the topology acts as a data transformation pipeline. At a superficial level the general topology structure is similar to a MapReduce job, with the main difference being that data is processed in real time as opposed to in individual batches. Additionally, Storm topologies run indefinitely until killed, while a MapReduce job DAG must eventually end.[5]

Storm became an Apache Top-Level Project in September 2014[6] and was previously in incubation since September 2013.[7][8]

## Contents

* 1 Development
* 2 Apache Storm Architecture
* 3 Peer platforms
* 4 See also
* 5 References
* 6 External links

## Development

Apache Storm is developed under the Apache License, making it available to most companies to use.[9] Git is used for version control and Atlassian JIRA for issue tracking, under the Apache Incubator program.

Major Releases[10] Version | Release Date

|

2.1.0  | 6 September 2019
1.2.3  | 18 July 2019
2.0.0  | 30 May 2019
1.1.4  | 8 January 2019
1.2.2  | 4 June 2018
1.1.3
1.0.7  | 3 May 2018
1.2.1  | 19 February 2018
1.2.0  | 15 February 2018
1.1.2
1.0.6  | 14 February 2018
1.0.5  | 15 September 2017
1.1.1  | 1 August 2017
1.0.4  | 28 July 2017
1.1.0 | 29 Mar 2017
1.0.3  | 14 February 2017
0.10.2  | 14 September 2016
0.9.7  | 7 September 2016
1.0.2  | 10 August 2016
1.0.1  | 6 May 2016
0.10.1  | 5 May 2016
1.0.0 | 12 April 2016
0.10.0 | 5 November 2015
0.9.6
0.9.5 | 4 June 2015
0.9.4 | 25 March 2015
0.9.3 | 25 November 2014
0.9.2 | 25 June 2014
0.9.1 | 10 February 2014
Historical (non-Apache) Version | Release Date
0.9.0 | 8 December 2013
0.8.2  | 11 January 2013
0.8.1  | 6 September 2012
0.8.0 | 2 August 2012
0.7.0 | 28 February 2012
0.6.0 | 15 December 2011
0.5.0 | 19 September 2011

## Apache Storm Architecture

The Apache Storm cluster comprises following critical components:

* Nodes- There are two types of nodes: Master Nodes and Worker Nodes. A Master Node executes a daemon Nimbus which assigns tasks to machines and monitors their performances. On the other hand, a Worker Node runs the daemon called Supervisor which assigns the tasks to other worker nodes and operates them as per the need. As Storm cannot monitor the state and health of cluster, it deploys ZooKeeper to solve this issue which connects Nimbus with the Supervisors.
* Components- Storm has three critical components: Topology, Stream, and Spout. Topology is a network made of Stream and Spout. Stream is an unbounded pipeline of tuples and Spout is the source of the data streams which converts the data into the tuple of streams and sends to the bolts to be processed.[citation needed]

## Peer platforms

Storm is but one of dozens of stream processing engines, for a more complete list see Stream processing. Twitter announced Heron on June 2, 2015[11] which is API compatible with Storm. There are other comparable streaming data engines such as Spark Streaming and Flink.[12]

## See also

* C++ AMP
* Data parallelism
* Lambda architecture
* Message passing
* OpenMP
* OpenCL
* OpenHMPP
* Parallel computing
* TPL
* Thread (computing)

## References

1. ↑ Marz, Nathan. "About Nathan Marz". Nathan Marz. http://nathanmarz.com/about/. Retrieved 28 March 2013.
2. ↑ "BackType Website (defunct)". BackType. http://www.backtype.com/. Retrieved 28 March 2013.
3. ↑ "A Storm is coming: more details and plans for release". Engineering Blog. Twitter Inc. https://blog.twitter.com/2011/storm-coming-more-details-and-plans-release. Retrieved 29 July 2015.
4. ↑ "Storm Codebase". Github. https://github.com/nathanmarz/storm/commit/9d91adbdbde22e91779b91eb40805f598da5b004. Retrieved 8 February 2013.
5. ↑ "Tutorial - Components of a Storm cluster". Documentation. Apache Storm. http://storm.apache.org/documentation/Tutorial.html. Retrieved 29 July 2015.
6. ↑ "Apache Storm Graduates to a Top-Level Project". http://hortonworks.com/blog/apache-storm-graduates-top-level-project/.
7. ↑ "Storm Project Incubation Status". Apache Software Foundation. http://incubator.apache.org/projects/storm.html. Retrieved 29 October 2013.
8. ↑ "Storm Proposal". Apache Software Foundation. http://wiki.apache.org/incubator/StormProposal. Retrieved 29 October 2013.
9. ↑ "Powered By Storm". Documentation. Apache Storm. http://storm.apache.org/documentation/Powered-By.html. Retrieved 29 July 2015.
10. ↑ "Apache Storm". http://storm.apache.org/.
11. ↑ "Flying faster with Twitter Heron". Engineering Blog. Twitter Inc. https://blog.twitter.com/2015/flying-faster-with-twitter-heron. Retrieved 3 June 2015.
12. ↑ "Benchmarking Streaming Computation Engines: Storm, Flink and Spark Streaming" (PDF). IEEE. May 2016. https://ieeexplore.ieee.org/document/7530084.

## External links

* Project Homepage

* v
* t
* e

Apache Software Foundation

Top-level
projects|

* Accumulo
* ActiveMQ
* Airflow
* Ambari
* Ant
* Apex
* Aries
* Apache HTTP Server
* APR
* Avro
* Axis
* Axis2
* Beam
* Bloodhound
* Brooklyn
* Buildr
* Calcite
* Camel
* CarbonData
* Cassandra
* Cayenne
* Chemistry
* CloudStack
* Cocoon
* Cordova
* CouchDB
* cTAKES
* CXF
* Derby
* Directory
* Drill
* Empire-db
* Felix
* Flex
* Flink
* Flume
* Forrest
* Geronimo
* Giraph
* Gump
* Hadoop
* Hama
* HBase
* Helix
* Hive
* Impala
* Jackrabbit
* James
* Jini
* JMeter
* Kafka
* Karaf
* Kudu
* Kylin
* Lucene
* Mahout
* Marmotta
* Maven
* MINA
* mod perl
* MyFaces
* NetBeans
* Nutch
* ODE
* OFBiz
* Oozie
* OpenEJB
* OpenJPA
* OpenNLP
* OрenOffice
* ORC
* PDFBox
* Parquet
* Phoenix
* POI
* Pig
* Pivot
* Qpid
* Roller
* RocketMQ
* Samza
* ServiceMix
* Shiro
* SINGA
* Sling
* Solr
* Spark
* Stanbol
* Storm
* SpamAssassin
* Sqoop
* Struts 1
* Struts 2
* Subversion
* SystemML
* Tapestry
* Thrift
* Tika
* Tomcat
* Trafodion
* Traffic Server
* UIMA
* Velocity
* Wicket
* Xalan
* Xerces
* Yetus
* ZooKeeper

Commons|

* BCEL
* BSF
* Daemon
* Jelly
* Logging

Incubator|

* MXNet
* Taverna
* XAP

Other projects|

* Batik
* Chainsaw
* FOP
* Ivy
* Log4j

Attic|

* Abdera
* AxKit
* Beehive
* Bluesky
* iBATIS
* Cactus
* Click
* Continuum
* Deltacloud
* Etch
* Excalibur
* Harmony
* HiveMind
* Jakarta
* Lenya
* Shale
* Shindig
* Slide
* stdcxx
* Tuscany
* Wave
* Wink
* XMLBeans

Licenses|

* Apache License

* Category

* v
* t
* e

Parallel computing

General|

* Distributed computing
* Parallel computing
* Massively parallel
* Cloud computing
* High-performance computing
* Multiprocessing
* Manycore processor
* GPGPU
* Computer network
* Systolic array

Levels|

* Bit
* Instruction
* Thread
* Task
* Data
* Memory
* Loop
* Pipeline

Multithreading|

* Temporal
* Simultaneous (SMT)
* Speculative (SpMT)
* Preemptive
* Cooperative
* Clustered Multi-Thread (CMT)
* Hardware scout

Theory|

* PRAM model
* PEM Model
* Analysis of parallel algorithms
* Amdahl's law
* Gustafson's law
* Cost efficiency
* Karp–Flatt metric
* Slowdown
* Speedup

Elements|

* Process
* Thread
* Fiber
* Instruction window
* Array data structure

Coordination|

* Multiprocessing
* Memory coherency
* Cache coherency
* Cache invalidation
* Barrier
* Synchronization
* Application checkpointing

Programming|

* Stream processing
* Dataflow programming
* Models
* Implicit parallelism
* Explicit parallelism
* Concurrency
* Non-blocking algorithm

Hardware|

* Flynn's taxonomy
* SISD
* SIMD
* SIMT
* MISD
* MIMD
* Dataflow architecture
* Pipelined processor
* Superscalar processor
* Vector processor
* Multiprocessor
* symmetric
* asymmetric
* Memory
* shared
* distributed
* distributed shared
* UMA
* NUMA
* COMA
* Massively parallel computer
* Computer cluster
* Grid computer
* Hardware acceleration

APIs|

* Ateji PX
* Boost.Thread
* Chapel
* HPX
* Charm++
* Cilk
* Coarray Fortran
* CUDA
* HIP
* Dryad
* C++ AMP
* Global Arrays
* MPI
* OpenMP
* OpenCL
* OpenHMPP
* OpenACC
* TPL
* PLINQ
* PVM
* POSIX Threads
* RaftLib
* UPC
* TBB
* ZPL

Problems|

* Automatic parallelization
* Deadlock
* Livelock
* Deterministic algorithm
* Embarrassingly parallel
* Parallel slowdown
* Race condition
* Software lockout
* Scalability
* Starvation

* Category: parallel computing
* Media related to Parallel computing at Wikimedia Commons

0.00

(0 votes)

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Storm_(event_processor)&oldid=737507"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template