Elliptics

Open source distributed key-value data storage

Elliptics

Developer(s)| Evgeniy Polakov with Yandex support
Initial release| 2008; 17 years ago (2008)
Repository|

* github.com/reverbrain/elliptics

Written in| C++, Python, Go
Type| NoSQL
License| GNU Lesser General Public License
Website| www.reverbrain.com

Elliptics is a distributed key–value data storage with open source code. By default it is a classic distributed hash table (DHT) with multiple replicas put in different groups (distributed hashes). Elliptics was created to meet requirements of multi-datacenter and physically distributed storage locations when storing huge amount of medium and large files (1 KB up to gigabytes in size, thousands to billions of objects).

## History

[edit]

Elliptics was created in 2007, initially as a part of POHMELFS,[1][2] a cache coherent distributed file system developed by Linux programmer Evgeniy Polyakov. POHMELFS was announced on January 31, 2008,[3] and merged into the staging area of the Linux kernel source tree in version 2.6.30, released June 9, 2009. The filesystem went practically unused and was removed again in February 2012.[4]

In 2008 Elliptics separated as an independent project. Polyakov tried different approaches to distributed data storage systems, some of them were not suitable because of their complexity and some of them were too far from a real-life (BerkeleyDB, LevelDB, Kyoto Cabinet backends for medium and big files, different datacenters in a single DHT ring, non eventual recovery). Elliptics is eventually consistent system with multiple updated in parallel replicas potentially living in physically distributed locations. Elliptics contains multiple layers from low-level on-disk store (named Eblob) up to SLRU caches and dynamic routing protocol.[5]

In 2012, Polyakov announced a new version of POHMELFS based on Elliptics.[4]

As of 2014, Elliptics is used in Yandex Maps, Disk, Music, Photos, Market and infrastructure,[6] Sputnik search engine and Coub.

## Architecture

[edit]

By default, Elliptics forms a distributed hash table in single group (a replica). Group may contains one or many servers as well as physical server can contain multiple elliptics groups (replicas) stored on different backends. Groups can live in different physical locations thus allowing to serve clients' requests when other locations are not accessible. A peer-to-peer (P2P) protocol can be used to access data directly from storage servers without proxying. Elliptics supports server-side scripting in C++, JavaScript, Python, based on the Cocaine technology, SLRU cache and multiple pluggable backends (eblob is the fastest for medium and large data and the most popular one).

Elliptics clients connect directly to all storage servers which helps to:

* Execute lookup in O(1) network requests (single network request per replica)
* Run write/update commands into multiple replicas in parallel

There are several application programming interfaces (APIs) for data access:

* Asynchronous feature-promise C++ library
* Python binding
* Go binding
* HTTP-proxy named Rift with buckets and ACLs based on TheVoid library (using boost::asio)
* Community-driven Erlang bindings

## Features

[edit]

* Distributed hash tables, no metadata servers, true horizontal scaling
* Data replication – replicas can be stored in different physical locations
* Range and bulk requests
* Different I/O storage backends, API to create own low-level storage backends
* Automatic data repartitioning in case of removed or added nodes
* Eventually consistent recovery
* Consistent hashing addressing model
* Cluster statistics
* Frontend: HTTP; bindings: C/C++, Go, Python
* Server-side script execution support (write trigger analog)
* Distributed SLRU cache with TTL
* P2P streaming support (eblob and file backends only - external applications like Nginx web server can stream data from eblob object files directly to clients without proxying)

## Problems and restrictions

[edit]

* Eventual consistency: as Elliptics is fully distributed in case of emergency server can possibly return a file copy which is older than an actual one. Sometimes it can be unacceptable. In these cases due to time loses it is better to use more reliable ways of data request.
* Network between client and servers can become a weak point as data is written on several servers in parallel.
* API may be inconvenient for high-level requests. Elliptics does not provide its users with SQL-like data requests.
* Elliptics does not have high-level transactions support that is why it is impossible to guarantee that a command group will be fully executed or will not be executed at all.
* Transactions are only atomic within group and are locked based on primary key.

## Documentation

[edit]

Elliptics and its supporting projects are being documented at community wiki Archived 2020-01-21 at the Wayback Machine. It contains high-level design docs, tutorial, low-level details and knowledge base. Elliptics and related projects are discussed in open Google group.

## See also

[edit]

* Free and open-source software portal

* MongoDB
* CouchDB
* Couchbase Server
* Apache Cassandra
* HBase
* Riak
* Elasticsearch
* Memcached
* Redis

## References

[edit]

1. ^ "POHMELFS: A New But Old Distributed Linux File-System - Phoronix". www.phoronix.com. Retrieved 8 May 2017.
2. ^ "POHMELFS returns [LWN.net]". lwn.net. Retrieved 8 May 2017.
3. ^ "Entry 2008.01.31". Archived from the original on 6 October 2008. Retrieved 8 May 2017.
4. ^ a b Evgeniy Polyakov (8 February 2012). "pohmelfs: call for inclusion".
5. ^ "Как устроены облака Яндекса: Elliptics". Retrieved 8 May 2017.
6. ^ "Elliptics — Yandex Technologies". api.yandex.com. Retrieved 8 May 2017.

## External links

[edit]

* Elliptics (official site).

* v
* t
* e

Yandex

People|

* Ilya Segalovich (co-founder)
* Arkady Volozh (co-founder)
* Tigran Khudaverdyan

Current Services|

* Alice
* Elliptics
* Kinopoisk
* MatrixNet
* Yandex Browser (Chromium)
* Yandex Disk
* Yandex.Direct
* Yandex.Drive
* Yandex Eda
* Yandex Mail
* Yandex Maps
* Yandex Market
* Yandex Launcher
* Yandex.Navigator
* Yandex News
* Yandex Search
* Yandex Taxi
* Yandex Translate

Former Services|

* Cocaine (PaaS)
* YooKassa
* YooMoney
* Zen (recommendation system)

Category

* v
* t
* e

File systems

* Comparison of file systems
* distributed
* Unix filesystem

Disk and
non-rotating|

* ADFS
* AdvFS
* Amiga FFS
* Amiga OFS
* APFS
* AthFS
* bcachefs
* BFS
* Be File System
* Boot File System
* Byte File System (z/VM)
* Btrfs
* CVFS
* CXFS
* DFS
* EFS
* Encrypting File System
* Extent File System
* Episode
* ext
* ext2
* ext3
* ext3cow
* ext4
* FAT
* exFAT
* Files-11
* Fossil
* GPFS
* HAMMER
* HAMMER2
* HFS (Classic Mac OS)
* HFS (MVS)
* HFS+
* HPFS
* HTFS
* JFS
* LFS
* MFS
* Macintosh File System
* TiVo Media File System
* MINIX
* NetWare File System
* Next3
* NILFS
* NILFS2
* NSS
* NTFS
* OneFS
* OpenZFS
* PFS
* QFS
* QNX4FS
* ReFS
* ReiserFS
* Reiser4
* Reliance
* Reliance Nitro
* RFS
* SFS
* Shared File System (VM)
* Smart File System
* SNFS
* Soup (Apple)
* Tux3
* UBIFS
* UFS/UFS2
* soft updates
* WAPBL
* VxFS
* WAFL
* Xiafs
* XFS
* Xsan
* zFS (z/OS)
* ZFS (Sun)

Optical disc|

* HSF
* ISO 9660
* ISO 13490
* UDF

Flash memory and SSD|

* APFS
* FAT
* exFAT
* TFAT
* EROFS
* F2FS
* JFS
* NVFS

| host-side wear leveling|

* CHFS
* JFFS
* JFFS2
* LogFS
* NILFS
* NILFS2
* YAFFS
* UBIFS

|

Distributed parallel|

* BeeGFS
* Ceph
* CXFS
* GFS2
* Google File System
* OCFS2
* OrangeFS
* PVFS
* QFS
* Xsan
* more...

NAS|

* 9P
* AFS (OpenAFS)
* AFP
* Coda
* DFS
* Google File System
* GPFS
* Lustre
* NCP
* NFS
* POHMELFS
* Hadoop
* SMB (CIFS)
* SSHFS
* more...

Specialized|

* Aufs
* AXFS
* Boot File System
* Compact Disc File System
* cramfs
* Davfs2
* EROFS
* FTPFS
* FUSE
* Lnfs
* LTFS
* NOVA
* MVFS
* SquashFS
* UMSDOS
* OverlayFS
* UnionFS

Pseudo|

* configfs
* devfs
* debugfs
* kernfs
* procfs
* specfs
* sysfs
* tmpfs
* WinFS

Encrypted|

* eCryptfs
* EncFS
* EFS
* Rubberhose
* SSHFS
* ZFS

Types|

* Clustered
* Global
* Grid
* Self-certifying
* Flash
* Journaling
* Log-structured
* Object
* Record-oriented
* Semantic
* Steganographic
* Synthetic
* Versioning

Features|

* Case preservation
* Copy-on-write
* Data deduplication
* Data scrubbing
* Execute in place
* Extent
* File attribute
* Extended file attributes
* File change log
* Fork
* Inode
* Links
* Hard
* Symbolic

Access control|

* Access-control list
* Filesystem-level encryption
* Permissions
* Modes
* Sticky bit

Interfaces|

* File manager
* File system API
* Installable File System
* Virtual file system

Lists|

* Cryptographic
* Default
* Log-structured

Layouts|

* Master Boot Record
* GUID Partition Table
* Apple Partition Map

* v
* t
* e

Cloud computing

Business models|

* Content as a service
* Data as a service
* Desktop as a service
* Function as a service
* Infrastructure as a service
* Integration platform as a service
* Backend as a service
* Network as a service
* Platform as a service
* Security as a service
* Software as a service

Technologies|

* Cloud database
* Cloud-native computing
* Cloud storage
* Cloud storage gateways
* Data centers
* Dew computing
* Distributed file system for cloud
* Hardware virtualization
* Internet
* Mobile cloud computing
* Native cloud application
* Networking
* Personal cloud
* Security
* Serverless computing
* Structured storage
* Virtual appliance
* Web APIs
* Virtual private cloud

Applications|

* Box
* Dropbox
* Google
* Workspace
* Drive
* HP Cloud (closed)
* IBM Cloud
* Lark
* Microsoft
* Office 365
* OneDrive
* Nextcloud
* Oracle Cloud
* Owncloud
* Proton Drive
* Rackspace
* Salesforce
* Seafile
* PlateSpin
* Workday
* Zoho

Platforms|

* Alibaba Cloud
* Amazon Web Services
* AppScale
* Box
* CloudBolt
* Cloud Foundry
* Cocaine (PaaS)
* Engine Yard
* Helion
* GE Predix
* Google App Engine
* GreenQloud
* Heroku
* IBM Cloud
* Inktank
* Jelastic
* Microsoft Azure
* MindSphere
* Netlify
* Oracle Cloud
* OutSystems
* openQRM
* OpenShift
* PythonAnywhere
* RightScale
* Scalr
* Force.com
* SAP Business Technology Platform
* Splunk
* Vercel
* vCloud Air
* WaveMaker

Infrastructure|

* Alibaba Cloud
* Amazon Web Services
* Abiquo Enterprise Edition
* CloudStack
* Citrix Cloud
* Deft
* DigitalOcean
* EMC Atmos
* Eucalyptus
* Fujitsu
* Google Cloud
* GreenButton
* GreenQloud
* IBM Cloud
* iland
* Joyent
* Linode
* Lunacloud
* Microsoft Azure
* Mirantis
* Netlify
* Nimbula
* Nimbus
* OpenIO
* OpenNebula
* OpenStack
* Oracle Cloud
* OrionVM
* Rackspace Cloud
* Safe Swiss Cloud
* Zadara
* libvirt
* libguestfs
* OVirt
* Virtual Machine Manager
* Wakame-vdc
* Vercel
* Virtual Private Cloud OnDemand

* Category
* Commons

*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template