Comparison of distributed file systems

Short description: None

In computing, a distributed file system (DFS) or network file system is any file system that allows access to files from multiple hosts sharing via a computer network. This makes it possible for multiple users on multiple machines to share files and storage resources.

Distributed file systems differ in their performance, mutability of content, handling of concurrent writes, handling of permanent or temporary loss of nodes or storage, and their policy of storing content.

## Contents

* 1 Locally managed
* 1.1 FOSS
* 1.2 Proprietary
* 2 Remote access
* 3 Comparison
* 4 See also
* 5 References

## Locally managed

### FOSS

Client  | Written in  | License  | Access API  | High availability | Shards | Efficient Redundancy | Redundancy Granularity  | Initial release year  | Memory requirements (GB)

|
|
|
|
|
|
|
|
|

Alluxio (Virtual Distributed File System) | Java  | Apache License 2.0  | HDFS, FUSE, HTTP/REST, S3 | hot standby  | No  | Replication[1] | File[2] | 2013  |
Ceph  | C++  | LGPL  | librados (C, C++, Python, Ruby), S3, Swift, FUSE | Yes  | Yes  | Pluggable erasure codes[3] | Pool[4] | 2010  | 1 per TB of storage
Coda | C  | GPL  | C  | Yes  | Yes  | Replication  | Volume[5] | 1987  |
GlusterFS | C  | GPLv3  | libglusterfs, FUSE, NFS, SMB, Swift, libgfapi  | mirror  | Yes  | Reed-Solomon[6] | Volume[7] | 2005  |
HDFS  | Java  | Apache License 2.0  | Java and C client, HTTP, FUSE[8] | transparent master failover  | No  | Reed-Solomon[9] | File[10] | 2005  |
IPFS  | Go  | Apache 2.0 or MIT  | HTTP gateway, FUSE, Go client, Javascript client, command line tool | Yes  | with IPFS Cluster | Replication[11] | Block[12] | 2015[13] |
JuiceFS | Go  | Apache License 2.0  | POSIX, FUSE, HDFS, S3 | Yes  | Yes  | Reed-Solomon  | Object  | 2021  |
Kertish-DFS | Go  | GPLv3  | HTTP(REST), CLI, C# Client, Go Client  | Yes  |  | Replication  |  | 2020  |
LizardFS | C++  | GPLv3  | POSIX, FUSE, NFS-Ganesha, Ceph FSAL (via libcephfs) | master  | No  | Reed-Solomon[14] | File[15] | 2013  |
Lustre | C  | GPLv2  | POSIX, NFS-Ganesha, NFS, SMB | Yes  | Yes  | No redundancy[16][17] | No redundancy[18][19] | 2003  |
MinIO | Go  | AGPL3.0  | AWS S3 API, FTP, SFTP | Yes  | Yes  | Reed-Solomon[20] | Object[21] | 2014  |
MooseFS | C  | GPLv2  | POSIX, FUSE | master  | No  | Replication[22] | File[23] | 2008  |
OpenAFS | C  | IBM Public License  | Virtual file system, Installable File System |  |  | Replication  | Volume[24] | 2000[25] |
OpenIO[26] | C  | AGPLv3 / LGPLv3  | Native (Python, C, Java), HTTP/REST, S3, Swift, FUSE (POSIX, NFS, SMB, FTP)  | Yes  |  | Pluggable erasure codes[27] | Object[28] | 2015  | 0.5
Ori[29] | C, C++  | MIT  | libori, FUSE |  |  | Replication  | Filesystem[30] | 2012  |
Quantcast File System | C  | Apache License 2.0  | C++ client, FUSE (C++ server: MetaServer and ChunkServer are both in C++)  | master  | No  | Reed-Solomon[31] | File[32] | 2012  |
RozoFS | C, Python  | GPLv2  | FUSE, SMB, NFS, key/value  | Yes  |  | Mojette[33] | Volume[34] | 2011[35] |
SeaweedFS | Go, Java  | Apache License 2.0  | HTTP (REST), POSIX, FUSE, S3, HDFS  | requires CockroachDB, undocumented config  |  | Reed-Solomon[36] | Volume[37] | 2015  |
Storj | Go  | Apache License 2.0, Affero General Public License v3  | HTTP (REST), S3, Native (Go, C, Python, Java)  | Yes  |  | Reed-Solomon[38] | Object[38] | 2018  |
Tahoe-LAFS | Python  | GNU GPL[39] | HTTP (browser or CLI), SFTP, FTP, FUSE via SSHFS, pyfilesystem  |  |  | Reed-Solomon[40] | File[41] | 2007  |
XtreemFS | Java, C++  | BSD License  | libxtreemfs (Java, C++), FUSE |  |  | Replication[42] | File[43] | 2009  |

### Proprietary

Client  | Written in  | License  | Access API

|
|
|

BeeGFS | C / C++  | FRAUNHOFER FS (FhGFS) EULA,[44]

GPLv2 client

| POSIX
ObjectiveFS[45] | C  | Proprietary | POSIX, FUSE
Spectrum Scale (GPFS) | C, C++  | Proprietary | POSIX, NFS, SMB, Swift, S3, HDFS
MapR-FS | C, C++  | Proprietary | POSIX, NFS, FUSE, S3, HDFS, CLI
PanFS | C, C++  | Proprietary | DirectFlow, POSIX, NFS, SMB/CIFS, HTTP, CLI
Infinit[46] | C++  | Proprietary (to be open sourced)[47] | FUSE, Installable File System, NFS/SMB, POSIX, CLI, SDK (libinfinit)
Isilon OneFS | C/C++  | Proprietary | POSIX, NFS, SMB/CIFS, HDFS, HTTP, FTP, SWIFT Object, CLI, Rest API
Qumulo | C/C++  | Proprietary | POSIX, NFS, SMB/CIFS, CLI, S3, Rest API
Scality | C  | Proprietary | FUSE, NFS, REST, AWS S3
Quobyte | Java, C++  | Proprietary | POSIX, FUSE, NFS, SMB/CIFS, HDFS, AWS S3, TensorFlow Plugin, CLI, Rest API

## Remote access

Name  | Run by  | Access API

|
|

Amazon S3 | Amazon.com  | HTTP (REST/SOAP)
Google Cloud Storage | Google | HTTP (REST)
SWIFT (part of OpenStack)  | Rackspace, Hewlett-Packard, others  | HTTP (REST)
Microsoft Azure | Microsoft | HTTP (REST)
IBM Cloud Object Storage  | IBM (formerly Cleversafe)[48] | HTTP (REST)

## Comparison

Some researchers have made a functional and experimental analysis of several distributed file systems including HDFS, Ceph, Gluster, Lustre and old (1.6.x) version of MooseFS, although this document is from 2013 and a lot of information are outdated (e.g. MooseFS had no HA for Metadata Server at that time).[49]

The cloud based remote distributed storage from major vendors have different APIs and different consistency models.[50]

## See also

* Distributed file system
* List of file systems, the Distributed file systems section

## References

1. ↑ "Caching: Managing Data Replication in Alluxio". https://docs.alluxio.io/os/user/stable/en/core-services/Caching.html#managing-data-replication-in-alluxio.
2. ↑ "Caching: Managing Data Replication in Alluxio". https://docs.alluxio.io/os/user/stable/en/core-services/Caching.html#managing-data-replication-in-alluxio.
3. ↑ "Erasure Code Profiles". https://docs.ceph.com/en/latest/rados/operations/erasure-code-profile/.
4. ↑ "Pools". https://docs.ceph.com/en/latest/rados/operations/pools/.
5. ↑ Satyanarayanan, Mahadev; Kistler, James J.; Kumar, Puneet; Okasaki, Maria E.; Siegel, Ellen H.; Steere, David C.. Coda: A Highly Available File System for a Distributed Workstation Environment. https://www.csee.umbc.edu/courses/graduate/CMSC621/fall2006/lectures/coda.pdf.
6. ↑ "Erasure coding implementation". 2 November 2021. https://github.com/gluster/glusterfs/blob/master/doc/developer-guide/ec-implementation.md.
7. ↑ "Setting up GlusterFS Volumes". https://docs.gluster.org/en/latest/Administrator Guide/Setting Up Volumes/.
8. ↑ "MountableHDFS". https://cwiki.apache.org/confluence/display/HADOOP2/MountableHDFS.
9. ↑ "HDFS-7285 Erasure Coding Support inside HDFS". https://issues.apache.org/jira/browse/HDFS-7285.
10. ↑ "Apache Hadoop: setrep". https://hadoop.apache.org/docs/r2.4.1/hadoop-project-dist/hadoop-common/FileSystemShell.html#setrep.
11. ↑ Erasure coding plan: "Reed-Solomon layer over IPFS #196". https://github.com/ipfs/notes/issues/196. , "Erasure Coding Layer #6". https://github.com/ipfs/ipfs-cluster/issues/6.
12. ↑ "CLI Commands: ipfs bitswap wantlist". https://docs.ipfs.io/reference/cli/#ipfs-bitswap-wantlist.
13. ↑ "Why The Internet Needs IPFS Before It's Too Late". 4 October 2015. https://techcrunch.com/2015/10/04/why-the-internet-needs-ipfs-before-its-too-late/.
14. ↑ "Configuring Replication Modes". https://docs.lizardfs.com/adminguide/replication.html.
15. ↑ "Configuring Replication Modes: Set and show the goal of a file/directory". https://docs.lizardfs.com/adminguide/replication.html#set-and-show-the-goal-of-a-file-directory.
16. ↑ "Lustre Operations Manual: What a Lustre File System Is (and What It Isn't)". https://doc.lustre.org/lustre_manual.xhtml#understandinglustre.whatislustre.
17. ↑ Reed-Solomon in progress: "LU-10911 FLR2: Erasure coding". https://jira.whamcloud.com/browse/LU-10911.
18. ↑ "Lustre Operations Manual: Lustre Features". https://doc.lustre.org/lustre_manual.xhtml#idm139974537188976.
19. ↑ File-level redundancy plan: "File Level Redundancy Solution Architecture". https://wiki.lustre.org/File_Level_Redundancy_Solution_Architecture.
20. ↑ "MinIO Erasure Code Quickstart Guide". https://docs.min.io/docs/minio-erasure-code-quickstart-guide.html.
21. ↑ "MinIO Storage Class Quickstart Guide". https://github.com/minio/minio/tree/master/docs/erasure/storage-class.
22. ↑ Only available in the proprietary version 4.x "[feature erasure-coding #8"]. https://github.com/moosefs/moosefs/issues/8.
23. ↑ "mfsgoal(1)". https://fossies.org/linux/moosefs/mfsmanpages/mfsgoal.1.
24. ↑ "Replicating Volumes (Creating Read-only Volumes)". http://docs.openafs.org/AdminGuide/HDRWQ192.html.
25. ↑ "OpenAFS". https://www.openafs.org/release/openafs-1.0.html.
26. ↑ "OpenIO SDS Documentation". https://docs.openio.io/.
27. ↑ "Erasure Coding". https://docs.openio.io/latest/source/admin-guide/configuration_ec.html.
28. ↑ "Declare Storage Policies". https://docs.openio.io/latest/source/admin-guide/configuration_storagepolicies.html.
29. ↑ "Ori: A Secure Distributed File System". http://ori.scs.stanford.edu.
30. ↑ Mashtizadeh, Ali Jose; Bittau, Andrea; Huang, Yifeng Frank; Mazières, David. Replication, History, and Grafting in the Ori File System. http://sigops.org/s/conferences/sosp/2013/papers/p151-mashtizadeh.pdf.
31. ↑ "The Quantcast File System". https://www.cs.utah.edu/~hari/teaching/bigdata/qfs-ovsiannikov.pdf.
32. ↑ "qfs/src/cc/tools/cptoqfs_main.cc". 8 December 2021. https://github.com/quantcast/qfs/blob/2.2.2/src/cc/tools/cptoqfs_main.cc#L259.
33. ↑ "About RozoFS: Mojette Transform". http://rozofs.github.io/rozofs/develop/AboutRozoFS.html#mojette-transform.
34. ↑ "Setting up RozoFS: Exportd Configuration File". http://rozofs.github.io/rozofs/develop/SettingUpRozoFS.html#exportd-configuration-file.
35. ↑ "Initial commit.". https://github.com/rozofs/rozofs/commit/9818e92f73fe4432c8d29236158e271da9ee3bf2.
36. ↑ "Erasure Coding for warm storage". https://github.com/chrislusf/seaweedfs/wiki/Erasure-coding-for-warm-storage.
37. ↑ "Replication". https://github.com/chrislusf/seaweedfs/wiki/Replication.
38. ↑ 38.0 38.1 "Storj: A Decentralized Cloud Storage Network Framework v3.0". 30 October 2018. https://www.storj.io/storjv3.pdf.
39. ↑ "About Tahoe-LAFS". 24 February 2022. https://github.com/tahoe-lafs/tahoe-lafs/blob/master/README.rst#licence.
40. ↑ "zfec -- a fast C implementation of Reed-Solomon erasure coding". 24 February 2022. https://github.com/tahoe-lafs/zfec.
41. ↑ "Tahoe-LAFS Architecture: File Encoding". https://tahoe-lafs.readthedocs.io/en/latest/architecture.html#file-encoding.
42. ↑ "Under the Hood: File Replication". http://www.xtreemfs.org/how_replication_works.php.
43. ↑ "Quickstart: Replicate A File". http://www.xtreemfs.org/quickstart_repl.php.
44. ↑ "FRAUNHOFER FS (FhGFS) END USER LICENSE AGREEMENT". Fraunhofer Society. 2012-02-22. http://www.fhgfs.com/docs/FraunhoferFS_EULA.txt.
45. ↑ "ObjectiveFS official website". https://objectiveFS.com.
46. ↑ "The Infinit Storage Platform". http://infinit.sh.
47. ↑ "Infinit's Open Source Projects". 13 August 2019. http://infinit.sh/open-source.
48. ↑ "IBM Plans to Acquire Cleversafe for Object Storage in Cloud" (in en-US). 2015-10-05. https://www-03.ibm.com/press/us/en/pressrelease/47776.wss.
49. ↑ Séguin, Cyril; Depardon, Benjamin; Le Mahec, Gaël. "Analysis of Six Distributed File Systems". https://hal.archives-ouvertes.fr/file/index/docid/789086/filename/a_survey_of_dfs.pdf.
50. ↑ "Data Consistency Models of Public Cloud Storage Services: Amazon S3, Google Cloud Storage and Windows Azure Storage". 4 February 2014. https://www.systutorials.com/3551/data-consistency-models-of-public-cloud-storage-services-amazon-s3-google-cloud-storage-and-windows-azure-storage/.

* v
* t
* e

Storage virtualization

* Block-level storage
* Disk aggregation
* Distributed file systems
* Comparison
* File virtualization
* Logical disk
* RAID
* Software-defined storage
* Virtual disk
* Virtual file system

* v
* t
* e

File systems

* Comparison of file systems
* distributed
* Unix filesystem

Disk|

* ADFS
* AdvFS
* Amiga FFS
* Amiga OFS
* APFS
* AthFS
* bcachefs
* BFS
* Be File System
* Boot File System
* Btrfs
* CVFS
* CXFS
* DFS
* EFS
* Encrypting File System
* Extent File System
* Episode
* ext
* ext2
* ext3
* ext3cow
* ext4
* FFS/FFS2
* FAT
* exFAT
* Files-11
* Fossil
* HAMMER
* HAMMER2
* HFS
* HFS+
* HPFS
* HTFS
* IBM Spectrum Scale
* JFS
* LFS
* MFS
* Macintosh File System
* TiVo Media File System
* MINIX
* NetWare File System
* Next3
* NILFS
* NILFS2
* NSS
* NTFS
* OneFS
* PFS
* QFS
* QNX4FS
* ReFS
* ReiserFS
* Reiser4
* Reliance
* Reliance Nitro
* RFS
* SFS
* SNFS
* Soup (Apple)
* Tux3
* UBIFS
* UFS
* soft updates
* WAPBL
* VxFS
* WAFL
* Xiafs
* XFS
* Xsan
* zFS
* ZFS

Optical disc|

* HSF
* ISO 9660
* ISO 13490
* UDF

Flash memory and SSD|

* APFS
* FAT
* exFAT
* CHFS
* TFAT
* EROFS
* FFS2
* F2FS
* HPFS
* JFFS
* JFFS2
* JFS
* LogFS
* NILFS
* NILFS2
* NVFS
* YAFFS
* UBIFS

Distributed|

* CXFS
* GFS2
* Google File System
* OCFS2
* OrangeFS
* PVFS
* QFS
* Xsan
* more...

NAS|

* AFS (OpenAFS)
* AFP
* Coda
* DFS
* Google File System
* IBM Spectrum Scale
* Lustre
* NCP
* NFS
* POHMELFS
* Hadoop
* SMB (CIFS)
* SSHFS
* more...

Specialized|

* Aufs
* AXFS
* Boot File System
* CDfs
* Compact Disc File System
* cramfs
* Davfs2
* EROFS
* FTPFS
* FUSE
* Lnfs
* LTFS
* NOVA
* MVFS
* SquashFS
* UMSDOS
* OverlayFS
* UnionFS
* WBFS

Pseudo and virtual|

* configfs
* devfs
* debugfs
* kernfs
* procfs
* specfs
* sysfs
* tmpfs
* WinFS

Encrypted|

* eCryptfs
* EncFS
* EFS
* Rubberhose
* SSHFS
* ZFS

Types|

* Clustered
* Global
* Grid
* Self-certifying
* Flash
* Journaling
* Log-structured
* Object
* Record-oriented
* Semantic
* Steganographic
* Synthetic
* Versioning

Features|

* Case preservation
* Copy-on-write
* Data deduplication
* Data scrubbing
* Execute in place
* Extent
* File attribute
* Extended file attributes
* File change log
* Fork
* Links
* Hard
* Symbolic

Access control|

* Access control list
* Filesystem-level encryption
* Permissions
* Modes
* Sticky bit

Interfaces|

* File manager
* File system API
* Installable File System
* Virtual file system

Lists|

* Cryptographic
* Default
* Log-structured

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/Comparison of distributed file systems. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=Software:Comparison_of_distributed_file_systems&oldid=748797"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template