NILFS

NILFSDeveloper(s)| Nippon Telegraph and Telephone Cyber Space Laboratories

|

Full name| New Implementation of a Log-structured File System
Introduced| 2005; 19 years ago (2005) with Linux kernel 2.6.13
Structures
File allocation| B-tree
Limits
Max. volume size| 8 EiB
Max. file size| 8 EiB
Max. filename length| 255 bytes
Features
Dates recorded| atime (planned)
Date resolution| 64-bit on-disk timestamps
Attributes| POSIX, with plans for extended attributes
File system permissions| POSIX, with plans for ACLs[1]
Transparent compression| No
Transparent encryption| No
Copy-on-write| Yes
Other
Supported operating systems| Linux kernel, (ReadOnly for NetBSD)

NILFS or NILFS2 (New Implementation of a Log-structured File System) is a log-structured file system implementation for the Linux kernel. It was developed by Nippon Telegraph and Telephone Corporation (NTT) CyberSpace Laboratories and a community from all over the world. NILFS was released under the terms of the GNU General Public License (GPL).

## Contents

* 1 Design
* 2 Features
* 2.1 Current status
* 2.2 Issues
* 2.2.1 Supported features
* 2.3 Additional features
* 3 Compatibility
* 4 Relative performance
* 5 License
* 6 Developers
* 7 See also
* 8 References
* 9 External links

## Design

"NILFS is a log-structured file system, in that the storage medium is treated like a circular buffer and new blocks are always written to the end.[…]Log-structured file systems are often used for flash media since they will naturally perform wear-leveling;[…]NILFS emphasizes snapshots. The log-structured approach is a specific form of copy-on-write behavior, so it naturally lends itself to the creation of file system snapshots. The NILFS developers talk about the creation of "continuous snapshots" which can be used to recover from user-initiated file system problems[…]."[2]

Using a copy-on-write technique known as "nothing in life is free", NILFS records all data in a continuous log-like format that is only appended to, never overwritten, an approach that is designed to reduce seek times, as well as minimize the kind of data loss that occurs after a crash with conventional file systems. For example, data loss occurs on ext3 file systems when the system crashes during a write operation. When the system reboots, the journal notes that the write did not complete, and any partial data writes are lost.

Some file systems, like UFS-derived file systems used by the Solaris operating system and BSDs, provide a snapshot feature that prevents[citation needed] such data loss, but the snapshot configuration can be lengthy on large file systems. NILFS, in contrast, can "continuously and automatically [save] instantaneous states of the file system without interrupting service", according to NTT Labs.[3]

The "instantaneous states" that NILFS continuously saves can actually be mounted, read-only, at the same time that the actual file system is mounted read-write — a capability useful for data recovery after hardware failures and other system crashes. The "lscp" (list checkpoint) command of an interactive NILFS "inspect" utility is first used to find the checkpoint's address, in this case "2048":

# inspect /dev/sda2
...
nilfs> listcp
1     6 Tue Jul 12 14:55:57 2005 MajorCP|LogiBegin|LogiEnd
2048  2352 Tue Jul 12 14:55:58 2005 MajorCP|LogiEnd
...
nilfs> quit

The checkpoint address is then used to mount the checkpoint:

# mount -t nilfs -r -o cp=2048 /dev/sda2 /nilfs-cp
# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/sda2             70332412   8044540  62283776  12% /nilfs
/dev/sda2             70332412   8044540  62283776  12% /nilfs-cp

## Features

NILFS provides continuous snapshotting. In addition to versioning capability of the entire file system, users can even restore files mistakenly overwritten or deleted at any recent time. Since NILFS can keep consistency like conventional LFS, it achieves quick recovery after system crashes.

Continuous snapshotting is not provided by most file systems, including those supporting point-in-time snapshotting (e.g. Btrfs)

NILFS creates a number of checkpoints every few seconds or per synchronous write basis (unless there is no change). Users can select significant versions among continuously created checkpoints, and can change them into snapshots which will be preserved until they are changed back to checkpoints.

There is no limit on the number of snapshots until the volume gets full. Each snapshot is mountable as a read-only file system. It is mountable concurrently with a writable mount and other snapshots, and this feature is convenient to make consistent backups during use.

Possible uses of NILFS include versioning, tamper detection, SOX compliance logging, data loss recovery.

The current major version of NILFS is version 2, which is referred to as NILFS2. NILFS2 implements online garbage collection to reclaim disk space with keeping multiple snapshots.

Other NILFS features include:

* B-tree based file and inode management.
* Immediate recovery after system crash.
* 64-bit data structures; support many files, large files and disks.
* 64-bit on-disk timestamps which are free of the year 2038 problem.

### Current status

### Issues

As of 2023, NILFS lacks in dedicated consistency checking utility (fsck), thus it can´t recover from severe errors that cause it to fail to find a valid checkpoint. [4] [5]

#### Supported features

* Basic POSIX file system features
* Snapshots
* Automatically and continuously taken
* No limit on the number of snapshots until the volume gets full
* Mountable as read-only file systems
* Mountable concurrently with the writable mount (convenient to make consistent backups during use)
* Quick listing
* Background Garbage Collection (GC)
* Can maintain multiple snapshots
* Selectable GC Policy, which is given by a userland daemon.
* Quick crash recovery on-mount
* Read-ahead for meta data files as well as data files
* Block sizes smaller than page size (e.g. 1KB or 2KB)
* Online resizing (since Linux-3.x and nilfs-utils 2.1)
* Related utilities (by contribution of Jiro SEKIBA)
* grub2
* util-linux (blkid, libblkid, uuid mount)
* udisks, palimpsest
* File system label (nilfs-tune)

### Additional features

* Fast write and recovery times
* Minimal damage to file data and system consistency on hardware failure
* 32-bit checksums (CRC32) on data and metadata for integrity assurance (per block group, in segment summary) [6][7]
* Correctly ordered data and meta-data writes
* Redundant superblock
* Internal data is processed in 64-bit wide word size
* Can create and store huge files (8 EiB)

## Compatibility

NILFS is available in various GNU/Linux distributions like Arch Linux,[8] Debian[9] (since version 5.0), Fedora, Gentoo,[10] Linux Mint, NixOS, Ubuntu (since version 9.10), etc. To use it, users typically need to install the nilfs-utils or nilfs-tools package. A boot-cd with NILFS is also available on PrRescue.

It is also supported by partition-editing application like GParted.[11]

A separate, BSD licensed implementation, currently with read-only support, is included in NetBSD.[12]

## Relative performance

In the January 2015 presentation SD cards and file systems for embedded systems at Linux.conf.au, it was stated:[13]

“  | If you've got a workload that's latency sensitive, you might want to use NILFS. If you've got one that's throughput sensitive, you might want to use F2FS.  | ”

|
|

— Peter Chubb
“  | NILFS2 works much better for the lots of small files case than F2FS or EXT4.  | ”

|
|

— Peter Chubb

## License

The NILFS2 file system utilities are made available under the GNU Public License version 2, with the exception of the lib/nilfs libraries and their header files, which are made available under the GNU Lesser General Public License Version 2.1.

## Developers

The Japanese primary authors and major contributors to the `nilfs-utils` who worked or are working at labs of NTT Corporation are:

* Ryusuke Konishi (Primary maintainer, 02/2008–Present)
* Koji Sato
* Naruhiko Kamimura
* Seiji Kihara
* Yoshiji Amagai
* Hisashi Hifumi and
* Satoshi Moriai.

Other major contributors are:

* Andreas Rohner [14]
* Dan McGee
* David Arendt
* David Smid
* dexen deVries
* Dmitry Smirnov
* Eric Sandeen
* Jiro SEKIBA
* Matteo Frigo
* Hitoshi Mitake
* Takashi Iwai
* Vyacheslav Dubeyko

## See also

* ZFS
* Btrfs
* F2FS, another log-structured file system implementation
* List of file systems
* Comparison of file systems
* Log-structured File System (BSD)
* Sprite operating system

## References

1. ↑ "NILFS Current Status". https://nilfs.sourceforge.io/en/current_status.html.
2. ↑ "BTRFS and NILFS [LWN.net]". https://lwn.net/Articles/238923/.
3. ↑ "An article about NILFS". http://www.linuxelectrons.com/news/linux/ntt-releases-nilfs-linux-file-system-prevention-data-loss.
4. ↑ Corrupted NILFS2 partition (/var) - Opinions on a better FS for /var?
5. ↑ Kernel archives: Can't mount nilfs - error searching super root.
6. ↑ the NILFS version 1: overview
7. ↑ does not verify: Re: Does nilfs2 checksum all data?; kernel 4.4.38-v7+ does not verify, too.
8. ↑ "Arch Linux - nilfs-utils 2.2.9-1 (X86_64)". https://archlinux.org/packages/core/x86_64/nilfs-utils/.
9. ↑ "Debian -- Details of package nilfs-tools in bullseye". https://packages.debian.org/bullseye/nilfs-tools.
10. ↑ "Sys-fs/Nilfs-utils – Gentoo Packages". https://packages.gentoo.org/packages/sys-fs/nilfs-utils.
11. ↑ "GParted -- Features". https://gparted.org/features.php.
12. ↑ NiLFS(2) source commit
13. ↑ Peter Chubb. "SD cards and filesystems for embedded systems". Linux.conf.au. http://mirror.linux.org.au/pub/linux.conf.au/2015/Case_Room_2/Friday/SD_Cards_and_filesystems_for_Embedded_Systems.webm.
14. ↑ https://www.complang.tuwien.ac.at/Diplomarbeiten/rohner18.pdf[bare URL PDF]

## External links

* "An Article Describing High-level Features". http://linuxdevices.com/news/NS9521569196.html.
* "World's Largest TELCO Joins Open Source Development Labs". http://www.linuxdevices.com/news/NS5705307352.html.
* NILFS: A File System to Make SSDs Scream[Usurped!]
* Manjaro tutorial NILFS: A filesystem designed to minimize the likelyhood [sic] of data loss

* v
* t
* e

File systems

* Comparison of file systems
* distributed
* Unix filesystem

Disk|

* ADFS
* AdvFS
* Amiga FFS
* Amiga OFS
* APFS
* AthFS
* bcachefs
* BFS
* Be File System
* Boot File System
* Btrfs
* CVFS
* CXFS
* DFS
* EFS
* Encrypting File System
* Extent File System
* Episode
* ext
* ext2
* ext3
* ext3cow
* ext4
* FFS/FFS2
* FAT
* exFAT
* Files-11
* Fossil
* HAMMER
* HAMMER2
* HFS
* HFS+
* HPFS
* HTFS
* IBM Spectrum Scale
* JFS
* LFS
* MFS
* Macintosh File System
* TiVo Media File System
* MINIX
* NetWare File System
* Next3
* NILFS
* NILFS2
* NSS
* NTFS
* OneFS
* PFS
* QFS
* QNX4FS
* ReFS
* ReiserFS
* Reiser4
* Reliance
* Reliance Nitro
* RFS
* SFS
* SNFS
* Soup (Apple)
* Tux3
* UBIFS
* UFS
* soft updates
* WAPBL
* VxFS
* WAFL
* Xiafs
* XFS
* Xsan
* zFS
* ZFS

Optical disc|

* HSF
* ISO 9660
* ISO 13490
* UDF

Flash memory and SSD|

* APFS
* FAT
* exFAT
* CHFS
* TFAT
* EROFS
* FFS2
* F2FS
* HPFS
* JFFS
* JFFS2
* JFS
* LogFS
* NILFS
* NILFS2
* NVFS
* YAFFS
* UBIFS

Distributed|

* CXFS
* GFS2
* Google File System
* OCFS2
* OrangeFS
* PVFS
* QFS
* Xsan
* more...

NAS|

* AFS (OpenAFS)
* AFP
* Coda
* DFS
* Google File System
* IBM Spectrum Scale
* Lustre
* NCP
* NFS
* POHMELFS
* Hadoop
* SMB (CIFS)
* SSHFS
* more...

Specialized|

* Aufs
* AXFS
* Boot File System
* CDfs
* Compact Disc File System
* cramfs
* Davfs2
* EROFS
* FTPFS
* FUSE
* Lnfs
* LTFS
* NOVA
* MVFS
* SquashFS
* UMSDOS
* OverlayFS
* UnionFS
* WBFS

Pseudo and virtual|

* configfs
* devfs
* debugfs
* kernfs
* procfs
* specfs
* sysfs
* tmpfs
* WinFS

Encrypted|

* eCryptfs
* EncFS
* EFS
* Rubberhose
* SSHFS
* ZFS

Types|

* Clustered
* Global
* Grid
* Self-certifying
* Flash
* Journaling
* Log-structured
* Object
* Record-oriented
* Semantic
* Steganographic
* Synthetic
* Versioning

Features|

* Case preservation
* Copy-on-write
* Data deduplication
* Data scrubbing
* Execute in place
* Extent
* File attribute
* Extended file attributes
* File change log
* Fork
* Links
* Hard
* Symbolic

Access control|

* Access control list
* Filesystem-level encryption
* Permissions
* Modes
* Sticky bit

Interfaces|

* File manager
* File system API
* Installable File System
* Virtual file system

Lists|

* Cryptographic
* Default
* Log-structured

0.00

(0 votes)

Original source: https://en.wikipedia.org/wiki/NILFS. Read more

|

Retrieved from "https://handwiki.org/wiki/index.php?title=NILFS&oldid=3148783"
*[v]: View this template
*[t]: Discuss this template
*[e]: Edit this template