Thanks Chris Murphy for noting that btrfs can create and mount 8 EB (approx 263 byte) filesystems effortlessly:
$ nbdkit -fv memory size=$(( 2**63-1 ))
# modprobe nbd # nbd-client -b 512 localhost /dev/nbd0 # blockdev --getss /dev/nbd0 512 # gdisk /dev/nbd0 Number Start (sector) End (sector) Size Code Name 1 2048 18014398509481948 8.0 EiB 8300 Linux filesystem # mkfs.btrfs -K /dev/nbd0p1 btrfs-progs v4.16 See http://btrfs.wiki.kernel.org for more information. Detected a SSD, turning off metadata duplication. Mkfs with -m dup if you want to force metadata duplication. Label: (null) UUID: 770e5592-9055-4551-8416-b6376802a2ad Node size: 16384 Sector size: 4096 Filesystem size: 8.00EiB Block group profiles: Data: single 8.00MiB Metadata: single 8.00MiB System: single 4.00MiB SSD detected: yes Incompat features: extref, skinny-metadata Number of devices: 1 Devices: ID SIZE PATH 1 8.00EiB /dev/nbd0p1 # mount /dev/nbd0p1 /tmp/mnt # df -h /tmp/mnt Filesystem Size Used Avail Use% Mounted on /dev/nbd0p1 8.0E 17M 8.0E 1% /tmp/mnt
I created a few files in there and it all seemed to work although I didn’t do any extensive testing. Good job btrfs!
3 responses to “nbdkit for loopback pt 5: 8 exabyte btrfs filesystem”
You can also mount -o compress=zstd to get read-write mounted compressed file systems, just tested this. Soon we’ll have a compression level switch for zstd, like for zlib in recent kernels. Separately compressing with zstd (or xz) does do a better job because everything including fs metadata is compressed, whereas doing it with Btrfs compression support only compresses data extents. Also, Btrfs tends to give up easily and not compress some extents, unless using compress-force= option. But yeah, might be one way to gain some compression savings without adding extra layers.
Pingback: nbdkit for loopback pt 6: giant file-backed disks for testing | Richard WM Jones
Pingback: Compressed RAM disks | Richard WM Jones