Ridiculously big “files”

In the last post I showed how you can combine nbdfuse with nbdkit’s RAM disk to mount a RAM disk as a local file. In a talk I gave at FOSDEM last year I described creating these absurdly large RAM-backed filesystems and you can do the same thing now to create ridiculously big “files”. Here’s a 7 exabyte file:

$ touch /var/tmp/disk.img
$ nbdfuse /var/tmp/disk.img --command nbdkit -s memory 7E &
$ ll /var/tmp/disk.img 
 -rw-rw-rw-. 1 rjones rjones 8070450532247928832 Nov  4 13:37 /var/tmp/disk.img
$ ls -lh /var/tmp/disk.img 
 -rw-rw-rw-. 1 rjones rjones 7.0E Nov  4 13:37 /var/tmp/disk.img

What can you actually do with this file, and more importantly does anything break? As in the talk, creating a Btrfs filesystem boringly just works. mkfs.ext4 spins using 100% of CPU. I let it go for 15 minutes but it seemed no closer to either succeeding or crashing. Update: As Ted pointed out in the comments, it’s likely I didn’t mean mkfs.ext4 here, which gives an appropriate error, but mkfs.xfs which consumes more and more space, appearing to spin.

Emacs said:

File disk.img is large (7 EiB), really open? (y)es or (n)o or (l)iterally

and I was too chicken to find out what it would do if I really opened it.

I do wonder if there’s a DoS attack here if I leave this seemingly massive regular file lying around in a public directory.


Filed under Uncategorized

3 responses to “Ridiculously big “files”

  1. Fuse filesystems already seem pretty ripe for DoS attacks … for example they can return bytes SUPER slowly on read. An absurdly large file might actually end up causing even bigger issues though (malloc size overflow for example; an unchecked error could lead to big problems!)

    • rich

      One thing I didn’t mention in the article is that you can create impossible file modes. These cause I/O errors when accessed:

      $ ls -l /var/tmp | grep disk.img
      ls: cannot access '/var/tmp/disk.img': Input/output error
      -??????????  ? ?      ?                ?            ? disk.img

      And note that I didn’t need to use root at any point here.

  2. Theodore Ts'o

    I wasn’t able to replicate the problem, although I wasn’t using nbdfuse since it’s not available on Debian. So instead I created a scratch XFS file system (since it supports sparse files larger than 16TB), and then created a 7 EiB sparse, and accessed it both directly and using a loop device:


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.