Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I am trying to make a large file system in linux, but have run into problems with many of the common file systems.

  • JFS has a bug that does not allow expansion over 32TB.
  • XFS has a bug in the fsck that causes the machine to use all available memory and crash when running the fsck on a disk that has a large amount of data (~20TB).
  • EXT4 is limited to 16TB due to a problem with e2fsprogs.
  • BTRFS will be nice, but they do not currently have a fsck, which I will need.

Any other ideas?

share|improve this question
    
Why would you "need" fsck ? ZFS has no fsck and doesn't need it by design. – jlliagre Apr 5 '11 at 16:10
    
How much space are we talking here? – tjameson Apr 5 '11 at 17:23
    
We have 200TB of disks. We are constantly adding data, but right now we have around 30TB of data. If fsck is designed out, that is fine. For BTRFS it list on their wiki that you may not be able to recover from a crash or losing power. We are going to crash and lose power occasionally, so we need something that can handle this. I imagine ZFS would work, and will consider it. There is the drawback that it is only on fuse though. – bjcubsfan Apr 5 '11 at 18:05
up vote 4 down vote accepted

It may not be as fast as the others, being only a userland fuse based system in Linux, but ZFS may fit the bill...

The name originally stood for "Zettabyte File System". The original name selectors happened to like the name, and a ZFS file system has the ability to store 258 zettabytes, where each ZB is 270 bytes.

ZFS is a 128-bit file system, so it can address 1.84 × 1019 times more data than 64-bit systems such as NTFS. The limitations of ZFS are designed to be so large that they would never be encountered. Some theoretical limits in ZFS are:

  • 248 — Number of entries in any individual directory
  • 16 exabytes (16×1018 bytes) — Maximum size of a single file
  • 16 exabytes — Maximum size of any attribute
  • 256 zettabytes (278 bytes) — Maximum size of any zpool
  • 256 — Number of attributes of a file (actually constrained to 248 for the number of files in a ZFS file system)
  • 264 — Number of devices in any zpool
  • 264 — Number of zpools in a system
  • 264 — Number of file systems in a zpool

There are some who say that there aren't enough atoms in the earth crust to make a file storage array big enough to exceed the limitations of ZFS.

share|improve this answer

ZFS is a possible solution, but nowadays one could also use filesystems that you excluded in the beginning

  • ext4: petabyte filesystemd should not be an issue in recent e2fsprog and kernel versions
  • btrfs: has working btfsck and can also scale to large sizes.
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .