Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Git stores content uniquely in its repo based on the calculated hash of any file. If my directory has two copies of the same file somewhere inside it, git will only actually store it once.

I am wondering if this same concept has been implemented at the operating-system level as some kind of file system?

If a file system acted this way by default it would nicely help with dll hell issues. Essentially, it would symlink automatically on your behalf. Any application could be packaged (like a jar) in a directory with all of its dependencies and no extra storage cost.

Ruby enthusiasts share libraries by publishing them as rubygems. Still, this effort to share gems resulted in deployment nightmares that lead to the Vendor Everything concept of copying all dependencies into local folders to avoid such nightmares.

share|improve this question
I'm not an expert, but check out ZFS. – ForeverWintr Aug 9 '13 at 16:31
up vote 19 down vote accepted

What you're looking for is called "deduplication". While it's usually implemented by vendors of specialized storage products, the ZFS filesystem implements it as well. Most Unix-derived operating systems can make use of ZFS, and I'd therefore recommend it as the first place to look.

share|improve this answer
I see "deduplication" can be implemented at the file-level which is what I was concerned with in particular. – Mario Aug 9 '13 at 16:45

Network Appliance, Inc. has had storage ability like this for many years, in fact they had complaints filed with Sun Microsystems for their ZFS file system, and does what Aaron Miller is mentioning in his accepted answer. For what it's worth, the complaints were settled with Oracle after about 3 years.

I have used this as a corporate solution since 2000, and it works well. The cost of storage after the first 'copy' comes about once there is a change in the file. Otherwise, with many 'copies' comes only a slight increase for namespace.

I don't think this answers the question about "operating system level" but rather on a "file system level".

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.