UnionFS-like filesystem for windows

Question

I'm looking for something that works like UnionFS in Windows 7. My usecase: having clean code working directory in a read-only directory, and compiling it in a union filesystem which would store compiled binaries logically in the same place, but physically in different directory.

Libraries do not work this way, even though they were accepted as an answer in UnionFS alike on windows, Virtually merge folders -- they only show top-level directories in one place.

The code itself is huge (2GB of data), and I'd like to avoid copying it for every build in my build server.

Answer 1

It's possible nowadays but fairly complicated using rclone union mount with local directory "remotes".

rclone union allows mergerfs style overlays including policies for where the new files get created etc.

rclone mount allows you to mount any kind of rclone locally. It' based on WinFSP, which is the FUSE equivalent for Windows. Performance is not going to be amazing...

Answer 2

There is a feature Unified Write Filter (UWF) overlay, but I don't know how to use it.

Answer 3

You can make a poor-man's unionfs with symbolic links: reproduce the directory structure of the source, but make links (symbolic or hard) for the files. This should be suitable for your purpose as long as the build system doesn't try to modify source files.

With GNU cp, it's as easy as cp -al or cp -as. But I don't know if there is a port of GNU tools that understands Windows links.

If you need to call the mklink command, it can be done with POSIX tools (warning, typed directly into the browser):

cd SOURCE
find -type d -exec sh -c 'cd BUILDDIR && mkdir "$@"' _ {} +
find -type f -exec sh -c 'mklink "$1" "BUILDDIR/$1"' _ {} \;

I suppose there is a native Windows way involving Powershell, but I have no idea what it looks like. It might be worth investigating if your directory tree is large and your disks are fast because forking under Windows tends to be slow, and the commands above need to fork a lot.

Answer 4

Could you use symbolic links to fake this? http://en.wikipedia.org/wiki/NTFS_symbolic_link It's not answering the exact question but I think it will solve the use case you've given.

Answer 5

I found this, but I don't really know how well it works:

• winunionfs

Answer 6

While there is no OS support for union mounts in Windows yet, I think it's worth a try to implement a solution on the basis of VHD images. The idea is to:

1. Create one base VHD image with common files
2. Create a differential VHD linked to the base image
3. Mount the differential VHD
4. Directory junction from your original file system point to the directory on the differential VHD
5. Goto 2 for as many new projects with common files

I haven't tried this myself, but the only problem I see is that to use many differential images derived from the same base image in parallel you'd somehow need for them to all have different drive ids, otherwise windows disk manager might refuse to mount.