67

I have two Dart applications I need to dockerize. These two apps use a shared source directory.
Because Docker prevents adding files from folders outside the context directory (project/app1) I can't add files from ../shared nor from shared (the symlink inside projects/app1).

I'm looking for a way to trick Docker to do it anyway.

My simplified project structure 

- projects
  - app1
   - Dockerfile
   - shared (symlink ../shared)
   - otherSource
  - app2
   - Dockerfile
   - shared (symlink ../shared)
   - otherSource
  - shared
    - source

I could move Dockerfile one level up and run docker build from there but then I need two Dockerfiles (for app1 and app2) in the same directory.

My current idea was, if I could somehow hide the fact that projects/app1/shared is a symlink this problem would be solved. I checked if I can share projects using Samba and remount it somewhere else and configure Samba to treat symlinks like normal folders but haven't found whether this is supported (I have not much experience with Samba and not tried it yet, just searched a bit).

Is there any other tool or trick that would allow that?

I would rather not change the directory structure because this would cause other troubles and also rather not copy files around.

Improve this question
0

3 Answers 3

Reset to default
51

I don't have much experience with docker so I can't promise this will work but one choice would be to mount the directory instead of linking to it:

$ cd projects/app1
$ mkdir shared
$ sudo mount -o bind ../shared shared/

That will attach ../shared to ./shared and should be completely transparent to the system. As explained in man mount:

The bind mounts.

Since Linux 2.4.0 it is possible to remount part of the file hierarchy somewhere else. The call is:

mount --bind olddir newdir

or by using this fstab entry:

/olddir /newdir none bind

After this call the same contents are accessible in two places.

Improve this answer
7
  • 1
    @zoechi this is perfectly on topic on both sites. As a general rule, I would post more technical questions like this on U&L and more user-space questions here. The choice is completely up to you though. On the one hand, there are more users here so more eyeballs, on the other, there is a much higher concentration of professional *nix people on U&L. Just make sure you don't post the same question on both sites. If you want to move it, either delete this or flag for mod attention and ask them to migrate.
    – terdon
    Nov 20, 2014 at 13:05
  • 3
    It was mandatory for me to restart docker daemon! Else the mounted dir was not visible in the container.
    – dim
    Jan 22, 2016 at 11:25
  • @dim yes! I tried to get it to work with Capistrano and it didn't work – turns out I mounted the shared directories after I started the container
    – csch
    Nov 7, 2016 at 14:18
  • 1
    Unfortunately this won't work for Windows or OS X users. The debate about this issue has been... lively.
    – Jason
    Feb 24, 2017 at 1:34
  • is mounting 'commited' to source control eg github? or would I have to do it every time?
    – Adam Pietrasiak
    Mar 14, 2019 at 11:22
60

This issue has come up repeatedly in the Docker community. It basically violates the requirement that a Dockerfile be repeatable if you run it or I run it. So I wouldn't expect this ability, as described in this ticket: Dockerfile ADD command does not follow symlinks on host #1676.

So you have to conceive of a different approach. If you look at this issue: ADD to support symlinks in the argument #6094, a friend of ours from U&L (@Patrick aka. phemmer) provides a clever workaround.

$ tar -czh . | docker build -

This tells tar to dereference the symbolic links from the current directory, and then pipe them all to the docker build - command.

excerpt from tar man page 
-c, --create
       create a new archive

-h, --dereference
       follow symlinks; archive and dump the files they point to

-z, --gzip, --gunzip --ungzip
Improve this answer
12
  • 9
    This is an EXCELLENT solution! I understand why Docker claims they want to omit this feature. However, there is a considerable difference in the workflow I use while developing my containerize project and how I expect it to be built for production. On my local machine I want a super tight feedback loop. My app has 1 git repo and the build environment for the containers has a 2nd repo. I need to be able to make edits and build tests locally before I can decide if I want to commit and push. I won't have symlinks or ADD instructions in my final project.
    – Bruno Bronosky
    Mar 16, 2015 at 18:44
  • 27
    Dockerfiles are not repeatable. Dockerfiles can not possibly be made repeatable, because they almost all have apt-get or something equivalent at the 2nd or 3rd layer and apt-get is not repeatable. Tying the Docker development strategy to a misguided attempt to make the impossible true will just saddle Docker with a series of bad abstractions that help nobody. nathanleclaire.com/blog/2014/09/29/…
    – Jason
    Feb 24, 2017 at 1:12
  • 2
    For the newbs, can you please explain in English what is going on here, linking to the tar man page is nice but
    – Alexander Mills
    Jul 14, 2017 at 21:02
  • 1
    Ok, so I don't really see why this is any better than a cp command, can you explain why it's better? I also think the pipe is confusing/overly convoluted. Why not just put the tar command above the build command. I guess because then you would overwrite the symlinked dir with the real dir.
    – Alexander Mills
    Jul 14, 2017 at 21:13
  • 2
    Can someone, anyone link to a full example that uses this? I tried running tar -czh . | docker build - in place of my regular docker build -t my-image-name ., but I get the error (on Windows): tar: Failed to clean up compressor failed to get console mode for stdin: The handle is invalid. [...] failed to solve with frontend dockerfile.v0: failed to create LLB definition: the Dockerfile cannot be empty
    – Venryx
    Aug 7, 2021 at 15:19
0

I got frustrated enough that I made a small NodeJS utility to help with this: file-syncer

Basic usage:

// starts live-sync of files under "common_files" to "my-app/HardLinked/common_files"
npx file-syncer --from common_files --to my-app/HardLinked

More info on this Stackoverflow answer, and the Github repo.

Improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .