55

As far as I can see rsync doesn't create the file in the target directory until it is complete.

This must mean that it creates the file in a temp directory somewhere and copies the file into the target directory when it is complete.

First off, is this correct?

If true, is it possible for rsync to set not use a temp directory and instead create the file in the target directory and just keep writing to it until it's complete?

3 Answers 3

69

Yes, it does create a temp file.

According to the man page you can specify the directory in which the temp file is stored with the parameter -T as in

-T, --temp-dir=DIR create temporary files in directory DIR

The answer to the second part of your question can also be found there:

--inplace

This option changes how rsync transfers a file when the file's data needs to be updated: instead of the default method of creating a new copy of the file and moving it into place when it is complete, rsync instead writes the updated data directly to the destination file.

1
  • 10
    Important note! The question assumes rsync "copies the file into place", whereas the rsync documentation specifies that rsync is actually "moving it into place", which is an IMORTANT DISTINCTION. This ensures that the file appears in the destination location in an atomic fashion. This is useful for preventing race conditions where another application is waiting for the destination file to appear and immediately opens it for reading, but the file is not done being copied. Jan 12, 2017 at 16:08
18

It does create a temp file, by default in the target directory and named .<FILE_NAME>.<RANDOM_STRING>. So, if you are copying foo.txt, it will create a tmp file called .foo.txt.GV4H3 (GV4H3 is the random string that will be different every time you run it). You can control this behavior using these rsync options:

   --partial
          By default, rsync will delete any partially transferred
          file if the transfer is interrupted.  In  some  circum‐
          stances  it  is more desirable to keep partially trans‐
          ferred files. Using the --partial option tells rsync to
          keep  the  partial  file which should make a subsequent
          transfer of the rest of the file much faster.

   --partial-dir=DIR
          A better way to keep partial files than  the  --partial
          option  is  to  specify a DIR that will be used to hold
          the partial data (instead of writing it out to the des‐
          tination file).  On the next transfer, rsync will use a
          file found in this dir as data to speed up the  resump‐
          tion  of  the  transfer and then delete it after it has
          served its purpose.

Please read the relevant parts of the rsync man page (the following is only a small extract of the large section on how to use --partial-dir).

3
  • You can't "control" the location or name of the temp file with the partial options, so I would say this answer is wrong. The --partial-dir option for example causes to move the partial file to a different directory. But this happens ONLY and AFTER the transfer got interrupted. So usual transfers still create the temp file directly in the target directory.
    – mgutt
    Dec 18, 2023 at 21:20
  • @mgutt I don't understand. The question is asking whether rsync creates a temp file and if it is possible to create one in the target directory. My answer explains that yes, it does create one, and it puts it in the target dir by default. It also gives options for where the temp file can be stored. Perhaps you misread the question?
    – terdon
    Dec 18, 2023 at 23:07
  • Your first sentence describes that rsync creates a temp file with a specific name in the target directory, and yes this answers the question, but then you say, that this behavior can be controlled with the partial options and I think this is false as only temp-dir allows influencing the temp file location. partial even does the opposite of which I would define as a temp file: It renames the temporay filename to the final destination filename (which even is not clearly explained in the manual).
    – mgutt
    Dec 19, 2023 at 0:28
5

rsync creates temporary files in the target directory. That file is named like the source but with an extension. It could be like that:

source: foo.bar 
target temp: foo.bar.hzmkjt7 (hzmkjt7 is just an example) 

The extension is removed after the file has been verified to be an exact copy of the source. During the renaming process ownership, permissions and modification time are set. So it is possible that you don't see the file because of the permissions. Make sure that you have enough permission to see all the files as well those not owned by you in order to see the temp files.

3
  • A process creates files with the permissions of the user who runs it. It is not possible to not see the files because of permissions in this case. Apr 6, 2013 at 22:00
  • @ЯрославРахматуллин rsync can as well be run in daemon mode. Or it can be scheduled as a cronjob. Either way it is possible that the user under whom rsync is running is not the user who wants to see the temp file. In that case it is possible that it is a permission issue.
    – Simon
    Apr 7, 2013 at 12:51
  • 1
    The temp file might start with ".". In this case, it is .foo.bar.hzmkjt7.
    – interskh
    Jun 28, 2013 at 20:35

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .