Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Why does redirecting the output of a file to itself produce a blank file?

Stated in Bash, why do

less foo.txt > foo.txt

and

fold foo.txt > foo.txt

produce an empty foo.txt? Since an append such as less eggs.py >> eggs.py produces a two copies of the text in eggs.py, one might expect that an overwrite would produce one copy of the text.

Note, I'm not saying this is a bug, it is more likely a pointer to something deep about Unix.

share|improve this question
add comment

2 Answers

When you use >, the file is opened in truncation mode so its contents are removed before the command attempts to read it.

When you use >>, the file is opened in append mode so the existing data is preserved. It is however still pretty risky to use the same file as input and output in this case. If the file is large enough not to fit the read input buffer size, its size might grow indefinitely until the file system is full (or your disk quota is reached).

Should you want to use a file both as input and output with a command that doesn't support in place modification, you can use a couple of workarounds:

  • Use an intermediary file and overwrite the original one when done and only if no error occurred while running the utility (this is the safest and more common way).

    fold foo.txt > fold.txt.$$ && mv fold.txt.$$ foo.txt
    
  • Avoid the intermediary file at the expense of a potential partial or complete data loss should an error or interruption happen. In this example, the contents of foo.txt are passed as input to a subshell (inside the parentheses) before the file is deleted. The previous inode stays alive as the subshell is keeping it open while reading data. The file written by the inner utility (here fold) while having the same name (foo.txt) points to a different inode because the old directory entry has been removed so technically, there are two different "files" with the same name during the process. When the subshell ends, the old inode is released and its data is lost. Beware to make sure you have enough space to temporarily store both the old file and the new one at the same time otherwise you'll lose data.

    (rm foo.txt; fold > foo.txt) < foo.txt
    
share|improve this answer
    
sponge from moreutils can also help. fold foo.txt | sponge foo.txt – or fold foo.txt | sponge !$ should also do. –  slhck May 19 '13 at 11:29
    
@slhck Indeed, sponge could do the job too. However, being neither specified by POSIX nor mainstream in Unix like OSes, it is unlikely to be present. –  jlliagre May 19 '13 at 20:17
    
It's not like it can't be made present though ;) –  slhck May 19 '13 at 21:44
add comment

The file is opened for writing by the shell before the application has a chance to read it. Opening the file for writing truncates it.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.