Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I wrote the following script to interactively and recursively remove orphan backup files, i.e. remove each file.txt~ that does not have a corresponding file.txt.

#!/bin/sh -x

set -o errexit
unalias -a

backups=$(find . -name "*~")

orphans=""
while read -r file
do
    [ ! -e "${file%~}" ] && orphans=$(echo "$file\n$orphans");
done << EOF
$backups
EOF

if [ -z "$orphans" ]; then
    echo "No orphans."
else
    echo "orphans:\n$orphans"
    echo "$orphans" | xargs --interactive -d '\n' rm
fi

This script does very weird things in a random fashion. Sometimes it behaves correctly, sometimes it ignores the -x options passed to sh, sometimes it executes commented code, sometimes the tests give just wrong results.

The problem seems to be related to the here script, since by redirecting the output of find to a temporary file all problems seem to disappear. But why, where is the error?


Solution (thanks to Dennis Williamson): Escape the ~ character. The unescaped ~ in the parameter expansion ${file%~} was creating somehow all the unpredictable behavior.


A more readable and deterministic solution, with some fat cut out, could be (thanks to the suggestions of Mikel):

#!/bin/sh
IFS='
'
for backup in $(find . -type f -name "*~"); do
    if [ ! -e "${backup%\~}" ]; then
        rm -i "$backup"
    fi
done

If you are a while read loop fan, things get less elegant because the interactive command rm -i can not be used (it will conflict with the read command). Anyway, a solution could be:

#!/bin/sh
orphans=""
while read -r backup; do
    if [ ! -e "${backup%\~}" ]; then
        orphans=$(echo "$backup\n$orphans");
fi
done << EOF
$(find . -type f -name "*~")
EOF

if [ ! -z "$orphans" ]; then
    echo "$orphans" | xargs --interactive -d '\n' rm
fi

Or, a more complex way is also suggested by Dennis Williamson.

share|improve this question

2 Answers 2

up vote 2 down vote accepted

You probably need to escape the tilde in the brace expansion, otherwise it will get expanded to your home directory.

Why don't you pipe the find into your while loop instead of creating variables to hold them? Inside your loop, just do rm -i "$file".

#!/bin/sh -x

set -o errexit
unalias -a

exec 3<&0    # open a duplicate of stdin
flag=false
find . -name "*~" | while IFS=$'\n' read -r file
do
    if [ ! -e "${file%\~}" ]
    then
        orphans="$file"$'\n'"$orphans"

        # use an alternate file descriptor so read and rm -i get along
        rm -i "$file" <&3
        flag=true
    fi
done
exec 3<&-    # close the file descriptor

if ! $flag
then
    echo "No orphans."
else
    echo "orphans:\n$orphans"
fi

If you want to use Bash, you need to move the find to the end of the loop so a subshell is not created.

#!/bin/bash

...

# use an alternate file descriptor so read and rm -i get along
while read -u 3 -r ...

    rm -i ...
    ...

done 3< <(find ...)

...
share|improve this answer
    
+1. Tilde definitely needs escaping. Also agree there would be simpler ways to write this. –  Mikel Jan 18 '11 at 22:08
    
Thanks! The problem was in the escaping of ~. Now everything works in a comfortable deterministic way. BTW, your script does not work with interactive rm, I suppose because the read command will conflict with it. –  mrucci Jan 18 '11 at 23:18
    
I am using bash 4.1.5(1)-release and the sub-shell is still created by the pipe. So the redirection of the find output into the while command must be done. You should also delete the leftover $ in the orphans concatenation line. –  mrucci Jan 18 '11 at 23:30
    
@mrucci: That dollar sign is to create the newline: $'\n' –  Dennis Williamson Jan 18 '11 at 23:34
    
Yes, I'm guessing rm -i reads from stdin, which is supplied by the heredoc, rather than from /dev/tty. –  Mikel Jan 18 '11 at 23:42

What do you mean "sometimes"? Sometimes on the same box? Or different behavior on different systems?

What do you mean "executes commented code"? Your example doesn't have any comments.

Some thoughts:

  • try set -x in place of /bin/sh -x
  • better to use set -e than set -o errexit
  • if you're using bash, then call /bin/bash, not /bin/sh, which could be something else
  • echo is unnecessary, just use = with a literal newline
  • if you do have to use echo, you should use echo -e or ensure xpg_echo is set
  • your orphans line is putting them in reverse order, is that deliberate?
  • your read loop will fail if file names contain spaces, you should set IFS first

A simpler version:

#!/bin/bash

set -x
set -e

IFS=$'\n'
orphans=false
for backup in $(find . -type f -name "*~"); do
    original=${backup%\~}
    if [ ! -e "$original" ]; then
        orphans=true
        rm -i "$backup"
    fi  
done

if ! $orphans; then
    echo "No orphans."
fi

Or if you want it to work using /bin/sh:

#!/bin/sh

set -x
set -e

IFS='
'
orphans=false
for backup in $(find . -type f -name "*~"); do
    original=${backup%\~}
    if [ ! -e "$original" ]; then
        orphans=true
        rm -i "$backup"
    fi  
done

if ! $orphans; then
    echo "No orphans."
fi
share|improve this answer
    
Why are your set commands better? It's more readable to use $'\n' (or echo) than a literal newline. In some shells echo` doesn't understand -e (they echo it literally) and do the escape processing by default. That's why it's often better to use printf since it is consistent. –  Dennis Williamson Jan 18 '11 at 22:23
    
He said he was using bash. This allows me to say use echo -e. Otherwise you really need printf for consistent behavior on all platforms, but that's an external command in bash, so echo -e is correct and faster in this case. –  Mikel Jan 18 '11 at 22:28
    
I said use set -x because he was using the -x option. For consistency and because I think the set -<flag> form is more common than set -o <optname>. –  Mikel Jan 18 '11 at 22:29
    
Thanks! I didn't know I could use find inside the for loop that easily. Good point about the unnecessary echo, I was totally in an "echo frenzy" state. The loop will not fail when files have spaces in them. Your solution does not work with /bin/sh, but only with bash but I can't see why. –  mrucci Jan 18 '11 at 23:23
    
If you're not using bash, you have to replace IFS=$'\n' with IFS='<ENTER>'. –  Mikel Jan 18 '11 at 23:35

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.