47

I found something in a script, but not belonging to the main script. There was :> in a line.

Could you explain to me what it means?

:> file
while read A B C D E; do echo "$A;$B;$D;$E;$C" >> file; done < otherfile
  • 6
    Importantly, :> is not a single operator. It may be easier to understand if you read it as : > file instead. – jpfx1342 Sep 1 '15 at 18:01
  • It means that the person writing the script should have redirected the output of the loop to the file: while read A B C D E; do echo "$A;$B;$D;$E;$C"; done < otherfile > file. Or better yet, they should have used the right tool for the job, awk, as suggested by Peter. As an aside, you almost always want to use the -r switch with read. – Tom Fenech Sep 2 '15 at 15:06
  • Outside bash, it would be a smiley for a crow. – smci Sep 2 '15 at 23:05
46

There was :> in a line of a bash script. What does it mean?

:> file

It is a short cut way of saying:

  • If file does not exist then create it else truncate it to 0 bytes.

This means you can be sure that file exists and it is empty.

You can also use > file but :> file is more portable.

See the Stack Overflow question What Is the Purpose of the ':' (colon) GNU Bash Builtin? for more information.

  • I don't understand second line. I thought, that read read variables. Command echo is strange too. Could you explain? – diego9403 Sep 1 '15 at 15:52
  • I'm not a Unix expert but I think the second line reads things from otherfile and echos them out to file. It also makes variables from whatever it reads... If you want a definite answer please ask your own question. – DavidPostill Sep 1 '15 at 15:56
  • 2
    @diego9403: read gets input from stdin. On its own, it would read what you type. Since stdin have been redirected to <otherfile then the contents of otherfile is "typed" into stdin. So read gets the values line-by-line into the variables $A, $B, $C, $D and $E. – slebetman Sep 1 '15 at 17:02
  • So it is just a more obscure alternative to truncate from coreutils? – Federico Poloni Sep 1 '15 at 17:37
  • 1
    @PeterCordes I didn't mean "obscure" as in "it's uncommon", but as in "it's less clear to the reader". – Federico Poloni Sep 2 '15 at 10:25
29

It looks like a fancy way of creating a new file. In bash : is a null command:

$ type : 
: is a shell builtin 
$ help : 
:: :
    Null command.

    No effect; the command does nothing.

    Exit Status:
    Always succeeds.

> redirects output of : to a file.

  • 2
    It will also truncate the file if it already exists ... – DavidPostill Sep 1 '15 at 15:46
  • 2
    yes, this is what > does – Arkadiusz Drabczyk Sep 1 '15 at 15:48
  • 2
    : is shorthand for true. Possibly in in some shells, true isn't a builtin? Both are built-ins in bash. – Peter Cordes Sep 2 '15 at 10:20
12

: is another name for true. Both are shell builtins in bash, but there's no /bin/:, only a /bin/true. Output redirection causes the shell to open(2) the file with O_CREAT|O_TRUNC. If nothing is written, it stays at zero length.

Putting those two pieces together, :> file is a fairly common idiom for truncating files. Most people would try to make it less weird-looking by writing : >file, though.


Since you asked in a comment about the 2nd line, I'll turn my comments into an answer. (even though you didn't ask this in your question.)

The 2nd line is a loop that reads lines from otherfile into some named variables. The loop body uses echo to print them with ; separators instead of whatever whitespace they had before. file is closed and re-opened (for append) each iteration, because the redirect is inside the loop. Using while ...;do read -r ...;done <otherfile >file would suck less, and avoid the need to truncate file first. read -r doesn't eat \ as an escape character.

Text processing in bash is quite slow. Part of that is unavoidable: read has to go one byte at a time (one read(2) system call per byte) to avoid overshooting the end of a line. It would be better to use the right tool for the job:

awk -vOFS=';' '{ print $1, $2, $4, $5, $3 }' -- otherfile  >file

-- means your script doesn't break if otherfile is named something silly like --version.

Setting the Output Field Separator to ; means you can just pass multiple fields as args to print. Shell read assigns the whole rest of the line with whitespace to the last variable, but there's no way to tell awk to only split into 5. If that's important, maybe just keep using a bash loop, because it's inconvenient in awk. Perl makes this easy, since its split can take a max-fields arg, but it's a lot slower to start up than awk.

Actually, it turned out to be not that hard, just an ugly regex to write. To get rest-of-the-line instead of $5 in awk, looping over fields still loses their original whitespace. My first viable idea is to use gensub on $0 (the whole line) to remove the first 4 fields (i.e. non-space followed by space), leaving everything else:

awk -vOFS=';' '{ tail = gensub("[[:space:]]*([^[:space:]]+[[:space:]]+){4}", "", 1); print $1, $2, $4, tail, $3 }' -- otherfile >file

I got it right on the first try, but the fact that I was impressed with myself for that says something about the readability of that awk code. >.<

Note how it's the same print as before, but with tail in place of $5.

echo 'A  B c DD    e      f g    f' | 
  awk -vOFS=\; '{ tail = gensub("[[:space:]]*([^[:space:]]+[[:space:]]+){4}", "", 1);
   print $1, $2, $4, tail, $3 }'

A;B;DD;e       f g    f;c

This would be more impressive if I could copy/paste the literal and show that it came through in the output. Type one in bash with ^Q. ctrl-Q means Quote the next keypress as a literal character, since bash's emacs-style line editing is the same as actual emacs for this.

http://mywiki.wooledge.org/BashFAQ has some useful stuff about scripting in ways that won't break no matter what data or filenames you throw at the script.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.