When I pipe multiple unix commands such as grep, sed, tr etc. I tend to specify the input file that is being processed using cat. So something like cat file | grep ... | awk ... | sed ... .

But recently after a couple of comments left on my answers indicating that this was a useless use of cat, I thought I would ask the question here.

I looked up the issue and came across Wikipedia's article on UUOC and The Useless Use of Cat Award and it seems to me that the arguments made are from the perspective of efficiency.

The closest question I came across here was this one: Is it wasteful to call cat? – but it's not quite what I'm asking.

I guess what the UUOC camp suggest is to use cmd1 args < file | cmd2 args | cmd3 .. or if the command has an option to read from file then to pass in the file as an argument.

But to me cat file | cmd1 ... | cmd2 seems much easier to read and understand. I don't have to remember different ways of sending input files to different commands, and the process flows logically from left to right. First input, then the first process ... and so on.

Am I failing to understand what arguments are being made about the useless use of cat? I understand that if I'm running a cron job that runs every 2 seconds that does a lot of processing, then in that case cat might be wasteful. But otherwise what's the general consensus on the use of cat?

  • 11
    I agree, here, the call to cat may be inefficient, but it makes the command much easier to understand and edit later, and (importantly, IMO) seperates each different command to having just one job, making the whole thing much easier to deal with. – Phoshi Aug 14 '11 at 18:20
  • 2
    The general consensus is that there is no consensus. – jwg Sep 3 '15 at 13:22
up vote 19 down vote accepted

It's useless in the sense that using it like that doesn't accomplish anything the other, possibly more efficient options can't (i.e. producing proper results).

But cat is way more powerful than just cat somefile. Consult man cat or read what I wrote in this answer. But if you absolutely positively only need the contents of a single file, you might get some performance advantage from not using cat to get at the file contents.

Regarding readability, this depends on your personal tastes. I like cating files into other commands for the same reason, especially if the performance aspects are negligible.

It also depends on what you're scripting. If it's your own shell and convenience methods for your desktop machine, nobody except you will care. If you stumble upon a case where the next tool in the chain would be better off being able to seek, and distribute this as a frequently used piece of software on some minimal Linux system on a low-performance router or similar device with real limits on processing ability, that's different. It always depends on the context.

In every day command line use it's not really much different. You especially aren't going to notice any speed difference since the time on CPU avoided by not using cat, your CPU is just going to be idle. Even if you're looping through hundreds or thousands (or even hundreds of thousands) of items in all practicality it's not going to make much difference, unless you're on a very loaded system (Load Average / N CPU > 1).

The where the rubber meets the road is about forming good habits and discouraging bad ones. To drag out a moldy cliché, the devil is in the details. And it's details like this that separate the mediocre from the great.

It's like while driving a car, why make a left turn when you can just make three rights instead? Of course you can, and it works perfectly. But if you understood the power of left turns then three rights just seems silly.

It's not about saving one file handle, 17k of RAM and 0.004 seconds of CPU time. It's about the entire philosophy of using UNIX. The "power of left turns" in my illustration isn't merely redirecting input, it's the UNIX philosophy. Fully grokking this will make you excel far better than those around you, and you will garner respect from those who do understand.

  • If you are thinking about turning left onto a 6-lane busy highway without a traffic light, then it may behoove you to consider turning right, or taking a different route. *nix gives you the choice of several routes. It is a matter of personal preference and readability. If you want to "cat file | cmd1 | cat | cmd2 |more", go ahead. (Sometimes that is useful if cmd1 paginates - cat will eliminate it.) $CPU time << $Brain time. – MikeP Aug 2 '16 at 21:27

I think the position being taken by some of those commenting on something being a UUOC is that if one really understands Unix and shell syntax, one would not use cat in that context. It's seen as like using poor grammar: I can write a sentence using poor grammar and still get my point across, but I also demonstrate my poor understanding of the language and by extension, my poor education. So saying that something is a UUOC is another way of saying someone doesn't understand what they're doing.

As far as efficiency goes, if you are executing a pipeline from the command line, it takes less time for the machine to execute cat somefile | than it does for you to think about whether it might be more efficient to use < somefile. It just doesn't matter.

  • 5
    For quite a while I've known that there were other ways to express cat somefile | prog in shell without cat, like prog < somefile but they always seemed to be in the wrong order to me, particularly with a chain of commands piped together. Now I see that something as elegant as < somefile prog does the trick, thank you. I have run out of the excuses I had left to use cat. – Alex Jun 20 '14 at 15:45

I often use cat file | myprogram in examples. Sometimes I am being accused of Useless use of cat (http://partmaps.org/era/unix/award.html). I disagree for the following reasons:

  • It is easy to understand what is going on.

    When reading a UNIX command you expect a command followed by arguments followed by redirection. It is possible to put the redirection anywhere but it is rarely seen - thus people will have a harder time reading the example. I believe

    cat foo | program1 -o option -b option | program2
    

    is easier to read than

    program1 -o option -b option < foo | program2
    

    If you move the redirection to the start you are confusing people who are not used to this syntax:

    < foo program1 -o option -b option | program2
    

    and examples should be easy to understand.

  • It is easy to change.

    If you know the program can read from cat, you can normally assume it can read the output from any program that outputs to STDOUT, and thus you can adapt it for your own needs and get predictable results.

  • It stresses that the program does not fail, if STDIN is not a file.

    It is not safe to assume that if program1 < foo works then cat foo | program1 will also work. However, it is in practice safe to assume the opposite. This program works if STDIN is a file, but fails if the input is a pipe, because it uses seek:

    # works
    < foo perl -e 'seek(STDIN,1,1) || die;print <STDIN>'
    
    # fails
    cat foo | perl -e 'seek(STDIN,1,1) || die;print <STDIN>'
    

I have looked at the performance penalty on http://oletange.blogspot.dk/2013/10/useless-use-of-cat.html The conclusion is don't use cat file | if the complexity of the processing is similar to a simple grep and performance matters more than readability. For other situations cat file | is fine.

I was not aware of the award until today when some rookie tried to pin the UUOC on me for one of my answers. It was a cat file.txt | grep foo | cut ... | cut .... I gave him a piece of my mind, and only after doing so visited the link he gave me referring to the origins of the award and the practice of doing so. Further searching led me to this question. Somewhat unfortunately despite conscious consideration none of the answers included my rationale.

I did not meant to be defensive when educating him. After all, in my younger years I would have written the command as grep foo file.txt | cut ... | cut ... because whenever you do the frequent single greps you learn the placement of the file argument and it is ready knowledge that the first is the pattern and the later ones are file names.

It was a conscious choice when I answered the question with the cat prefix partly because of a reason of "good taste" (in the words of Linus Torvalds) but chiefly for a compelling reason of function.

The latter reason is more important so I will put it out first. When I offer a pipeline as a solution I expect it to be reusable. It is quite likely that a pipeline would be added at the end of or spliced into another pipeline. In that case having a file argument to grep screws up reusability, and quite possibly do so silently without an error message if the file argument exists. I. e. grep foo xyz | grep bar xyz | wc will give you how many lines in xyz contain bar while you are expecting the number of lines that contain both foo and bar. Having to change arguments to a command in a pipeline before using it is prone to errors. Add to it the possibility of silent failures and it becomes a particularly insidious practice.

The former reason is not unimportant either since a lot of "good taste" merely is an intuitive subconscious rationale for things like the silent failures above that you cannot think of right at the moment when some person in need of education says "but isn't that cat useless".

However, I will try to also make conscious the former "good taste" reason I mentioned. That reason has to do with the orthogonal design spirit of Unix. grep does not cut and ls does not grep. Therefore at the very least grep foo file1 file2 file3 goes against the design spirit. The orthogonal way of doing it is cat file1 file2 file3 | grep foo. Now, grep foo file1 is merely a special case of grep foo file1 file2 file3, and if you do not treat it the same you are at least using up brain clock cycles trying to avoid the useless cat award.

That leads us to the argument that grep foo file1 file2 file3 is concatenating, and cat concatenates so it is proper to cat file1 file2 file3 but because cat is not concatenating in cat file1 | grep foo therefore we are violating the spirit of both the cat and the almighty Unix. Well, if that were the case then Unix would need a different command to read the output of one file and spit it to stdout (not paginate it or anything just a pure spit to stdout). So you would have the situation where you say cat file1 file2 or you say dog file1 and conscientiously remember to avoid cat file1 to avoid getting the award, while also avoiding dog file1 file2 since hopefully the design of dog would throw an error if multiple files are specified.

Hopefully at this point you sympathize with the Unix designers for not including a separate command to spit a file to stdout, while also naming cat for concatenate rather than giving it some other name. <edit> there is such a dog, the unfortunate < operator. It is unfortunate its placement at the end of the pipeline preventing easy composability. There is no syntactically or aesthetically clean way to place it at the beginning. It is also unfortunate in not being general enough so you start with the dog but simply add another filename if you also want it to be processed after the previous one. (The > on the other hand is not half as bad. It has almost perfect placement at the end. It is typically not a reusable part of a pipeline, and accordingly it is distinguished symbolically.)</edit>

The next question is why is it important to have commands that merely spit a file or the concatenation of several files to stdout, without any further processing? One reason is to avoid having every single Unix command that operates on standard input to know how to parse at least one command line file argument and use it as input if it exists. The second reason is to avoid users having to remember: (a) where the filename arguments go; and (b) avoid the silent pipeline bug as mentioned above.

That brings us to why grep does have the extra logic. The rationale is to allow user-fluency for commands that are used frequently and on a stand-alone basis (rather than as a pipeline). It is a slight compromise of orthogonality for a significant gain in usability. Not all commands should be designed this way and commands that are not frequently used should completely avoid the extra logic of file arguments (remember extra logic leads to unnecessary fragility (the possibility of a bug)). The exception is to allow file arguments like in the case of grep. (by the way note that ls has a completely different reason to not just accept but pretty much require file arguments)

Finally, what could have been done better is if such exceptional commands as grep (but not necessarily ls) generate an error if the standard input is available. This is reasonable because the commands include logic that violates the orthogonal spirit of the almighty Unix for user convenience. For further user convenience, i. e. for preventing the suffering caused by a silent failure, such commands should not hesitate to violate their own violation by alerting the user if there is a possibility of silent failure.

  • As discussed on the cross-site duplicate of this answer and question, grep pattern f1 f2 f3 is not simple concatenation. grep knows about files, and prints filenames (and optionally line numbers, and whatever else). grep . /sys/kernel/mm/transparent_hugepage/* is a nice hack for printing filename: file-contents with lots of single-line files. The classic Unix design is that most utilities work on *.txt without needing cat. cat is for flattening multiple files into one stream. – Peter Cordes Mar 25 at 16:54
  • @PeterCordes I didn't write so much just about grep. There are substantive issues that I have observed regarding robustness to errors / copy-paste; correctness vs performance, that you have conveniently chose to ignore in favor of some petty/peripheral concern. – randomstring Mar 26 at 22:39
  • You do make some interesting and valid points, especially about the errors you can get when copy/pasting pipelines. I'd suggest replace your grep example with a program like cut that doesn't have any reason to care about multiple files, and could always just be fed from its stdin. Some utilities, like tr, don't accept file args at all, and only work as a filter, so the choice is between cat and <. – Peter Cordes Mar 26 at 22:56
  • My biggest problem with your post is that you discount input redirection. <file cmd1 | cmd2 >out is not wonderful, I admit, but it's totally possible to get used to it. You keep going on about "the spirit of the almighty Unix" in a mocking way, which totally falls flat for me because it sounds like you either don't get or don't want to get the way the Unix designers really did think. It's fine if you don't like the Unix design, but it's not inherently dumb. I'm not sure if the design of the OS predates shell syntax, and how it all evolved, but an extra cat in 1970 was worth avoiding! – Peter Cordes Mar 26 at 23:03
  • 1
    @PeterCordes In hindsight I was rather long-winded in my answer, which detracts from the most important point -- correctness first, optimization second. The extra cat helps reuse and splice pipelines, whereas without it you can get silent failures (search for "silently" in my answer). – randomstring Apr 8 at 3:22

What would be really nice is a shell that supports syntax like:

< filename cmd | cmd2 cmd2arg1... | cmd3

In the meantime, I think cat filename | realcmd1... is acceptable, as it keeps the syntax standardised with initial commands that require the filename as an argument.

  • 16
    Bash and similar shells support < filename cmd | cmd2 .... Is that close enough? – garyjohn Aug 14 '11 at 19:18
  • @garyjohn: I think you should post that as an answer. – Kevin Reid Aug 14 '11 at 19:41
  • 12
    Obligatory old shell-hacker comment: Bourne-style shells have supported the < file command ... since at least the mid-80s and probably as far back as the 70s when the original sh was written. More generally, i/o redirections are parsed left to right, and can be interspersed in any order within the command line. So, cmd <file arg arg... would also be valid. – Dale Hagglund Aug 15 '11 at 2:28
  • 2
    Yeah, it's partly because of how easy it is to type that that I invented the UUOC. – Randal Schwartz May 2 '13 at 2:48
  • 2
    One shifted character vs. four unshifted isn't that big a difference, and I'd rather spawn an extra process, which even my phone barely notices, than pipe a file into my prompt, which gives me a headache every time I see it. – Aaron Miller Jun 20 '13 at 16:26

In defense of Useless Uses of Cat

(A few paragraph to help balance the tsunami of nagging comments against this practice)

I use cat although I can avoid it every day and frankly for my daily CLI routine I don't feel even a tiny bit compelled to change. Here are my reasons:

I feel using cat is convenient and ... "smells" better. Granted this is not scientific. Maybe there is a real reason behind this feeling and maybe it's just bad habit. However even if it is a bad habit does it worth my time to fix this specific bad habit? I don't think so, I'm not perfect and I have more important things to fix in myself.

What I'm sure about is that rarely the extra CPU/memory expense of cat is worth my limited time. I'll only give it a second thought when writing scripts I plan to deploy for use by third parties and even then I'll mostly do it to avoid the nagging and not because it would be more efficient in any noticeable way. Yes, sometimes it is but that's rare. I certainly don't write tight loops inside bash scripts every month.

For all who say cat is acceptable to use because it "smells" better or "is more readable" I would only say this:

To you maybe... but not to others who may read or try to understand your code. If you will never ever try to instruct others with your examples or share your code, then by all means please use it at your own leisure.

I will also add this comment, as a long time Linux user and Admin/Engineer... (and there are many of us) it makes our eyes bleed to see this. Why? Because it uses resources on systems that we control resources tightly on. The cat command and the pipe itself use extra memory and file handles that are completely useless. You have tied up resources that my system needs free and you have gained NOTHING that can explain the usage of these resources. This is a huge no no.

Now I can sit here and debate things like code smell or readability all day with anyone, but at the end of the day it is a matter of write or wrong and any time you use resources on a system and gain nothing for it... it is wrong.

As a home user you can learn from my advice and learn better ways to do things or you can choose to be blinded by the "smell" of cats, your choice... but know that if you openly use this practice you will be called on this practice all the time and you will silently have to admit they are right and you are stubborn because it's true. :-)

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.