Pipes or named pipes are a feature of the POSIX standard that allow separate processes to communicate with each other without having been designed explicitly to work together.

learn more… | top users | synonyms

73
votes
4answers
15k views

Less and Grep: Getting colored results when using a pipe from grep to less

I use the --colour option of grep a lot, but I often use less as well. How can I pipe grep results to less and still preserve the coloring. (Or is that possible?) grep "search-string" -R * --colour | ...
24
votes
2answers
3k views

How to rate-limit a pipe under linux?

Is there a filter which I could use to rate-limit a pipe on linux? If this exists, let call it rate-limit, I want to be able to type in a terminal something like cat /dev/urandom | rate-limit 3 -k | ...
20
votes
6answers
2k views

What is the general consensus on “Useless use of cat”?

When I pipe multiple unix commands such as grep, sed, tr etc. I tend to specify the input file that is being processed using cat. So something like cat file | grep ... | awk ... | sed ... . But ...
18
votes
4answers
17k views

How to pipe command output to other commands?

Example: ls | echo prints nothing ( a blank line, actually ). I'd expect it to print a list of files. ls | grep 'foo', on the other hand, works as expected ( prints files with 'foo' in their name ). ...
17
votes
4answers
3k views

How can I save the current contents of less to a file?

If I've piped the results of a command to less and then decided that I want to save the contents to a file, is this possible? I've tried setting a mark a at the end of the buffer, and then returning ...
15
votes
2answers
973 views

What is the magic separator between filenames in ls output?

The output of ls (with no arguments) appears to separate filenames with linebreaks. Evidence: ls | grep foo works as expected, with grep treating each filename as a separate line of input. ls > ...
13
votes
4answers
11k views

cURL: how to suppress strange output when redirecting?

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell. But when I redirect stdout like this: curl -v http://somehost/somepage > /dev/null ...
10
votes
2answers
4k views

Pipe gunzip and mysql to gunzip a dump and import it

I have a .gz sql dump file (example: foo.sql.gz) that i want import in my database with the classic mysql command. gunzip -c foo.sql.gz > foo.sql mysql -uroot -ppassword foo < foo.sql foo is ...
10
votes
5answers
4k views

What is a “pipe” and how can it be “broken”?

I've gotten the "broken pipe" error from Xcode one time too many. I'm now curious to know exactly what a pipe is. What is the concept of a "pipe" and how can it be "broken"?
9
votes
7answers
5k views

Advantages of cat'ing file and piping to grep

Are there any additional advantages of cat'ing a file and piping it to grep, besides convenience? The convenience being that, when I retrieve commands such as those below from my history, the cursor ...
8
votes
4answers
12k views

How can I pipe output of ffmpeg to ffplay?

How can I pipe the output of ffmpeg to ffplay? At the moment I use a workaround in bash : mkfifo spam (ffplay spam 2> /dev/null &) ; capture /dev/stdout | ffmpeg -i - spam
8
votes
1answer
4k views

how to escape pipe symbol | in bat scripts?

I need to echo commad usage like this in batch script named command.bat command.bat on|off I cant use echo %0 on|off I know I can use echo "%0 on|off" but it puts quotes along Is there any ...
8
votes
3answers
2k views

Does pipe have to write temporary file?

I found that if I transfer a great amount of data between two processes via pipe, some temporary file will be created by linux in /tmp directory. If the pipe operation succeeds, the corresponding ...
7
votes
6answers
3k views

how to beep on tail -f event

I want my PC to make a system beep on every tail event I have the following command tail -f development.log | grep "something rare" is there an easy way like pipeing it to somthing that beeps? ...
7
votes
2answers
2k views

Why does Powershell silently convert a string array with one item to a string

Consider the following Powershell script, which searches for folders in C:\ with a 'og' in their name: PS C:\> (ls | %{$_.Name} | ?{$_.Contains("og")}) PerfLogs Program Files setup.log Now I ...
7
votes
2answers
2k views

Comments in a multi-line bash command

This single-command BASH script file is difficult to understand, so I want to write a comment for each of the actions: grep -R "%" values* \ | sed -e "s/%/\n%/" \ | grep "%" \ | grep -v " % " \ | ...
7
votes
1answer
650 views

Is backwards redirection the same as a pipe?

In Linux if you type sort < txtfile is that the same thing as cat txtfile | sort
7
votes
2answers
6k views

Passing two arguments to a command using pipes

Usually, we only need to pass one argument: echo abc | cat echo abc | cat some_file - echo abc | cat - some_file Is there a way to pass two arguments? Something like {echo abc , echo xyz} | cat ...
6
votes
2answers
1k views

How do you determine the actual command that is piping into you?

Let's say I have a bash script called log.sh. In this script, I want to read in input from a pipe, but I also want to know the command used to pipe input into me. Example: tail -f ...
6
votes
2answers
13k views

Howto pipe: cp | tar | gzip without creating intermediary files?

Can anyone tell me if it's possible to pipe | this without having to create a physical file anywhere between A and B.tar.gz? This is what I'm trying to do: File A Rename A to B Tar B.tar gzip -9 ...
6
votes
4answers
3k views

Bash: create anonymous fifo

We all know mkfifo and pipelines. The first one creates a named pipe, thus one has to select a name, most likely with mktemp and later remember to unlink. The other creates an anonymous pipe, no ...
6
votes
1answer
3k views

Is it possible to pipe a list of files to RMDIR on Windows?

I am writing a batch file for the Windows command prompt to delete all directories matching a certain expression. I am first using DIR to return a plain list of directories matching the expression. I ...
5
votes
3answers
2k views

How can I specify command line arguments using pipes in Linux?

I'm a newbie to shell programming and have no idea how to solve this problem. I just downloaded a file from the Internet to the default directory ~/Downloads. I want to move that file to another ...
5
votes
6answers
2k views

Make a pipe conditional on non-empty return

Is there a way to pipe the output of one function to another, but only if there is an output?
5
votes
1answer
981 views

Pipe not working in cmd.exe on Windows 7

I have started seeing some very strange issues with the pipe command not working on Windows 7 64-bit on my machine. Commands like this: echo test | more used to work without a problem, but now ...
5
votes
1answer
2k views

Find what process is on the other end of a pipe

I'm trying to trace some odd behavior of a few processes and ran into a point I'm not sure how to trace past. The hung process, which I attached to using strace -p showed this: Process 7926 attached ...
5
votes
4answers
2k views

How to get both PIPESTATUS and output in bash script

I'm trying to get last modification date of a file with this command TM_LOCAL=`ls -l --time-style=long-iso ~/.vimrc | awk '{ print $6" "$7 }'` TM_LOCAL has value like "2012-05-16 23:18" after ...
5
votes
2answers
2k views

monitor interprocess pipe traffic

I have two Linux processes communicating via a nameless pipe. How can monitor the traffic in the pipe? How can I inject data into the pipe? I have root access and know the pipe inode.
5
votes
1answer
4k views

What are reasons for local Windows named-pipes to fail?

I've been working hard on this one all day and I'm stuck. This morning our asian collegues called me because a SolidWorks addin for our product data management system could not communicate with the ...
5
votes
1answer
226 views

cat-ing a file to a unix pipe causes Matlab to freeze

I've created a named pipe on Debian using mkfifo pipe.in. I want to write to this pipe from Matlab. To do this, I use the following matlab command: unix( 'cat <myfile> > pipe.in' ) ...
5
votes
1answer
368 views

Locked out of host after reaching some ulimit

So, I have a sudoer account in a remote host, in which I create a process with thousands of threads and sockets. The thing is, if I create too many threads/sockets (not sure which one, it might even ...
5
votes
1answer
57 views

How many ways does Unix feign files?

I know Unix filesystem sometimes has things that appear to be files but are not actually files. The files in dev actually point to devices. Its also possible to created named pipes which also appear ...
4
votes
5answers
279 views

How do I pipe a list of numbers straight from the shell into a command?

How do I pipe a list of numbers straight from the shell into a command? For exampe something like this [1,2,3,4] | sort would give 1 2 3 4 EDIT: In response to the answers kindly posted so ...
4
votes
3answers
108 views

Transparent usage of temporary files as means of piping

As many of us are surely aware, it's always a good idea to make your program accept stdin input. Very many programs do allow that *nix environments. This lets us do cool things like piping echo "foo" ...
4
votes
1answer
83 views

Launch only the command if the previous one worked inside SSH, shell

I've got a SHELL script which is using a pipe to separate my two commands: ssh -oBatchMode=yes user@hostname "mysql -u yop -pyop -c yop | echo test" The problem is even if my connection to MySQL ...
4
votes
2answers
631 views

Is there any way to keep text passed to head, tail, less, etc. to be colored?

Is there any way to keep colorization of text passed through pipe | to head, tail, less, etc.?
4
votes
5answers
656 views

commands executing in piping in UNIX

when I write a command $ echo date then it prints "date" as it is i.e it doesn't run date program. But when I write $ echo date | wc then correct answer is produced as if date was run. How ...
4
votes
1answer
899 views

emacs as a pager?

I am looking for a way to use emacs as my pager command in the shell (for example with man or to scroll the output of an asynchronous command with a large amount of output). I use emacsclient as my ...
3
votes
3answers
2k views

How can I grep in source files for some text?

At the moment I'm using two commands, I'm sure there must be a better way... wim@wim-acer:~/ffmpeg$ find . -name "*.h" -print0 | xargs -0 grep -i invalid\ preset wim@wim-acer:~/ffmpeg$ find . -name ...
3
votes
2answers
233 views

Why is xargs necessary?

Suppose I want to remove all files in a directory except for one named "notes.txt". I would do this with the pipeline, ls | grep -v "notes.txt" | xargs rm. Why do I need xargs if the output of the ...
3
votes
2answers
63 views

Sort files with grep

I have a few hundreds of output files, of which some contain error messages. Lets say I want to automatically put all files containing the string "ERROR_1" in a "subfolder_1", and the rest in a second ...
3
votes
1answer
2k views

extract a few members from tar archive and pipe through network

edit: I want to extra member01 and member02 and directory blah/ tarball_1.tar.gz contains directory test/ with 20 files. I want to extract only member test/member01 and test/member02 and directory ...
3
votes
1answer
624 views

Unix commandline to repeat command with pipes

I want to write a script that will repeat a commandline that usually contains pipes ./myscript.sh ls -lart |grep ^d And in myscript.sh I do a bunch of stuff, and then want to execute ls -lart |grep ...
3
votes
3answers
1k views

How to pipe data over tcp from the command line?

I would like to pipe data from one machine in the command line to another machine over tcp. I guess I could write a socket server but this must already be implemented. For example I could use it to xz ...
3
votes
2answers
336 views

How can I retain grep's match highlighting when piping find into grep?

If I run grep foo bar.txt, grep highlights each occurrence of "foo" in bar.txt. But sometimes I want to use find to determine which files grep searches. So I do something like this: find . -iname ...
3
votes
3answers
238 views

linux tar -T - does not work on the fly

I found some problem with linux gnu tar. wheh I use option -T - (for file list from stdin) or -T named_pipe_file , this desn't work on the fly. for example, simple interactive script: while ...
3
votes
2answers
292 views

How can I run several parallel shell jobs and monitor all of them at once?

I have a large number of large files (hundreds of files, hundreds of MB each) which I need to pipe through a number of programs to filter and convert them. I'm taking advantage of multiple CPU cores ...
3
votes
1answer
199 views

Redirect stdout to multiple programs

I'm doing some stream processing of some quite substantial files-- I was wondering if there was a way to split stdout in order to pipe to multiple programs. What I'd like to do is something like this: ...
3
votes
2answers
90 views

Mutual piping on linux

I would like the output of A to be input for B and at the same time the output of B to be the input for A, is that possible? I tried the naïve thing: creating named pipes for A (pipeA) and B (pipeB) ...
3
votes
2answers
340 views

How to utilize vim's syntax color to replace the `highlight' utility?

I feel that the `highlight' utility doesn't support as many file types as vim does, and the project isn't very active. When the file is gzipped, highlight doesn't recognize it at all. But in ...