Pipes or named pipes are a feature of the POSIX standard that allow separate processes to communicate with each other without having been designed explicitly to work together.

learn more… | top users | synonyms

150
votes
4answers
35k views

Less and Grep: Getting colored results when using a pipe from grep to less

I use the --colour option of grep a lot, but I often use less as well. How can I pipe grep results to less and still preserve the coloring. (Or is that possible?) grep "search-string" -R * --colour | ...
68
votes
2answers
11k views

Why is piping 'dd' through gzip so much faster than a direct copy?

I wanted to backup a path from a computer in my network to another computer in the same network over a 100 Mbit/s line. For this I did dd if=/local/path of=/remote/path/in/local/network/backup....
61
votes
6answers
13k views

How can I save the current contents of less to a file?

If I've piped the results of a command to less and then decided that I want to save the contents to a file, is this possible? I've tried setting a mark a at the end of the buffer, and then returning ...
49
votes
4answers
75k views

How to pipe command output to other commands?

Example: ls | echo prints nothing ( a blank line, actually ). I'd expect it to print a list of files. ls | grep 'foo', on the other hand, works as expected ( prints files with 'foo' in their name ). ...
41
votes
3answers
8k views

How to rate-limit a pipe under linux?

Is there a filter which I could use to rate-limit a pipe on linux? If this exists, let call it rate-limit, I want to be able to type in a terminal something like cat /dev/urandom | rate-limit 3 -k | ...
28
votes
6answers
5k views

What is the general consensus on “Useless use of cat”?

When I pipe multiple unix commands such as grep, sed, tr etc. I tend to specify the input file that is being processed using cat. So something like cat file | grep ... | awk ... | sed ... . But ...
25
votes
4answers
33k views

cURL: how to suppress strange output when redirecting?

I'm trying to print just the verbose sections of a cURL request (which are sent to stderr) from the bash shell. But when I redirect stdout like this: curl -v http://somehost/somepage > /dev/null ...
21
votes
2answers
13k views

Pipe gunzip and mysql to gunzip a dump and import it

I have a .gz sql dump file (example: foo.sql.gz) that i want import in my database with the classic mysql command. gunzip -c foo.sql.gz > foo.sql mysql -uroot -ppassword foo < foo.sql foo is ...
21
votes
2answers
42k views

Howto pipe: cp | tar | gzip without creating intermediary files?

Can anyone tell me if it's possible to pipe | this without having to create a physical file anywhere between A and B.tar.gz? This is what I'm trying to do: File A Rename A to B Tar B.tar gzip -9 B....
20
votes
1answer
7k views

Find what process is on the other end of a pipe

I'm trying to trace some odd behavior of a few processes and ran into a point I'm not sure how to trace past. The hung process, which I attached to using strace -p showed this: Process 7926 attached ...
17
votes
2answers
2k views

What is the magic separator between filenames in ls output?

The output of ls (with no arguments) appears to separate filenames with linebreaks. Evidence: ls | grep foo works as expected, with grep treating each filename as a separate line of input. ls > ...
15
votes
2answers
9k views

Why does Powershell silently convert a string array with one item to a string

Consider the following Powershell script, which searches for folders in C:\ with a 'og' in their name: PS C:\> (ls | %{$_.Name} | ?{$_.Contains("og")}) PerfLogs Program Files setup.log Now I ...
14
votes
7answers
11k views

Advantages of cat'ing file and piping to grep

Are there any additional advantages of cat'ing a file and piping it to grep, besides convenience? The convenience being that, when I retrieve commands such as those below from my history, the cursor ...
13
votes
2answers
4k views

Preserve colors while piping to tee

ls -l --color=auto | tee output.log Without pipe/tee it's colored. How can I make it so that it stays colored while using tee (can be colored only on the screen, I don't care about colors in logs).
13
votes
1answer
10k views

how to escape pipe symbol | in bat scripts?

I need to echo commad usage like this in batch script named command.bat command.bat on|off I cant use echo %0 on|off I know I can use echo "%0 on|off" but it puts quotes along Is there any ...
12
votes
5answers
19k views

What does the linux pipe symbol “|” do? [duplicate]

Here is a command that sorts files in a folder in reverse order ls | sort -r What does the | symbol in that command do? What I'm really looking for here is high level (easy to understand) ...
12
votes
5answers
9k views

Bash: create anonymous fifo

We all know mkfifo and pipelines. The first one creates a named pipe, thus one has to select a name, most likely with mktemp and later remember to unlink. The other creates an anonymous pipe, no ...
12
votes
3answers
6k views

Comments in a multi-line bash command

This single-command BASH script file is difficult to understand, so I want to write a comment for each of the actions: grep -R "%" values* \ | sed -e "s/%/\n%/" \ | grep "%" \ | grep -v " % " \ | ...
12
votes
6answers
6k views

Make a pipe conditional on non-empty return

Is there a way to pipe the output of one function to another, but only if there is an output?
12
votes
4answers
36k views

How can I pipe output of ffmpeg to ffplay?

How can I pipe the output of ffmpeg to ffplay? At the moment I use a workaround in bash : mkfifo spam (ffplay spam 2> /dev/null &) ; capture /dev/stdout | ffmpeg -i - spam
12
votes
3answers
18k views

Passing two arguments to a command using pipes

Usually, we only need to pass one argument: echo abc | cat echo abc | cat some_file - echo abc | cat - some_file Is there a way to pass two arguments? Something like {echo abc , echo xyz} | cat ...
12
votes
5answers
7k views

What is a “pipe” and how can it be “broken”?

I've gotten the "broken pipe" error from Xcode one time too many. I'm now curious to know exactly what a pipe is. What is the concept of a "pipe" and how can it be "broken"?
10
votes
4answers
18k views

Why does this not work? “ls *.txt | xargs cat > all.txt” (all files into single txt document)

Why does this not work? ls *.txt | xargs cat > all.txt (I want to join the contents of all text files into a single 'all.txt' file.) find with -exec should also work, but I would really like to ...
10
votes
1answer
2k views

Rendering HTML from a pipe

I would like to be able to generate HTML then pipe it to a program which will render it, something like this: for i in 1 2 3 do for j in a b c do echo "<table border="1"><tr&...
9
votes
1answer
1k views

Is backwards redirection the same as a pipe?

In Linux if you type sort < txtfile is that the same thing as cat txtfile | sort
8
votes
3answers
1k views

How to a open a file in vim using pipe

I get to use the locate command extremely often. So if I run the following command. locate updatedb | head -1 Then it gives me the O/p /usr/updatedb.conf I wonder if there is any such command ...
8
votes
2answers
2k views

Why is xargs necessary?

Suppose I want to remove all files in a directory except for one named "notes.txt". I would do this with the pipeline, ls | grep -v "notes.txt" | xargs rm. Why do I need xargs if the output of the ...
8
votes
7answers
5k views

how to beep on tail -f event

I want my PC to make a system beep on every tail event I have the following command tail -f development.log | grep "something rare" is there an easy way like pipeing it to somthing that beeps? ...
8
votes
3answers
4k views

Does pipe have to write temporary file?

I found that if I transfer a great amount of data between two processes via pipe, some temporary file will be created by linux in /tmp directory. If the pipe operation succeeds, the corresponding ...
8
votes
2answers
17k views

In windows, can I redirect stdout to a (named) pipe in command line?

Is there a way to redirect the standard output of a process in Win32 console to a named pipe? Named pipes are built in to Windows and while they would be an useful concept, I've never seen them used ...
8
votes
1answer
13k views

What are reasons for local Windows named-pipes to fail?

I've been working hard on this one all day and I'm stuck. This morning our asian collegues called me because a SolidWorks addin for our product data management system could not communicate with the ...
7
votes
3answers
6k views

How can I specify command line arguments using pipes in Linux?

I'm a newbie to shell programming and have no idea how to solve this problem. I just downloaded a file from the Internet to the default directory ~/Downloads. I want to move that file to another ...
7
votes
3answers
1k views

Why are not all files compressed and how to improve the solution

I have a folder with about 20K files. The files are named according to the pattern xy_{\d1,5}_{\d4}\.abc, e.g xy_12345_1234.abc. I wanted to compress the first 10K of them using this command: ls | ...
7
votes
2answers
2k views

How do you determine the actual command that is piping into you?

Let's say I have a bash script called log.sh. In this script, I want to read in input from a pipe, but I also want to know the command used to pipe input into me. Example: tail -f /var/log/httpd/...
7
votes
4answers
9k views

Can I use pipe output as a shell script argument?

Suppose I have a bash shell script called Myscript.sh that need on argument as input. But I want the content of the text file called text.txt to be that argument. I have tried this but it does not ...
7
votes
1answer
464 views

Multiple unix pipes not working

This first pipeline works fine (printing "c"): echo "a" | sed 's/a/b/' | sed 's/b/c/' This one does not do what I expect (nothing gets printed when I feed an "a" into my fifo ): mkfifo fifo; cat ...
7
votes
5answers
5k views

How to get both PIPESTATUS and output in bash script

I'm trying to get last modification date of a file with this command TM_LOCAL=`ls -l --time-style=long-iso ~/.vimrc | awk '{ print $6" "$7 }'` TM_LOCAL has value like "2012-05-16 23:18" after ...
7
votes
2answers
2k views

Is there any way to keep text passed to head, tail, less, etc. to be colored?

Is there any way to keep colorization of text passed through pipe | to head, tail, less, etc.?
7
votes
2answers
4k views

monitor interprocess pipe traffic

I have two Linux processes communicating via a nameless pipe. How can monitor the traffic in the pipe? How can I inject data into the pipe? I have root access and know the pipe inode.
7
votes
1answer
6k views

Is it possible to pipe a list of files to RMDIR on Windows?

I am writing a batch file for the Windows command prompt to delete all directories matching a certain expression. I am first using DIR to return a plain list of directories matching the expression. I ...
6
votes
3answers
4k views

Windows how to redirect file parameter to stdout? (Windows equivalent of `/dev/stdout`)

Windows console: Tool A can write binary data to a file, but has no option for telling it to use stdout. Tool B can read binary data from stdin and process the info in it. How can I get the output ...
6
votes
1answer
4k views

extract a few members from tar archive and pipe through network

edit: I want to extra member01 and member02 and directory blah/ tarball_1.tar.gz contains directory test/ with 20 files. I want to extract only member test/member01 and test/member02 and directory ...
6
votes
1answer
3k views

Pipe not working in cmd.exe on Windows 7

I have started seeing some very strange issues with the pipe command not working on Windows 7 64-bit on my machine. Commands like this: echo test | more used to work without a problem, but now ...
6
votes
1answer
454 views

cat-ing a file to a unix pipe causes Matlab to freeze

I've created a named pipe on Debian using mkfifo pipe.in. I want to write to this pipe from Matlab. To do this, I use the following matlab command: unix( 'cat <myfile> > pipe.in' ) ...
5
votes
4answers
14k views

I would like to pipe output of find into input list of scp, how?

I'm a novice linux user and I am trying to send a long list of files from one computer to another. The argument list is too long, so I am using find. I am having trouble setting up the expression, ...
5
votes
2answers
3k views

How do I pipe output to date -d “value”?

I have a date like "2014-01-30 05:04:27 GMT", and if I run date -d "2014-01-30 05:04:27 GMT", the output is in my server's timezone ("Thu Jan 30 16:04:27 EST 2014"). With the use of grep and cut, I ...
5
votes
3answers
6k views

How to pipe awk output (with periodic, continuous input) to output file?

I am trying to write a command that pipes the continuous output of a free command (run every second) to an awk command that parses a specific value (available free memory) and outputs this to a file ...
5
votes
1answer
727 views

Locked out of host after reaching some ulimit

So, I have a sudoer account in a remote host, in which I create a process with thousands of threads and sockets. The thing is, if I create too many threads/sockets (not sure which one, it might even ...
5
votes
2answers
3k views

Notepad++, Run command and open output in new tab

Is there any way to execute a command from notepad++ and have the results piped back into the program in a new tab?
5
votes
1answer
66 views

How many ways does Unix feign files?

I know Unix filesystem sometimes has things that appear to be files but are not actually files. The files in dev actually point to devices. Its also possible to created named pipes which also appear ...