This first pipeline works fine (printing "c"):

echo "a" | sed 's/a/b/' | sed 's/b/c/'

This one does not do what I expect (nothing gets printed when I feed an "a" into my fifo ):

mkfifo fifo;
cat fifo | sed 's/a/b/' | sed 's/b/c/' 

However, if I remove the second "sed" command from the latter pipeline, I do get a "b" printed. I think my understanding of pipes and redirects must be too simplistic. Can someone explain to me how to fix the 2nd case so that I can run two successive commands on the contents of the fifo?

(note this isn't a problem specific to fifo, the same behavior occurs with netcat too. I posted a similar question about netcat but got no answers)

  • 2
    Your code works for me. Maybe, you could spell out more detail on how you "feed an 'a'" into your fifo. – John1024 Aug 4 '14 at 6:34
  • As @John1024 said, WorksForMe. – Bob Aug 4 '14 at 6:35
  • Me too, no problem whatsoever. – MariusMatutiae Aug 4 '14 at 6:36
  • echo "a" >fifo – Sridhar Sarnobat Aug 4 '14 at 6:36
  • It seems this is buffering related. What version of sed are you using? Mine is GNU sed version 4.2.1 – Sridhar Sarnobat Aug 4 '14 at 6:40

Use the -u (--unbuffered) option of sed, so it writes it's output, instead of collecting it first for a fast, large write later.

cat fifo | sed -u 's/a/b/' | sed -u 's/b/c/'

cat is always unbuffered.

If it's not about sed, but some program with output buffering, but without options to disable it, there are tools that can help to force buffering off - see man unbuffer and man stdbuf.

If it's a shell script, it should be unbuffered by default: when a command creates output in the script, and exits, the output is flushed. Otherwise, you use unbuffer inside the shell script.

For background, see this overview of stdio buffering.

  • Suppose instead of sed, I had my own custom script. What would I do in this case to avoid the buffering? Or should it be unbuffered by default? – Sridhar Sarnobat Aug 4 '14 at 6:42
  • Just some more followup - if I wanted to use perl regular expressions with buffering, it would go something like nc -k -l 4458 | perl -ne 's{HTTP}{FTP}g; $|++;print $_' | perl -ne 's{FTP}{RTSP}g; $|++;print $_' – Sridhar Sarnobat Aug 5 '14 at 4:14
  • Yes, what do you mean? Is there a problem, or is it an example? Oh, and you can use ssed -R if you like sed with perl regular expressions. – Volker Siegel Aug 5 '14 at 4:32
  • Sorry, that was not a question. I apologize for not being clear, it was just another example of the same concept. Unfortunately it's too late for me to edit that comment. I didn't know sed could understand Perl regex. Perhaps I should start using that instead from now on. – Sridhar Sarnobat Aug 5 '14 at 4:35
  • Oh, no problem! – Volker Siegel Aug 5 '14 at 4:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.