7

This first pipeline works fine (printing "c"):

echo "a" | sed 's/a/b/' | sed 's/b/c/'

This one does not do what I expect (nothing gets printed when I feed an "a" into my fifo ):

mkfifo fifo;
cat fifo | sed 's/a/b/' | sed 's/b/c/' 

However, if I remove the second "sed" command from the latter pipeline, I do get a "b" printed. I think my understanding of pipes and redirects must be too simplistic. Can someone explain to me how to fix the 2nd case so that I can run two successive commands on the contents of the fifo?

(note this isn't a problem specific to fifo, the same behavior occurs with netcat too. I posted a similar question about netcat but got no answers)

6
  • 2
    Your code works for me. Maybe, you could spell out more detail on how you "feed an 'a'" into your fifo.
    – John1024
    Aug 4, 2014 at 6:34
  • As @John1024 said, WorksForMe.
    – Bob
    Aug 4, 2014 at 6:35
  • Me too, no problem whatsoever. Aug 4, 2014 at 6:36
  • echo "a" >fifo Aug 4, 2014 at 6:36
  • It seems this is buffering related. What version of sed are you using? Mine is GNU sed version 4.2.1 Aug 4, 2014 at 6:40

1 Answer 1

8

Use the -u (--unbuffered) option of sed, so it writes it's output, instead of collecting it first for a fast, large write later.

cat fifo | sed -u 's/a/b/' | sed -u 's/b/c/'

cat is always unbuffered.

If it's not about sed, but some program with output buffering, but without options to disable it, there are tools that can help to force buffering off - see man unbuffer and man stdbuf.

If it's a shell script, it should be unbuffered by default: when a command creates output in the script, and exits, the output is flushed. Otherwise, you use unbuffer inside the shell script.

For background, see this overview of stdio buffering.

5
  • Suppose instead of sed, I had my own custom script. What would I do in this case to avoid the buffering? Or should it be unbuffered by default? Aug 4, 2014 at 6:42
  • Just some more followup - if I wanted to use perl regular expressions with buffering, it would go something like nc -k -l 4458 | perl -ne 's{HTTP}{FTP}g; $|++;print $_' | perl -ne 's{FTP}{RTSP}g; $|++;print $_' Aug 5, 2014 at 4:14
  • Yes, what do you mean? Is there a problem, or is it an example? Oh, and you can use ssed -R if you like sed with perl regular expressions. Aug 5, 2014 at 4:32
  • Sorry, that was not a question. I apologize for not being clear, it was just another example of the same concept. Unfortunately it's too late for me to edit that comment. I didn't know sed could understand Perl regex. Perhaps I should start using that instead from now on. Aug 5, 2014 at 4:35
  • Oh, no problem! Aug 5, 2014 at 4:36

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .