Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I need to search for and replace three consecutive newline characters in an input file and filter them out of an output file for a makefile rule on Centos 4. I am using GNU sed v4.1.2 and GNU make v3.82. I've tried variations on the following with no success thus far:

THREE_LINE_FEEDS := $(shell echo "\012\012\012")

SED_RULE := 's/'$(THREE_LINE_FEEDS)'//'

output.txt: input.txt
    sed -r $(SED_RULE) input.txt > output.txt

Using the proposed perl, I get this issue in the shell (adapted from my make rule):

> cat input.txt | perl -e '$/ = undef; _ = <>; s/\n{3}//g; print;' > output.txt
Can't modify constant item in scalar assignment at -e line 1, near "<>;"
Execution of -e aborted due to compilation errors.
share|improve this question

sed only sees one line at a time, so searching for consecutive line-feeds will not work. One way is to push all the data into the hold space, and then do the replace when all input has been gathered, e.g.:

sed -r -n '
  $! {             # When it is not the last line
    1  { h }       # Replace hold space with first line
    1! { H }       # Otherwise append to hold space
  }
  $  {             # When last line reached
    H              # Append it to hold space
    g              # Replace pattern space with hold space
    s/\n{3}//g     # Remove all occurrences of \n{3}
    p              # Print pattern space
  }
' input.txt

Here is a simpler option using perl, but works in the same manner:

perl -e '
  $/ = undef;      # Slurp mode, read whole file
  $_ = <>;         # Read file to $_
  s/\n{3}//g;      # Remove all occurrences of \n{3}
  print;           # Output $_
' input.txt

Edit

A shorter perl version suggested by Peter O.:

perl -0777 -pe 's/\n{3}//g' input.txt
  • -0777 enables slurp mode.
  • -p implicitly prints result.
share|improve this answer
    
The perl is not working when inserted into my makefile rules, please see above updated question. – WilliamKF Jul 28 '12 at 15:49
    
When using $ in makefiles they need to be protected, i.e. add an extra $ on there, so perl -e '$$/ = undef; $$_ = <>; s/\n{3}//g; print' input.txt > output.txt should work. – Thor Jul 28 '12 at 15:52
    
Yes, had figured that out, but still not working. Getting error: Can't modify constant item in scalar assignment at -e line 1, near "<>;" Execution of -e aborted due to compilation errors. Can you pipe into perl? Why is it executing -e? – WilliamKF Jul 28 '12 at 16:14
    
The error you're getting comes from this bit: _ = <>;, should be $_ = <>;. Either of perl -e ... infile and cat infile | perl -e ... works. -e specifies the command perl should run, see perlrun(1). – Thor Jul 28 '12 at 17:02
2  
perl -0777pe 's/\n{3}//g' is exactly the same. 0777 puts Perl into file-slurp mode, and p makes a read loop which prints – Peter.O Jul 28 '12 at 21:32
up vote 0 down vote accepted

For those interested, here is the actual makefile rule (I couldn't get the newlines to work in the original, so used a comment to add the documentation):

# 0777  -  Enables slurp mode (i.e. $/ = undef;  -  Slurp mode, read whole file
#                                   $_ = <>;     -  Read file to $_)
# s/\n{3}//g;                                    -  Remove occurrences of \n{3}
# p     -  Implicitly print result (i.e. print;  -  Output $_)
output.txt: input.txt
    $(SED) -r $(SED_RULE) input.txt | perl -0777 -pe 's/\n{3}//g' > output.txt
share|improve this answer

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .