Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I need to search for and replace three consecutive newline characters in an input file and filter them out of an output file for a makefile rule on Centos 4. I am using GNU sed v4.1.2 and GNU make v3.82. I've tried variations on the following with no success thus far:

THREE_LINE_FEEDS := $(shell echo "\012\012\012")

SED_RULE := 's/'$(THREE_LINE_FEEDS)'//'

output.txt: input.txt
    sed -r $(SED_RULE) input.txt > output.txt

Using the proposed perl, I get this issue in the shell (adapted from my make rule):

> cat input.txt | perl -e '$/ = undef; _ = <>; s/\n{3}//g; print;' > output.txt
Can't modify constant item in scalar assignment at -e line 1, near "<>;"
Execution of -e aborted due to compilation errors.
share|improve this question
add comment

2 Answers

sed only sees one line at a time, so searching for consecutive line-feeds will not work. One way is to push all the data into the hold space, and then do the replace when all input has been gathered, e.g.:

sed -r -n '
  $! {             # When it is not the last line
    1  { h }       # Replace hold space with first line
    1! { H }       # Otherwise append to hold space
  }
  $  {             # When last line reached
    H              # Append it to hold space
    g              # Replace pattern space with hold space
    s/\n{3}//g     # Remove all occurrences of \n{3}
    p              # Print pattern space
  }
' input.txt

Here is a simpler option using perl, but works in the same manner:

perl -e '
  $/ = undef;      # Slurp mode, read whole file
  $_ = <>;         # Read file to $_
  s/\n{3}//g;      # Remove all occurrences of \n{3}
  print;           # Output $_
' input.txt

Edit

A shorter perl version suggested by Peter O.:

perl -0777 -pe 's/\n{3}//g' input.txt
  • -0777 enables slurp mode.
  • -p implicitly prints result.
share|improve this answer
    
The perl is not working when inserted into my makefile rules, please see above updated question. –  WilliamKF Jul 28 '12 at 15:49
    
When using $ in makefiles they need to be protected, i.e. add an extra $ on there, so perl -e '$$/ = undef; $$_ = <>; s/\n{3}//g; print' input.txt > output.txt should work. –  Thor Jul 28 '12 at 15:52
    
Yes, had figured that out, but still not working. Getting error: Can't modify constant item in scalar assignment at -e line 1, near "<>;" Execution of -e aborted due to compilation errors. Can you pipe into perl? Why is it executing -e? –  WilliamKF Jul 28 '12 at 16:14
    
The error you're getting comes from this bit: _ = <>;, should be $_ = <>;. Either of perl -e ... infile and cat infile | perl -e ... works. -e specifies the command perl should run, see perlrun(1). –  Thor Jul 28 '12 at 17:02
2  
perl -0777pe 's/\n{3}//g' is exactly the same. 0777 puts Perl into file-slurp mode, and p makes a read loop which prints –  Peter.O Jul 28 '12 at 21:32
show 2 more comments
up vote 0 down vote accepted

For those interested, here is the actual makefile rule (I couldn't get the newlines to work in the original, so used a comment to add the documentation):

# 0777  -  Enables slurp mode (i.e. $/ = undef;  -  Slurp mode, read whole file
#                                   $_ = <>;     -  Read file to $_)
# s/\n{3}//g;                                    -  Remove occurrences of \n{3}
# p     -  Implicitly print result (i.e. print;  -  Output $_)
output.txt: input.txt
    $(SED) -r $(SED_RULE) input.txt | perl -0777 -pe 's/\n{3}//g' > output.txt
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.