Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I'm trying to familiarize myself a little with Perl to use for regular expression searches in Terminal (Mac). Now, I'm not really looking to learn Perl rigourously, just trying to find out how to do some simple regular expressions.

But I can't figure out how to do this in Terminal:

I'd like to be able to match expressions over several lines, and I'll take HTML tags as an example. PLEASE NOTE, that the HTML tag is just an example of something to match, and specifically something that goes over multiple lines. Whether matching HTML with regular expressionS is a good idea or not is not the issue. I just want to understand the syntax of matching with Perl on the command line!

Say I want to match the entire ul tag here:

<ul>
 <li>item 1</li>
 <li>item 2</li>
</ul>

I would like to:

  1. Be able to match this in a file and output the match to the stdout (don't ask why, I would just want to to understand how it works :-))
  2. Be able to replace it with something else.

For matching, I found something like this (using 'start' and 'end' as an example here from a simple text file when I was testing, but please give the example for the ul tag instead:

perl -wnE 'say $1 if /(start(.*?)end)/' test.txt 

This matches a part, but only on one line. Surprisingly, adding the s at the end didn't work to make it "dotall" or "single-line mode", it still just matched one line...

For replacing, I tried something like this:

perl -pe 's/start(.*?)end/replacement text/'s test.txt

This didn't work either...

share|improve this question
    
On parsing HTML with regexes. –  grawity Apr 24 '12 at 22:02
    
Ok, sorry, I shouldn't have named my question Matching html tags... That was unfortunate. What I'm really after is how to use perl to match on the command line, and the syntax for it, and also to get matching on several lines to work with the /s option... –  Anders Svensson Apr 24 '12 at 22:29
    
Edited my question accordingly! –  Anders Svensson Apr 24 '12 at 22:32
    
On parsing HTML with regexes, mark 2. –  tchrist Aug 15 '12 at 1:28
add comment

2 Answers

up vote 2 down vote accepted

Well, here's for matching or replacing with Perl one liners. I did this in Cygwin:

The /s makes dot match new line.

The -0777 makes it apply the regular expression to the whole thing instead of line by line.

\n can match new line as well.

$ echo -e 'a\nb\nc\nd' | perl -0777 -pe 's/.*c//s'

d

user@comp ~
$ echo -e 'a\nb\nc\nd' | perl -pe 's/.*c//s'
a
b

d

Here is the other form, -ne with print $1:

user@comp ~
$ echo -e 'a\nb\nc\nd' | perl -ne 'print $1 if /(.*c)/s'
c
user@comp ~
$ echo -e 'a\nb\nc\nd' | perl -0777 -ne 'print $1 if /(.*c)/s'
a
b
c
user@comp ~
$

Some further examples

$ cat t.t
<ul>
 <li>item 1</li>
 <li>item 2</li>
</ul>

$ perl -0777 -ne 'print $1 if /\<ul\>(.*?)\<\/ul>/s' t.t

 <li>item 1</li>
 <li>item 2</li>

user@comp ~
$ perl -0777 -ne 'print $1 if /(.*)/s' t.t
<ul>
 <li>item 1</li>
 <li>item 2</li>
</ul>

user@comp ~
$

An example of Global for the -ne one (change "if" to "while"):

$ echo -e 'bbb' | perl -0777 -ne 'print $1 while /(b)/sg'
bbb

For the -pe one, just add the g at the end (/sg or /gs, same thing):

$  echo -e 'aaa' | perl -0777 -pe 's/a/z/s'
zaa

user@comp ~
$  echo -e 'aaa' | perl -0777 -pe 's/a/z/sg'
zzz
share|improve this answer
    
Perfect! Thank you! Just a minor question: why -0777? I thought the s option at the end was supposed to take care of making it match "DOTALL" and therefore include everything (possibly with a g option to take more than one match)? –  Anders Svensson Apr 25 '12 at 6:04
    
i've just updated it for some global examples. There is a difference between dot matches new line, and having the regex apply to the whole thing. If you don't have -0777 then the only new line dot could ever see, would be the \n at the end of the line but it won't see past that. Similarly without -0777, the only new line \n could match, is the only one that is there, which is the \n at the end of one line. It won't see past that, as the regex is only being applied line by line. So you can have any combination of "dot matches new line"(or not). And -0777 (or not). –  barlop Apr 25 '12 at 15:15
    
Great, thanks again! –  Anders Svensson Apr 26 '12 at 12:11
    
@AndersSvensson yeah I could see that before, you getting an answer that actually replied to your question properly wasn't looking promising! Removing the HTML might've made the question more idiot proof –  barlop Apr 26 '12 at 19:25
add comment

Here's one of a thousand reasons why you shouldn't try to parse XML or HTML using regex

$ cat su.txt
<ul>
 <li>item 1</li>
 <li>item 2</li>
</ul>

$ perl -e 'while(<>){$x.=$_;} print "$1\n" if $x=~m!<li>(.*)</li>!s;' su.txt
item 1</li>
 <li>item 2   

You may eventually think you are getting somewhere

$ perl -e 'while(<>){$x.=$_;} print "$_\n" for $x=~m!<li>(.*?)</li>!sg;' su.txt
item 1
item 2

But it is a brief illusion. Consider:

<ul>
  <li id="foo">item 1</li>
  <li>item 2
  </li>
</ul>

Thus begins your journey down the rat-hole that is regex + XML or HTML.

If you want to play around with regex, it will be a happier experience if you use some data other than XML or HTML.

If you want to parse XML use one of Perl's many XML parsing modules. Search CPAN for XML or use Google.

share|improve this answer
    
Naw, you just aren’t trying hard enough. That said, I very strongly encourage the use of parsing modules. –  tchrist Aug 15 '12 at 1:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.