Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to search for a specific string across multiple lines in a file and get the line in which the match was found.

However, my problem is, that the file contains one very long string, and not, i.e. words, and I want to search for a sub-sequence of this long string. Hence, I can not use pcregrep and just search for word1\nword2. Because I actually want to get the line number in which the match was found, I can not just remove all newline characters...

This is an example how my file looks like, I just capitalized the matching string so you could find it:

String to search:

gcbcdbfceebcfhfchaaccdgfcegffgedffaeaedcbaedhacebeeebcechbcbfeeccbdhcbfg

File to search in:

abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde
abcdeabcde***GCBCDBFCEEBCFHFCHAACCDGFCEGFFGEDFFAEAEDC
BAEDHACEBEEEBCECHBCBFEECCBDHCBFG***ggfbhbgcedabceedfa
fbaaechaabdbffbebecebaacfcfcdcggfchddcefbcbdegbbba

Does someone of you have an easy solution for this?

If there is no tool at hand to do it, I would just write a short python script to do it, but I guess any bash tool would be more efficient than that ...

EDIT:

Thank you very much for your answers, they work very well, if the position of the newline character is known.

However, I am sorry for being imprecise in my question. My problem is, that I do not know if there is a newline, or even more than one newline, within the string in the file and moreover, I do not know where it is. I corrected my search string by deleting the newline I had unwittingly inserted.

Is there some way to allow for a newline in any position of the string?

share|improve this question

3 Answers 3

up vote 1 down vote accepted

I'm adding a new answer now that I understand the problem better. I'm only posting this as a working example but I don't claim that it is a good one. :)

Also, I understand that the question seemed to want to NOT use Python because of some fears of inefficiencies. So I understand that this approach doesn't fulfill the entire request. :(

#!/usr/bin/env python
import sys

def findall_iter(S, pat):
  index = -1
  while True:
    try:
      index = S.index(pat, index+1)
      yield index
    except ValueError:
      raise StopIteration

def findall(S, pat):
  return list(findall_iter(S, pat))

# read in arguments
S = open(sys.argv[2]).read()
pattern = sys.argv[1]

# get indices of all newlines
newline_indices = findall(S, '\n')

# get psudo-indices of all pattern matches
pat_indices = findall(S.replace('\n', ''), pattern)

# iterate through each pattern match psudo-index and
# correlate it back to a real line number from the file
line_numbers = []
for pi in pat_indices:
  for i, ni in enumerate(newline_indices):
    if ni > pi+i:
      line = i + 1
      if line not in line_numbers:
        line_numbers.append(i+1)
      break

print '\n'.join(map(str, line_numbers))

Pros:

  • If the file is not too large (<1GB) all operations are performed in memory.
  • Uses the str.index method to find substrings instead of (slower) regular expression matching
  • Clearer than using regular expressions

Cons:

  • Won't work well with large files.
  • Creates two temporary strings to do the job.
  • The last for-loop is hard to understand.
  • Is Python (which I personally do not think is a con).
share|improve this answer
    
Hi Dave, thanks for your effort. This python script actually does exactly what I want and it is really quick! I tested it on a 400 MB text file and it took just a couple of seconds. The python script I wrote was more than 10 times slower. (I used a couple of more loops, probably that's the reason...) Thanks again to everyone who helped! –  Tabea Apr 2 '13 at 9:19
    
Glad I could help. You could add larger file support into it by doing some buffering tricks. But that would take more time and make the code less readable. –  Dave Apr 3 '13 at 19:02

I would do this with a sed script. Put this in a file, then use sed -nf to run it.

:restart
/gcbcdbfceebcfhfchaaccdgfcegffgedffaeaedc$/{
    #   Found the first part, now discard it
    s/^.*$//
    #   Read a new line into the buffer
    N
    #   Discard the new line inserted by the N operation
    s/^\n//
    #   If next line isn't a match, start over
    /^baedhacebeeebcechbcbfeeccbdhcbfg/!b restart
    #   If it is a match, print the line number
    =
    }

Here's what it looks like to run it under bash. Notice that it prints the line number of the second line matched.

bash-4.1$ cat sample.txt
abcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcdeabcde
abcdeabcde***gcbcdbfceebcfhfchaaccdgfcegffgedffaeaedc
baedhacebeeebcechbcbfeeccbdhcbfg***ggfbhbgcedabceedfa
fbaaechaabdbffbebecebaacfcfcdcggfchddcefbcbdegbbba
bash-4.1$
bash-4.1$ cat findmatch.sed
:restart
/gcbcdbfceebcfhfchaaccdgfcegffgedffaeaedc$/{
   #  Found the first part, now discard it
   s/^.*$//
   #  Read a new line into the buffer
   N
   #  Discard the new line inserted by the N operation
   s/^\n//
   #  If next line isn't a match, start over
   /^baedhacebeeebcechbcbfeeccbdhcbfg/!b restart
   #  If it is a match, print the line number
   =
   }
bash-4.1$
bash-4.1$ sed -nf findmatch.sed sample.txt
3
bash-4.1$
share|improve this answer

I'm a little confused as to what constraints you are operating under. However, if you need the line number, both grep and pcregrep can give it to you ala the -n flag.

$ pcregrep -nM "gcbcdbfceebcfhfchaaccdgfcegffgedffaeaedc\nbaedhacebeeebcechbcbfeeccbdhcbfg" | cut -d: -f1
2
baedhacebeeebcechbcbfeeccbdhcbfg***ggfbhbgcedabceedfa

pcregrep shows only the number of the first line matched, apparently so you'll have to skip every other line of the output with sed ( pipe the above to sed -n 'p;N') if you want only the line numbers as output.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.