Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a large amount of log files that I need to remove sensitive data from. The sensitive data is provided to me in a text file and is prone to change.

I had hoped to do the equivalent of this:

#!/usr/bin/bash
pattern=""
for val in 'sed -e 's/.*=//' Client_clean.txt
do
      pattern=$pattern$val"|"
done
#egrep -e $pattern $1
sed -i 's/$pattern/CLIENT/g' $1
exit 0

The commented out egrep works fine, the sed doesn't.

Am I right to use sed for this? Or is there a more apt route to take?

Any help appreciated.

Steve

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

To be able to expand a shell variable in the sed command, you need to use double quotes.

You either need to use the -r option to sed or precede the pipe characters with backslashes.

You also need to avoid adding a pipe character at the end. If you have it there, empty strings will be matched.

#!/usr/bin/bash
pattern=""
delim=""
for val in 'sed -e 's/.*=//' Client_clean.txt
do
      pattern=$pattern$delim$val
      # setting delim after its first use and using it before the new value
      # results in "foo|bar|baz" instead of "foo|bar|baz|"
      delim='|'
done
sed -ri "s/$pattern/CLIENT/g" "$1"
share|improve this answer
    
Thanks so much, worked like a charm. I hadn't realized the problem with the delim character at the end either, so thanks for that too. –  Steve Feb 21 '11 at 19:20
add comment

Yes, sed is the way to go.

What I see wrong in your sed command are the quotes. the $pattern won't be replaced with it's contents if you use ' for quoting. Instead of using ' you should use " :

sed -ri "s/$pattern/CLIENT/g" $1

You also need "-r" for extended regular expressions.

And pattern=$pattern$val"|" should probably be

pattern=$pattern"|"$val
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.