I have a string with comma separated values, like:


As you can see, the 3rd comma separated value has sometimes special character, like the dash (-), in the end. I want to used sed, or preferably perl command to replace this string (with the -i option, so as to replace at existing file), with same string at the same place (i.e. 3rd comma separated value) but without the special character (like the dash (-)) at the end of the string. So, result at above example string should be:


Since such multiple lines like the above are inside a file, I am using while loop at shell/bash script to loop and manipulate all lines of the file. And I have assigned the above string values to variables, so as to replace them using perl. So, my while loop is:

while read mystr
myNEWstr=$(echo $mystr | sed s/[_.-]$// | sed s/[__]$// | sed s/[_.-]$//)
perl -pi -e "s/\b$mystr\b/$myNEWstr/g" myFinalFile.txt
done < myInputFile.txt


$mystr is the "SOME-STRING_A_-BLAHBLAH_1-4MP0-"
$myNEWstr result is the "SOME-STRING_A_-BLAHBLAH_1-4MP0"

Note that the myInputFile.txt is a file that contains the 3rd comma separated values of the myFinalFile.txt, so that those EXACT string values ($mystr) will be checked for special characters in the end, like underscore, dash, dot, double-underscore, and if they exist to be removed and form the new string ($myNEWstr), then finally that new string ($myNEWstr) to be replaced at the myFinalFile.txt, so as to have the resulting strings like the example final string shown above, i.e. with the 3rd comma separated sub-string value WITHOUT the special character in the end (which is dash (-) at above example).

After investigation and much testing, I have concluded that problem is that at such cases for some reason the $mystr cannot be parsed to be replaced, because, for example, it has the special character at the end of the existing string in file (dash (-) at our above example)!!

So, I am desperately looking for a way to replace such string with special character in the end, with the same string WITHOUT the special character in the end at the file and at the string line in question, at exactly the same place, i.e. at the 3rd comma separated value of the string line.

Thank you.


This perl one-liner does the job:

perl -ape 's/^(?:[^,]+,){2}[^,]+\K-(?=,)//;' inputfile > outputfile

or, if you want to replace in-place:

perl -i.backup -ape 's/^(?:[^,]+,){2}[^,]+\K-(?=,)//;' file


s/          # substitute
  ^             # beginning of line
    (?:         # non capture group
      [^,]+     # 1 or more non comma
      ,         # a comma
    ){2}        # end group, must appear twice
    [^,]+?      # 1 or more non comma
    \K          # forget all we have seen until this position
    -           # an hyphen
    (?=,)       # positive lookahead, make sure we have a comma after
//          # with nothing
  • I tried that but did nothing. Please note the following that I have said: "Note that the myInputFile.txt is a file that contains the 3rd comma separated values of the myFinalFile.txt". – Kostas75 Oct 25 '19 at 14:48
  • @Kostas75: This perl one-liner does the whole job in a single pass, from your original file to the final file, no needs for bash, sed or any other thing. – Toto Oct 25 '19 at 14:56
  • I understand what you said, but unfortunately I tried your perl 1-liner and does not work. – Kostas75 Oct 28 '19 at 21:21
  • @Kostas75: In which way it doesn't work? What do you get? Do you have some error messages? How and where do you run it? What is your Perl version (perl -v)? – Toto Oct 29 '19 at 9:34
  • No error msgs, however result file is same with source file. Perl version: v5.8.3 built for IA64.ARCHREV_0-thread-multi – Kostas75 Oct 29 '19 at 15:11

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.