I have a file in the format:


Now if data has a " in it, then my parser cannot parse the file. So I want to use sed to replace " with """ but not where it sees "," as that is essentially the delimiter.

I can't figure out what the sed parameters should be.

  • What if the data contain ","? – choroba Apr 17 '13 at 13:26
  • @choroba there is nothing I can do about that case. It is not a regular language, but a CFG (think I got my terminology correct there) – Cheetah Apr 17 '13 at 13:34
  • Why not three steps: first change "," to something like ~. Then change " to """. Then change ~ back to "," . (if needed, use something instead of ~ that won't appear in your data) – yosh m Apr 17 '13 at 13:59

I am sure you can also do this in sed but it is much easier for me in Perl:

perl -pe 's/([^,])\"/$1xxx/g; s/xxx,/\",/g; s/xxx/\"\"\"/g' data.txt


  • perl -pe : execute the script given on the command line on each line of the input file
  • s/([^,])\"/$1xxx/g; : substitute every occurrence of " preceded by any non-comma character with a random string xxx. This assumes that your data will never contain xxx.
  • s/xxx,/\",/g; : substitute those xxx that are followed by a comma back to ",.
  • s/xxx/\\\"/g : replace the rest of the xxx with escaped quotes \", these will be the " that were in your data.


$ cat test.txt
"data","da"t"a","data","data","foo "bar" foo", "data"

$ perl -pe 's/([^,])\"/$1xxx/g; s/xxx,/\",/g; s/xxx//g' test.txt 
"data","data","data","data","foo bar foo","data
  • I don't want to remove them, just escape them! – Cheetah Apr 17 '13 at 16:14
  • @Ben, OK, modified the answer to escape instead of remove. – terdon Apr 17 '13 at 16:17
  • thanks, I actually got the picture from the excellent explanation you gave...just didn't want to accept the answer until it answered the question...I hate to nit-pick, but the escape is a double double quote not a backslash as per the op – Cheetah Apr 17 '13 at 16:22
  • You're not nitpicking @Ben, I should learn to read questions. Thanks for pointing it out, it should work now. – terdon Apr 17 '13 at 16:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.