Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a file in the format:

"data","data","data",data".

Now if data has a " in it, then my parser cannot parse the file. So I want to use sed to replace " with """ but not where it sees "," as that is essentially the delimiter.

I can't figure out what the sed parameters should be.

share|improve this question
    
What if the data contain ","? – choroba Apr 17 '13 at 13:26
    
@choroba there is nothing I can do about that case. It is not a regular language, but a CFG (think I got my terminology correct there) – Cheetah Apr 17 '13 at 13:34
    
Why not three steps: first change "," to something like ~. Then change " to """. Then change ~ back to "," . (if needed, use something instead of ~ that won't appear in your data) – yosh m Apr 17 '13 at 13:59
up vote 2 down vote accepted

I am sure you can also do this in sed but it is much easier for me in Perl:

perl -pe 's/([^,])\"/$1xxx/g; s/xxx,/\",/g; s/xxx/\"\"\"/g' data.txt

EXPLANATION:

  • perl -pe : execute the script given on the command line on each line of the input file
  • s/([^,])\"/$1xxx/g; : substitute every occurrence of " preceded by any non-comma character with a random string xxx. This assumes that your data will never contain xxx.
  • s/xxx,/\",/g; : substitute those xxx that are followed by a comma back to ",.
  • s/xxx/\\\"/g : replace the rest of the xxx with escaped quotes \", these will be the " that were in your data.

EXAMPLE:

$ cat test.txt
"data","da"t"a","data","data","foo "bar" foo", "data"

$ perl -pe 's/([^,])\"/$1xxx/g; s/xxx,/\",/g; s/xxx//g' test.txt 
"data","data","data","data","foo bar foo","data
share|improve this answer
    
I don't want to remove them, just escape them! – Cheetah Apr 17 '13 at 16:14
    
@Ben, OK, modified the answer to escape instead of remove. – terdon Apr 17 '13 at 16:17
    
thanks, I actually got the picture from the excellent explanation you gave...just didn't want to accept the answer until it answered the question...I hate to nit-pick, but the escape is a double double quote not a backslash as per the op – Cheetah Apr 17 '13 at 16:22
    
You're not nitpicking @Ben, I should learn to read questions. Thanks for pointing it out, it should work now. – terdon Apr 17 '13 at 16:24

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .