I have some data like the following

"brand": "Some brand name  ", "otherbrand": "Something else ", 

"brand":"Some  third way"

although a little bit more unstructured like that, however one structure that is the same is that there are recurring instances of "brand": around, with the following value having inconvenient whitespace.

What I need to do is to normalize whitespace, and to trim it in the value so that

"brand": "Some brand name  "

gets turned in to

"brand": "Some brand name"


"brand":"Some  third way"

gets turned in to

"brand":"Some third way"

the size of the file is

I would like to know how to do this with Sed, because I think I am going to have to do this kind of minor fixing in the data here and there, and I would like a way to do it ad hoc with just a line in the terminal.

  • Is this specific to the "brand" value? – glenn jackman Mar 18 '15 at 15:07

This should work:

sed 's/ *"/"/g;s/  */ /g'

The first part removes any spaces that are trailing within quotes, and the second cuts down any multiple spaces to a single space:

 $ echo "Before:"; cat $TEMP/test.txt; echo "After:";sed 's/ *"/"/g;s/  */ /g' $TEMP/test.txt
"brand": "Some brand name  ", "otherbrand": "Something else ",
"brand":"Some  third way"
"brand":"Some brand name","otherbrand":"Something else",
"brand":"Some third way"
  • is it just my test, or does this result in an extra space after "some brand name " and the following comma? i get: 'brand: Some brand name , otherbrand: Something else' – Daniel Mar 18 '15 at 15:42
  • @Daniel i've added example output, I think it's ok – gogoud Mar 18 '15 at 15:54
  • it does look correct -- mine gives an extra space. version of sed is 4.2.1 – Daniel Mar 18 '15 at 16:21
  • @Daniel: I'm using sed (GNU sed) 4.2.2. – gogoud Mar 18 '15 at 17:21
sed ':in
s/^\(\([^"]*"[^"]*"\)*[^"]*"[^"]*\) \{2,\}/\1 /
t in
s/^\(\([^"]*"[^"]*"\)*[^"]*"[^"]*\) \{1,\}"/\1"/
s/^\(\([^"]*"[^"]*"\)*[^"]*"\) \{1,\}/\1/
t edge
' YourFile
  • POSIX version so --posix -e for gnu sed
  • recursive remove multi space inside quote string (don't touch between, could be done with few modification)
  • recursive remove edge space inside quoted string (one for begin and one for ending)

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.