I want to change characters in a XML file by using sed. The input looks like this:

<!-- Input -->
  <tree foo="abcd" bar="abccdcd" />
  <dontTouch foo="asd" bar="abc" />

Now I want to change all c to X in the bar tag of the tree element.

<!-- Output -->
  <tree foo="abcd" bar="abXXdXd" />
  <dontTouch foo="asd" bar="abc" />

How is the correct sed command? Please consider, there can be more than one occurence of c (next to each other or not) in one tag...

I tried this myself, but it won't change multiple c, and it does append a X :(

sed -i 's/\(<tree.*bar=\".*\)c\(.*\"\/>\)/\1X\2/g' Input.xml

Edit: Some more details ;)

  • This is a once in a life time job, after the document is changed, I won't touch it ever again

  • The structure is as easy as above. That means, I can grab all lines (this works) with:

    cat input.xml | grep ""

So assuming I have the correct string extracted, and know where to write it after modification: How to change 'abcdeccd' to 'abXdeXXd'? This isn't really a XML problem but a regex one, or am I wrong here?

  • 3
    The usual advice is to use an XML parser to parse XML. Regular expressions are not the best tool for this and need a lot of care in construction. – RedGrittyBrick Mar 28 '12 at 9:32
  • 1
    what @RedGrittyBrick is trying to say is that its a terrible idea. Don't bother trying. – soandos Mar 28 '12 at 9:38
  • As the file is really big, I don't want to use a parser. I just want to replace this c with X. Can't be too hard? – powerbar Mar 28 '12 at 10:00
  • And even if I use a XML parser, I would need a regex to change abccdcd to abXXdXd... – powerbar Mar 28 '12 at 10:16
  • "Can't be too hard?" - there are endless cases where it can fail. If the input is very regular, then it might work, but once you run into a caveat, you either have to add mind-bending logic to the replacement regexpes or scrap the solution altogether to make a correct one. Learning how to make the correct one directly will save time and aggravation in the future. – Daniel Andersson Mar 28 '12 at 10:43

This might work for you (GNU sed?):

sed '/^\s*<tree.*\<bar="/!b;s//&\n/;:a;s/\n\([^c"]\+\)/\1\n/;ta;s/\nc/X\n/;ta;:b;s/\n//' XML

As RedGrittyBrick said, the best way to do it is using an XML parser, picking out the element, translate characters and then write it back using an XML library. This will not give you nasty surprises, it will stand the test of time, etc. It is not only best, it is far superior to other things. Other solutions more or less instantly become nightmares to debug, and there will certainly be hidden problems more or less everywhere.

If it's just a simple task that needs to be done once, and one is very careful, and one checks the result, etc., etc., etc., then it might be less work to do it the bad way. But it will surprise you some day if you make it a habit.

As example, here is one of the bad ways that seem to work, but it relies not just on valid XML, but the more or less exact syntax you described earlier, which is just a subset of valid XML, and thus valid XML is certainly able to make the code fail (what if someone adds a '>' sign in one of the tags? Add a special case. What if someone doesn't use quotation marks? Add a special case, and so on). This is the problem of not using a real parser. Some care has been taken below to act like a pseudoparser at least, reading the tag, then acting on it, then writing it back, but there are ready tools for this that have been tested extensively.

while read i; do
    if $(printf -- "${i}" | grep -qE '<tree [^>]+ bar="[^'"${1}"'"]*'"${1}"); then
        ORIGTAG=$(printf -- "${i}" | sed 's#^.*<tree [^>]\+ bar="\([^"]\+\)".*$#\1#g')
        NEWTAG=$(printf -- "${ORIGTAG}" | tr "${1}" "${2}")
        printf -- "${i}\n" | sed 's#\(^.*<tree [^>]\+ bar="\)'"${ORIGTAG}"'\(".*$\)#\1'"${NEWTAG}"'\2#g'
        printf -- "${i}\n"
done < "${3}"

Usage: script.sh [character to replace] [replacing character] [filename], e.g.

script.sh c X myfile

IFS sets the "internal field separator" in the shell to newline, to keep whitespace in the beginning of the lines.

while read reads the input file (given as argument 3 to the script) line by line.

grep checks if the specific tag is in the current line AND if the tag contains the character to be translated. If so, go to sed logic; if not, return the line as-is.

sed picks out the old tag, runs a character translation on it and returns the line with the new tag.

As you can see, no one would like to find this script and have to debug it. If this is anything else than a one-off job, don't do it like this. For the sanity of future observers.

  • Thank you for you answer, I edited my question to make some stuff clearer: It really is a once in a lift time job. And I can test results... I am just looking for a quick and dirty solution like yours. I will test and report. – powerbar Mar 28 '12 at 11:56
  • First thing I notice: You solution isn't too fast, the input file has some 100MB and the written output is 10mb after 15 minutes... – powerbar Mar 28 '12 at 12:12
  • grep is invoked once for every line, which with many lines will take time. It is complex to correctly identify the wanted tag this way. For a faster solution one should use e.g. Perl/Python, but if it's a one time job, the coding time will probably be longer than the program time. I hope that you tested the code on a subset first so you know that it works for you. It worked on your minimal given example, at least. One could enhance the script to only run the full grep regexp on portions where the tag could be found, but then we are reinventing the SAX parser, bit by bit, badly. – Daniel Andersson Mar 28 '12 at 12:44
  • Output looks quite good. I think your solution did it - but I really thought this would be much easier... – powerbar Mar 28 '12 at 12:51
  • It's tricky to make sure that only the correct parts of the file will be changed. Can't just change all 'c':s to 'X', obviously. Can't do it in the wrong tags. Can't do it with the wrong arguments. After the correct argument has been found, we must know when to stop, etc. XML is not meant to be processed as text in this way, it's meant to be used with a parser. Then you get structured data that can be acted on in an isolated way. Had the data already been "liberated" in an SQL database or similar, this would have been trivial, but the sed way with at least some robustness is kludgier. – Daniel Andersson Mar 28 '12 at 12:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.