0

I have many .xml files that are encoded in ISO-8859-1, and I want them all in UTF-8.

I figured out the code to convert them individually which is :

iconv -f ISO-8859-1 -t UTF-8 filename.xml > newfilename.xml

and it works fine.

What I want to know is how to write a shell script to automatically clean up all the .xml files in a directory and then replace the old files with the new converted files, but still keeping the same file-name.

I'm new to shell scripting, so help will be much appreciated.

  • This isn't safe, because it will copy the XML declaration at the start of the file unchanged. The XML declaration needs to be changed to reflect the new encoding. – Michael Kay Sep 10 '14 at 13:31
  • That can be done by using SED... sed -i 's/olddec/newdec/g' Just insert it into the for loop before the iconv line – Fegnoid Sep 11 '14 at 7:41
2

you can do this on the command line in a bash shell

for i in *.xml; do
    iconv -f ISO-8859-1 -t UTF-8 "$i" > "$i.1" && mv "$i.1" "$i"
done

this basically takes all xml file, puts them through Iconv to filname.xml.1 and then moves that file back to the original file name

had a quick look at iconv and heres what i thing it should be (if you are using the GNU version)

for i in *.xml; do
    iconv -sc -f ISO-8859-1 -t UTF-8 "$i" > "$i.1" && mv "$i.1" "$i"
done

the -s option silences warnings, and -c option omits invalid characters from output.

  • Hi Fegnoid, the code worked but it seems like it ran until the first character sequence that was incorrect, and there after deleted the rest of the lines, for example the first character that was incorrect was at line 5000, it corrected, then deleted everything after line 5000. The lines tun close to 35000 lines... any idea why its doing that ? – Tren Sep 10 '14 at 12:00
  • that might be more to do with iconv... which i have to admit i dont know very much about – Fegnoid Sep 10 '14 at 13:19
  • @Fegnoid: Try recode, it has a -force option. "With this option, irreversible or otherwise erroneous recodings are run to completion". It can also work on multiple files at a time. – RedGrittyBrick Sep 10 '14 at 13:31
  • I added the -sc option to my answer, this should let it run through to completion – Fegnoid Sep 10 '14 at 13:32

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.