Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

my operating system is Arch Linux. I am trying to extract a .zip archive that contains CJK characters in its filenames. It was most likely created on a Windows machine.

I tried the unzip utility and it produced invalid symbols. The same with 7za, but with slightly different ones. My LANG variable was set to en_US.UTF-8, but setting it to ja_JP.ujis seems to have no effect. I'm assuming that this means that the CJK filenames were encoded into the archive in a format other than UTF-8, and I need to convert it to UTF-8 for them to display properly.

I know of convmv, and I used a shell script to test every possible encoding from convmv --list to no avail. I have the Unicode equivalents of the filenames for the most part, but in a format that's cumbersome to manually rename all of them, but with them I can verify if the conversion was successful or not.

Observing the hex dump of ls and with positional deduction I concluded that U+4EBA (人) is represented as 0xC9 0x6C with the unzip output and 0xC2 0x90 0x6C with the 7za output. This also means it isn't impossible that I'm not dealing with the original encoding in the first place.

So, why would two zip extractors produce different results, and are there any other leads to help me convert these filenames to UTF-8 correctly?

share|improve this question
    
I can't find any codec that would encode to 0xC9 0x6C. –  Ignacio Vazquez-Abrams Apr 14 '11 at 1:28

1 Answer 1

My first guess, when dealing with UTF8 pathnames, is to attempt to use the Python zipfile library - I'm guessing that its cross-platform enough for your needs (OTOH, the module docs mention nothing about UTF8...).

Here is a small script to attempt this:

#!/usr/bin/python
import zipfile
import sys
import os

if len(sys.argv) < 2:
    print "I require a file name and a directory to unzip to"
    sys.exit()
zip = zipfile.ZipFile(sys.argv[1])
if not os.path.exists(sys.argv[2]):
    os.mkdir(sys.argv[2])

zip.extractall(sys.argv[2])

This can be chmod +x'd and run - see if it works in your case.

In all infinite improbability, this will solve your problem.

share|improve this answer
    
AttributeError: 'module' object has no attribute 'Zipfile' –  kaykun Apr 14 '11 at 1:57
    
This is basically what Info-ZIP already does, and that's producing bogus output. –  Ignacio Vazquez-Abrams Apr 14 '11 at 1:58
    
@kaykun Apologies for the typo, it's zipfile.ZipFile. @Ignacio Do they use the same decoder? –  new123456 Apr 14 '11 at 2:06
    
人 is now 0xC3 0x89 0x6C with this method. –  kaykun Apr 14 '11 at 2:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.