Create a zip/tgz in Linux such that Windows han proper filenames?

Question

How can I create a zip/tgz archive so Windows users can extract it will all filenames encoded properly?

Currently, tar -zcf arch.tgz files/* encodes filenames in UTF, so Windows users see all characters spoiled in filenames which are not english, and can do nothing with it.

zip -qq -r arch.zip files/* has the same behavior.

Answer 1

Currently, tar encodes filenames in UTF

Actually tar doesn't encode/decode filenames at all, It simply copies them out of the filesystem as-is. If your locale is UTF-8-based (as in many modern Linux distros), that'll be UTF-8. Unfortunately the system codepage of a Windows box is never UTF-8, so the names will always be mangled except on tools such as WinRAR that allow the charset used to be changed.

So it is impossible to create a ZIP file with non-ASCII filenames that work across different countries' releases of Windows and their built-in compressed folder support.

It is a shortcoming of the tar and zip formats that there is no fixed or supplied encoding information, so non-ASCII characters will always been non-portable. If you need a non-ASCII archive format you'll have to use one of the newer formats, such as recent 7z or rar. Unfortunately these are still wonky; in 7zip you need the -mcu switch, and rar still won't use UTF-8 unless it detects characters not in the codepage.

Basically it's a horrible mess and if you can avoid distributing archives containing filenames with non-ASCII characters you'll be much better off.

Answer 2

Here is a simple Python script that I've written to unpack tar files from UNIX on Windows:

import tarfile

archive_name = "archive_name.tar"

def recover(name):
    return unicode(name, 'utf-8')

tar = tarfile.open(name=archive_name, mode='r', bufsize=16*1024)
updated = []
for m in tar.getmembers():
    m.name = recover(m.name)
    updated.append(m)

tar.extractall(members=updated)
tar.close()

Answer 3

I believe you're running into problems with the Zip container format itself. Tar may be suffering from the same problem.

Use the 7zip (.7z) or RAR (.rar) archive formats instead. Both are available for Windows and Linux; the p7zip software handles both formats.

I just tested creating .7z, .rar, .zip, and .tar files on both WinXP and Debian 5, and the .7z and .rar files store/restore filenames correctly while the .zip and .tar files don't. It doesn't matter which system is used to create the test archive.

Answer 4

POSIX-1.2001 specified how TAR uses UTF-8.

As of 2007, changelog version 6.3.0 in the PKZIP APPNOTE.TXT (http://www.pkware.com/documents/casestudies/APPNOTE.TXT) specified how ZIP uses UTF-8.

It's only which tools support these standards properly, that remains an open question.

Answer 5

As far as I know, Windows NT series (XP, Vista, others) use Unicode for storing the file names. So I'm guessing it's a problem with the Windows' archiver... which one are you using?