0

I want to create an archive, through a program I am writing in C#, and later decompress it.

The archive will contain folders from completely different locations, grouped on 3 major subdivisions.

Basically, group A can have folders \x\y\z, \x\y\m, \x\p\q, group B can have folders \x\p\r, \x\y\t... So the original location can give only slight indication of the division, I can't use the path for grouping.

I was hoping that I can use 7z options to place folders in different folders inside the zip, but I couldn't find any such options, so I thought I will create 3 zips inside one big zip.

The problem I see is, the amount of data is big (hundreds, thousands of GB) and as far as I know, zip over zip is not a great idea. (data types: mostly image type, could be tif or other uncompressed formats, but may also be jpg)

It would be actually great if I could do zip over rar... But 7z doesn't use rar on compression, only on decompression.

I don't want to have to install multiple programs - I will only install 7z.

7z has, for packing, the following options (from their site): 7z, XZ, BZIP2, GZIP, TAR, ZIP and WIM. I would use one of them for the inner archives and then the 7z option for the outer archive.

What is the best option ? (to get a better archive that doesn't create a bigger zip than by just doing it all in one pass)

Edit: Windows !

7
  • It's not clear what you're asking, if in fact there's a question.
    – user3463
    Jul 23, 2012 at 17:17
  • edited, I hope I made it clear
    – Thalia
    Jul 23, 2012 at 17:21
  • I'd do tar.gz, honestly. Create a tarball of each of the groups you need, and then gzip the whole lot. However, this is going to take a very long time due to the size of the archives.
    – user3463
    Jul 23, 2012 at 17:24
  • 1
    I'd do tar.xz instead - xzip uses essentially the same LZMA-based algorithm as 7z, which compresses a large corpus a lot better, especially if there's a lot of repeated data across different files. If you're going to use 7zip, use its own .7z format, for the same reason.
    – fluffy
    Jul 23, 2012 at 17:28
  • 2
    Most of you seem to have lost sight of what the actual problem/question was in the OP. The "problem" as he stated it was that he wanted to divide separate topics up by subfolder within the archive, but since he didn't see a way to do that using the 7z command, he thought he'd resort to nested zip files. The nested zip files is NOT a requirement of his question, only an artifact of his confusion about archive folder structure. Jul 23, 2012 at 17:35

1 Answer 1

3

If you're on Linux, you can create symbolic links to the folder paths on your hard drive and add those folders to the archive. Heck, you can do this on Windows with symbolic links, too.

So, example:

You have folders /a, /b, /c and /d (on Windows this would be C:\a, etc.)

Your desired folder structure inside the archive is:

/SUBDIVISION-1/a
/SUBDIVISION-1/b
/SUBDIVISION-2/c
/SUBDIVISION-3/d

or some arbitrary split of the folders between subdivisions.

So when you go to compress, you'd just

mkdir /SUBDIVISION-1
mkdir /SUBDIVISION-2
mkdir /SUBDIVISION-3

Then

ln -s /a /SUBDIVISION-1/a
ln -s /b /SUBDIVISION-1/b
ln -s /c /SUBDIVISION-2/c
ln -s /d /SUBDIVISION-3/d

and you can have an arbitrary number of directories within each subdivision, from 0 to 1 to N.

On Windows you can't use the ln command; you'll have to use the mklink command instead.

You'll need a program that's smart enough to follow symbolic links (on Linux), but on Windows, it should be automatic... the archive shouldn't just plop a symlink in there; it should have the data found after dereferencing the symlink. At least that's how it should be if you use the Win32 subsystem (which I infer 7z does).

Windows version upon implicit request from OP (from cmd.exe shell):

mkdir C:\SUBDIVISION-1
mkdir C:\SUBDIVISION-2
mkdir C:\SUBDIVISION-3

Then

mklink /d C:\SUBDIVISION-1\a C:\a
mklink /d C:\SUBDIVISION-1\b C:\b
mklink /d C:\SUBDIVISION-2\c C:\c
mklink /d C:\SUBDIVISION-3\d C:\d
8
  • Windows :-( Edit my question too. Thanks, I'll try this, not sure how from my program, but it is what I want !
    – Thalia
    Jul 23, 2012 at 17:27
  • It should work fine on Windows, you'll just have to use different paths and a different symlink command. Jul 23, 2012 at 17:28
  • After testing a number of combinations, I found out that I must be an administrator in order to be able to create symbolic links... (???)
    – Thalia
    Jul 23, 2012 at 20:29
  • Yep, correct. Or install Cygwin and use its symbolic link emulation and the 7za compressor from there.... Jul 23, 2012 at 20:33
  • 1
    Never call a command from a program when you can do it from code... adding an external dependency like that makes your code fragile and you depend on the existence of files, you run into potential errors with file paths, you run into issues with the downstream program parsing command line options, etc. Stick to code, APIs that you can control and concretely see and debug. Jul 23, 2012 at 21:16

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .