I've completely rewrote this question as I didn't describe it very well in my first attempt, but it is still the same question. Let's say you have a bunch of files of varying size and need to put them on discs to back them up. These files do not need any directories preserved and do not need to be stored in any particular order. The end goal is just to leave the smallest amount of free space per disc. I do not want to use a compression program like 7zip to split an archive across the discs because if one disc is lost/damaged then all the data is unreadable. This is a question simply to learn HOW you would do this, I am aware storage is cheap and that this process could be time consuming. I'm just curious and want to learn how this would be done. As a very basic example look below;

I want to store these six files totaling 89MiB on two 50MiB discs. This is a very basic example. Increasing the number of files and having more random sizes is what seems to make this impossible, from a mental equation standpoint at least.
A.7z - 10MiB
B.7z - 20MiB
C.7z - 30MiB
D.7z - 10MiB
E.7z - 07MiB
F.7z - 12MiB
If I went by ABC order these files would need a third disc. However if file B and C go together on the first and A, D, E, and F go on the second it only takes two discs.

  • You are asking an off-topic question (software shopping). Questions seeking product, service, or learning material recommendations are off-topic. See On Topic. Try softwarerecs.stackexchange.com but please first read What is required for a question to contain "enough information". – DavidPostill Jun 1 '17 at 23:45
  • Edited to remove software references. – herculeesjr Jun 1 '17 at 23:49
  • This is a relatively simple issue: You put the files on the disk. If the files are of a type that will compress well, you may benefit from compressing them, but if they are not, there is little to be gained from compression. As your needs are only for a few months, discs should work fine, but have you checked out the cost of purchasing large and inexpensive USB memory sticks? How does that compare with BD prices? – music2myear Jun 1 '17 at 23:57
  • Linux tool fpart may help. I have never used it so I'm not sure. Do your own research. – Kamil Maciorowski Jun 1 '17 at 23:57
  • herculeesjr, I wouldn't spend time trying to fit the files the most densely. Instead, buy more storage. Optical disks are cheap, external hard disks are cheap, your time is not, and losing files or misplacing files is not. When using archives, don't make a single huge archive and split it, because then if you lose a single piece, it's all lost. Use sensible file management. Sort your files into folders, then archive the folders. You can burn the archived folders to optical disk, and mark the disks using permanent marker. Redundant backups (making two backup copies) are good too. – Christopher Hostage Jun 2 '17 at 0:05

There are conceivably better ways of doing this -particularly in edge cases, but I imagine this would provide a fairly tight fit for the generic case.

  1. Take a copy of all the files and compress them individually using the best compression program you can use. (7z or xz ?)

  2. Looking at the compressed files, Find the largest compressed file which will fit on the disk, then the largest compressed file which will fit and repeat until there is no space left. Compress the uncompressed versions of each of these files onto the disk as a single archive. If there the space left over is no smaller then any remaining compressed files, Redo this process adding more uncompressed versions of compressed files. (Its possible that compressing multiple files will result in better compression then the original files - both because of wasted space at the end of the files, and better dictionaries enabling better compression when more data is compressed).

  3. Remove compressed individual files which have already been compressed onto a finalised archive.

  4. Repeat steps 2-3 until there are no more files left.

I observe that compressing the files individually and then copying them to disks using the biggest first principle, and provided that the disks are blank, will give you a much greater possibility of retrieving data from partially corrupted. This could be a good compromise if the files are not tiny (in which case wasted disk space will be an issue) You would want to ensure that the compression program you use has a signature which can be identified by programs like photorec.

Depending on the disk you are copying to and size of the resulting files, you may want to either use the disk without formatting for a single large archive (harder to retrieve, but maximum space) or reduce the block size to reduce wasted space padding after small files.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.