0
votes
0answers
60 views

Error uncompressing file when piping curl to gunzip

I need to download a gzipped xml feed to parse in my Ruby app. It may well be quite large, 30Mb compressed, 300Mb uncompressed. So I do this: curl -s http://example.com/file.xml.gz | gunzip and read ...
2
votes
2answers
73 views

Compressing compressed .tar.gz files deterministically

I want to perform a series of recompressions of compressed files to study their properties. At first, I tried simply recompressing .zip files in a loop: for f in $(seq 1 100) do zip $f.zip -9 -v ...
1
vote
1answer
76 views

How to convert a file of format tar.gz to tar.bz2?

Like the title says, how do I do this conversion from the Linux console?
0
votes
3answers
144 views

What are good compression algorithms for delta synchronization?

When syncing large files over a slow link, it is often useful to use delta compression in order to reduce the bandwidth used. It is also useful to compress files as they take up much less space. ...
1
vote
2answers
95 views

How to extract such a compressed file?

I have been asked to extract a ZIP file having in it 3 tar files and each tar file contains around 3000 gz files. I had to extract the main ZIP file first then extract the tar files second then ...
0
votes
2answers
388 views

What is the difference in various commands for compressing files in linux?

What are the commands for compressing files in linux and what are the difference between them? (for example: tar, gzip, rar). I'm working on Linux Mint 11 and doing a script for compressing files. I ...
1
vote
1answer
2k views

Between xz, gzip, and bzip2, which compression algorithim is the most efficient?

Between xz, gzip, and bzip2, which compression algorithm gives the smallest file size and fastest speed when compressing fairly large tarballs?
2
votes
2answers
1k views

gzip several files in different directories and copy to new directory

I'm working with several files that are located in different directories that I need to compress into individual .gz files. I also need to move the compressed files to a single directory while leaving ...
0
votes
3answers
368 views

What is the equivalent unix command to a Mac OSX “Compress” menu action?

When I right-click a file in Mac OSX and select "Compress 'File Name'", what is the equivalent bash or terminal command to get the same result? Is it just gzip -9 /path/to/filename?
2
votes
1answer
468 views

Why is ZFS gzip-9 compression so much worse than actually using zip files?

I am working with some very large packet capture files (~150GB across 30 files) but because of how highly redundant packet captures are, these compress to ~7% (14.29x) of their originally file size ...
2
votes
1answer
2k views

Shell Script to GZIP files without overwrite .gz files

I'm trying to create a shell script which receive as first parameter of a path to a folder which I would like to compress all files using gzip without overwrite the .gz files that already exist in ...
8
votes
4answers
8k views

How to obtain maximum compression with .tar.gz?

The way i understand the use of tar + gzip is that tar is normally used to consolidate a grouping of files into a single file, then gzip is used to compress that file. I recently learned that tar can ...
0
votes
1answer
643 views

General Compressibility Of Various File Formats [closed]

Has anyone come across list of which file formats generally compress well (in general algorithms)? Generally, text files .txt compress well, and media-files (.mp3, .avi, etc) do not compress well. ...
22
votes
5answers
1k views

Why doesn't Gzip compression eliminate duplicate chunks of data?

I just did a little experiment where I created a tar archive with duplicate files to see if it would be compressed, to my awe, it was not! Details follow (results indented for reading pleasure): $ dd ...
0
votes
0answers
212 views

Is it possible to recover partial compressed files?

Of the most popular compression methods (7z, RAR, Zip, GZip) is there any of them that allows to retrieve the uncompressed data from a partial file? Imagine there is only one compressed file, and it ...
2
votes
1answer
1k views

Zip/gzip compression ratio much less in Linux than in Windows

I have a bunch of very huge files located on a Linux machine that I would like to compress and save some space. I have tried using the tar/gzip combination and I have noticed that the compression ...
1
vote
1answer
93 views

Guess compressability of a large file

I have a large file (~20 GB) and I want to quickly make an (approximate) estimate of how well it can compress. I want an program which can provide an immediate guess at the compressed filesize, and ...
12
votes
5answers
5k views

Fastest GZIP utility

I'm looking for the fastest gzip (or zip) utility. I have a LVM volume which 95% exists out of blank 0's, so compressing that is very easy. I'm looking for the most fastest solution, and don't really ...
1
vote
5answers
1k views

Any method to batch-compress multiple files?

Does anyone know how multiple files can be GZIP compressed? I have lots of files in a folder and I need to compress them individually. Actually, I've found a way to do it. GZIP command line binary ...
2
votes
2answers
164 views

Modern command line file compression utilities

I am frequently moving large files around from server to server. UNIXes comes with standard gzip and bzip2 compressors. However, what open options there exist beyond these, especially ones utilizing ...
0
votes
1answer
292 views

Weird GZip / Disk space issue

I'm working on a server which is using PCI Compliant partitioning rules, which means that each of the following directories has its own partition: /, /var, /boot, /usr, /tmp and /home. The server is ...
0
votes
2answers
3k views

How can I compress a large file into smaller parts?

I'm looking for a way to compress a large file (~10GB) into several files that wont exceed 150MB each. Any thoughts?
1
vote
2answers
626 views

Excluding Path While Compressing with TAR/GZ

I am compressing list of files like this: tar cvzf mycompress.tar.gz /dir1/dir2/file1.txt /dir1/dir2/file2.txt However when I uncompressed them the directory /dir1/dir2/ is still preserved. How to ...
22
votes
6answers
2k views

How can I evaluate the best choice of archive format for compressing files?

In general, I've observed the following: Linux-y files or tools use bzip2 or gzip for distributing archives Windows-y files or tools use ZIP for distributing archives Many people use 7-Zip for ...
39
votes
4answers
38k views

How to specify level of compression when using tar -zcvf?

I gzip directories very often at work. What I normally do is tar -zcvf file.tar.gz /path/to/directory Is there a way to specify the compression level here? I want to use the best compression ...
5
votes
2answers
6k views

What is the fastest compression method for a large number of files?

I need to compress a directory with around 350,000 fairly small files that amount to about 100GB total. I am using OSX and am currently using the standard "Compress" tool that converts this directory ...
9
votes
6answers
16k views

How do I uncompress vmlinuz to vmlinux?

I have already tried uncompress, gzip, and all other solutions that come up as google results and these have not worked for me. To get just the image search for the GZ signature - 1f 8b 08 00. ...
3
votes
2answers
175 views

What type of data is compressed more effectively by bzip2 than gzip/zlib?

I'm comparing compression functions, and I'm surprised by what I'm seing. Maybe I'm doing something wrong, but everything I've tested has been more compressed by gzip than bzip2! I was under the ...
4
votes
1answer
512 views

Decompressing these .gz files gives strange/unexpected results, can you get it working?

I have a collection of mailing list archive files all gzip'd, they're in a nested directory structure that starts with what appears to be a blank folder/jargon name. The files are here: ...
1
vote
4answers
4k views

How do I know the gzip compression level?

Given a gzip compressed file, how do I know what compression level (1-9) was used for it?
0
votes
2answers
3k views

How to make sure a .tar.gz file is valid and will uncompress correctly?

I have a few BIG .tar.gz files, and I want to make sure they will uncompress, but without actually doing it. If the file opens on "Archive Manager" on Ubuntu, and I can view it contents there, does ...
32
votes
5answers
19k views

Pros and cons of bzip vs gzip?

I've known gzip for years, recently I saw bzip being used at work. Are they basically equivalent, or are there significant pros and cons to one of them over the other?
1
vote
2answers
3k views

gzip for 64-bit Windows

Does anyone know any version of Gzip which supports 64-bit Windows? I could not find anything.
14
votes
5answers
37k views

Command to 'gzip' a folder?

I am new to Mac command prompt stuff. How do you create a gzip of a folder in Mac OS X? I was told by a few folks that if you want to create a gzip of a folder you should 'tar' it first and then ...
19
votes
5answers
31k views

What is the maximum compression ratio of gzip?

What is the largest size a gzip (say 10kb for the sake of an example) can be decompressed to?