Tag Info

Hot answers tagged

69

dd by default uses a very small block size -- 512 bytes (!!). That is, a lot of small reads and writes. It seems that dd, used naively in your first example, was generating a great number of network packets with a very small payload, thus reducing throughput. On the other hand, gzip is smart enough to do I/O with larger buffers. That is, a smaller number ...


62

Gzip and bzip2 are functionally equivalent. (There once was a bzip, but it seems to have completely vanished off the face of the world.) Other common compression formats are zip, rar and 7z; these three do both compression and archiving (packing multiple files into one). Here are some typical ratings in terms of speed, availability and typical compression ...


55

you have to apply the unix-philosophy to this task: one tool for each task. tarring and compression is a job for tar and gzip or bzip2, crypto is a job for either gpg or openssl: Encrypt % tar cz folder_to_encrypt | \ openssl enc -aes-256-cbc -e > out.tar.gz.enc Decrypt % openssl aes-256-cbc -d -in out.tar.gz.enc | tar xz Or using gpg % ...


51

TAR creates a single archived file out of many files, but does not compress them. Format Details A tar file is the concatenation of one or more files. Each file is preceded by a 512-byte header record. The file data is written unaltered except that its length is rounded up to a multiple of 512 bytes and the extra space is zero filled. The end of an ...


47

According to RFC 1952, the gzip file header includes the modification time of the original file (field MTIME). You can display the header in plain text1) with gzip -lv renew.log.gz: method crc date time compressed uncompressed ratio uncompressed_name defla 64263ac7 Jun 21 17:59 314 597 52.1% renew.log ...


43

Tar is the archive tool and gzip is the compression tool. In order to compress a full directory, first you need to archive it to a single file. That's what the job is tar. and then you compress the archived file. You can do both task in a single tar command with proper option. tar -czf folder_name.tar.gz folder_name/ If you don't want to make a tar archive ...


42

It very much depends ont he data being compressed. A quick test with a 1Gb file full for zeros give a compressed size of ~120Kb, so your 10Kb file could potentially expand into ~85Mbytes. If the data has low redundancy to start with, for instance the archive contains images files in a format that is compressed natively (gif, jpg, png, ...), then gzip may ...


38

If you're using Chrome, you can open the developer tools (in the wrench menu: Tools > Developer tools, or alternatively by pressing CtrlShiftI. Once you have the developer tools open, you can click on the Network tab, and do a refresh on the page. If you then click on the top entry in the list and select Headers on the right side, it'll list all the ...


36

Instead of using the gzip flag for tar, gzip the files manually after the tar process, then you can specify the compression level for the gzip program: tar -cvf files.tar /path/to/file0 /path/to/file1 ; gzip -9 files.tar Or you could use: tar cvf - /path/to/file0 /path/to/file1 | gzip -9 - > files.tar.gz The -9 in the gzip command line tells gzip to ...


35

You will get that message if, for any reason, tar can't add all of the specified files to the tar. One if the most common is not having read permission on one of the files. This could be a big problem since you are using this for backup. If you are using the -v flag, try leaving it off. This should reduce the output and let you see what is going on.


29

Package gzip has priority "required", meaning it must always be installed for the system to work properly. Probably aptitude is therefore refusing to remove it. What output did you get from the remove command? BTW: Why do you want to remove gzip? Doing so is a really bad idea... Edit: If you see gzip consuming a lot of CPU, that's because some other ...


27

Nicole Hamilton correctly notes that gzip won't find distant duplicate data due to its small dictionary size. bzip2 is similar, because it's limited to 900 KB of memory. Instead, try: LZMA/LZMA2 algorithm (xz, 7z) The LZMA algorithm is in the same family as Deflate, but uses a much larger dictionary size (customizable; default is something like 384 MB). ...


25

You have this tagged with both zip and gzip, but here's both. These are both pretty standard Unix tools, whic Mac OS X supports (as far as I know, anyway): zip file: unzip -vl file.zip Example: [23:02:22] ~/Download $ unzip -vl lightbox2.04.zip Archive: lightbox2.04.zip Length Method Size Cmpr Date Time CRC-32 Name -------- ------ ...


25

7zip will handle gzip format. It also offers lzma compression which is much better than gzip. If you want a command-line gzipper just like gzip in linux, try this


24

You want to tar your files together and gzip the resulting tar file. tar cfvz cvd.tar.gz cvd*.txt To untar the gzip'd tar file you would do: tar xfvz cvd.tar.gz -C /path/to/parent/dir This would extract your files under the /path/to/parent/dir directory


19

Gzip gzip is based on the DEFLATE algorithm, which is a combination of LZ77 and Huffman coding. It's a lossless data compression algorithm that works by transforming the input stream into compressed symbols using a dictionary built on-the-fly and watching for duplicates. But it can't find duplicates separated by more than 32K. Expecting it to spot ...


18

How about using rsync instead with the -z option enabled for compression? rsync -az --progress source_dir/* remote_host:/destination_dir This also has the added benefit that if the file already exists and has not changed on the destination, it will not be transferred.


18

You may be using the wrong tool. 'gzip' is file compressor/decompressor for files that generally have the file extension '.gz' and can not extract files from a file like 'colorbox.zip'. On linux you would use 'zip' and 'unzip' to add and extract files in a zip file. Maybe gnuwin32 has the same zip/unzip programs? Hope this helps. Jeff


17

Modern versions of tar support the xz archive format (GNU tar, since 1.22 in 2009, Busybox since 1.17.0 in 2010). It's based on lzma2, kind of like a 7-Zip version of gz. This gives better compression if you are ok with the requirement of needing xz support. tar -Jcvf file.tar.xz /path/to/directory I just found out here (basically a dupe of this ...


15

Maybe you misunderstood what the author of that post meant. The vmlinuz file contains other things besides the gzipped content, so you need to find out where the gzipped content starts. To do that, use: od -A d -t x1 vmlinuz | grep '1f 8b 08 00' What this does is to show you where in that file you can find the gzip header. The output looks like: ...


14

the problem is the argument, f argument take the next as the filename, so it must be the last of argument tar cvzf output.tgz folder or tar -cvzf output.tgz folder is the same and no take error.


14

There are a large variety of compression formats and methods available, some don't compress at all and are designed to store a number of files in one archive, and other newer experimental compressors (PAQ based) are designed to compress as aggressively as possible, regardless of the time it takes to perform said operation. You need to evaluate the features ...


14

my gzip (on ubuntu and fedora) has the --rsyncable option. So create the tarballs using: tar -c whatever/ | gzip --rsyncable > file.tar.gz


14

Although I personally have not yet used it, I think using parallel gzip could speed up things a bit: pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data.


13

i wasn't reading carefully when i tried to uninstall it. this is what it says WARNING: Performing this action will probably cause your system to break! Do NOT continue unless you know EXACTLY what you are doing! To continue, type the phrase "I am aware that this is a very bad idea": Abort.


12

Use: gzip -c -d test.tgz > test.tar The '-c' option means write to standard output (rather than modifying the input file); the '-d' option means 'decompress'. You can also use: gunzip -c test.tgz > test.tar If you have a modern enough version of GNU 'tar', you can simply use: tar -xf test.tgz If you have a slightly older version, you need to ...


12

As far as I can tell, gzip is overall faster, while bzip overall produces better (smaller) compression.


12

Or, you can tell tar to user maximum compression this way: export GZIP=-9 tar cvzf file.tar.gz /path/to/directory Additionally, to keep your envvars clutter-free, you can do this: env GZIP=-9 tar cvzf file.tar.gz /path/to/directory


11

As you stated- "tar can also compress", implies that - tar does not always compress data by itself. It does so only when used with the z option. That too not by itself, but - by passing the tarred data through gzip. However instead, as noted in this answer, you can pipe the two commands: tar & gzip such that you can explicitly specify compression ...



Only top voted, non community-wiki answers of a minimum length are eligible