Questions tagged [gzip]
`gzip` (or GNU zip) is a file format and corresponding utility for compressing files using the DEFLATE algorithm. The usual extension is `.gz`.
238
questions
0
votes
1
answer
24
views
Mounting a directory of zipped files for direct unzipped access
I have a directory tree with LOTS of .gz files. I once saw a way to mount it (via FUSE I think) so that I could see all the files directly (i.e. unzipped) under another directory.
I've been searching ...
- 170
0
votes
0
answers
40
views
Why are not all my files included when I gzip a directory?
I know this is a question that has been asked before, for example here Recursively count all the files in a directory,
but this is driving me crazy. Let me explain:
I have a directory called "...
- 101
1
vote
1
answer
121
views
dd + gz = Even Worse /// Mount Valid Partition of Whole-Disk dd Image
I've got a 250G backup of a whole disk via dd, which was then piped to gz, thanks to the advice of one of my -genius- friends.
Looks like the EFI boot partition was corrupted (pretty sure it's an ...
- 11
0
votes
2
answers
77
views
zcat + pigz = undeletable write-only "zombie" file - what happened?
A colleague (anonymized to protect the innocent, the guilty and wherever in this range accidental demon summoners fall) followed a tutorial that included combining a handful of gzipped files with
zcat ...
- 101
0
votes
0
answers
314
views
Windows tar compress command doesnt appear to include subdirectories
I am trying to compress all files and folders in the current folder (on my Windows 10 environment) so I can extract them on my Raspbian environment. Im using a Powershell script with the tar command
...
- 161
1
vote
0
answers
215
views
gzip -d introduces wrong ASCII characters
I am struggling with the following command: gzip -d /path/to/compressed.sql.gz
I always run it inside a container from the same Docker image. It works fine on my laptop as well as in our production ...
- 111
0
votes
2
answers
710
views
Can one extract the files in a .tar.gz and upload them individually onto cPanel?
I am in the process of migrating my website from Bluehost to a different hosting company. I downloaded the entire backup of the cPanel (61 GIG) successfully, but uploading files is incredibly much ...
Dude
1
vote
1
answer
690
views
Error: (InvalidBSON) incorrect BSON length in element with field name 'clone_class_metrics.CE' in object with _id: ObjectId
I have windows 10 on my system with enough storage.
I have a database that is 208 GB in a file with the .agz extension.
When I import the database into my MongoDB, I face the following error:
2021-09-...
- 13
0
votes
1
answer
279
views
How do I maintain integrity on zip and unzip?
I'm taking a dd of a drive and trying to share it with a friend. It's rather big after dd (~8gb). When I tar -czvf my-image.img I can get it down to ~1.5gb. However, if I try to test unzip it, I see ...
- 155
0
votes
0
answers
346
views
Unable to decompress file with gzip
I have a file called myfile.dat.gz that I need to decompress. I have been trying gzip -d myfile.dat.gz, but this results in the error gzip: myfile.dat.gz: invalid compressed data--format violated.
...
- 109
1
vote
1
answer
337
views
recompressing .gz and .bz2 files to attain the same hash/checksum (mod timestamps?)
i want to duplicate the entire contents of wikimedia's dumps, monthly, and upload them to amazon glacier and/or tape backup and/or a third backup medium that can survive corporation collapse/...
- 23
3
votes
1
answer
441
views
I have downloaded large gzipped xml files from Pubchem but my computer says they are corrupted
I have tried twice downloading the large gzipped xml files from PubChem FTP server, once through wget and one through chrome manually, but both times result in a corrupt file. For example, the first ...
- 351
0
votes
0
answers
445
views
How could I use logrotate and group by day a bunch of gzip files with an uncompresssed extension?
Doing a test in a production enviroment with rotation logs with 2 days just for testing purposes with gizp files. Created a test file at /tmp/.
/etc/init.d/test
/tmp/test/*.gz {
daily
...
- 329
0
votes
1
answer
1k
views
How to split a large gzip file into smaller gzip files without uncompressing it first?
Is there a way to split a large gzip file into smaller gzip files without using zcat?
zcat large.bed.gz| split --numeric-suffixes - -b 7M --filter='gzip > $FILE.gz' large.part.
- 282
0
votes
1
answer
3k
views
How can I count the number of files inside a tar.gz file (don't want to decompress)
I have a tar.gz file (neatly 1TB size). I want to count the number of files inside this tar.gz file. I used the following command
tar -tzf file.tar.gz
but it only lists the file name while I need the ...
- 1
1
vote
1
answer
1k
views
Transfer a directory over ssh with sudo
I am trying to transfer a directory from a remote server(CentOS) to my local machine(Ubuntu) over ssh. There are two users : A and B. User A can ssh into remote server and has sudo access. User B owns ...
-2
votes
1
answer
165
views
I want to tar gz file of the month of January
-rw-r--r--. 1 root root 4788074 Jan 1 01:50 2020-01-01-00-50-18.done.csv
-rw-r--r--. 1 root root 4842864 Jan 1 02:50 2020-01-01-01-50-18.done.csv
-rw-r--r--. 1 root root 4923486 Jan 1 03:50 ...
0
votes
1
answer
329
views
Perform operations on the output of find command with -exec options
I want to perform operations on the files found by the find command which already has a -exec option.
find . -type f -exec zgrep -li "4168781103" {} +
Output:
./results/...
1
vote
1
answer
982
views
How to make Dovecot maildir format emails be gziped using zlib
I activate compression plugin by set following on Dovecot:
# /etc/dovecot/conf.d/20-imap.conf
(...)
protocol imap {
mail_plugins = $mail_plugins zlib
}
# /etc/dovecot/conf.d/90-plugin.conf
(...)
...
1
vote
0
answers
272
views
"gzip --version" gives "Abort trap: 6" in macOS
I found that doing gzip --version in the terminal on my Mac just gives output Abort trap: 6. I came across this because it appears to cause another program to exit when it searches for the installed ...
- 111
0
votes
1
answer
412
views
gzip: CHARTEVENTS.csv.gz: not in gzip format
I'm trying to expand a file named CHARTEVENTS.csv.gz by using guzip CHARTEVENTS.csv.gz , but always I have this error gzip: CHARTEVENTS.csv.gz: not in gzip format . can any one tell me what is the ...
- 123
0
votes
1
answer
173
views
how to send the decompressed output of archives to standard output?
When you decompress a gzip archive with gunzip without any arguments, the archive disappears, and you're left with the decompressed file in its place.
However, if you use the -c option, the archive ...
- 103
0
votes
1
answer
706
views
Extracting a file in solaris
I'm try to extract a file in solaris to a folder.
I have tried following combinations but no luck.
gunzip -c jdk-8u211-solaris-sparcv9.tar.gz > /opt/java/jdk8.2.11
I'm getting a is directory ...
0
votes
1
answer
256
views
Serving tar.gz file through AWS Cloudfront?
I would like to keep things tidy on the S3 bucket where my website's assets live.
Currently I am uploading all assets to S3, as part of the CI/CD pipeline, this has got a bit messy since generated ...
- 101
0
votes
0
answers
352
views
gz file corrupted while transfer
I have huge gz files generated in linux machines and everyday night there is a script running to move files to remote network storage which is also linux host with terabytes data. The problem is, I am ...
3
votes
1
answer
7k
views
MongoDB, Using mongorestore on an .agz file
I am using MongoDB 3.6 and am having trouble understanding how to use mongorestore on an .agz file. If I specify the below, then I get an error saying Failed: error connecting to db server: no ...
- 195
0
votes
2
answers
652
views
How to restore the original file.gz after running a gunzip -f file.gz?
The default behavior of gunzip is to delete the .gz file after it decompresses.
There's a way to restore the original file from the decompressed version on Mac OS?
- 101
8
votes
1
answer
1k
views
While trying to see the content of a zipped file, I only get information about the zipping process
In Ubuntu, installed on Windows Subsystem for Linux, I want to see the content of a zip file, using gzip --list.
I expected to see a list of included files like this:
FileName ratio
File1 ...
- 1,921
2
votes
1
answer
3k
views
How do modern browsers deflate .tar.gz files?
Assuming the server is set up correctly, modern browsers are able to automatically decompress .gzip files when they are downloaded. Is the same true for a .tar.gz file?
For example, say you have an ...
- 133
0
votes
2
answers
511
views
Extract folder inside large gzip file in windows
I have a 200 gb tar.gz file I need to extract one folder inside this file. I know exact location of the folder need. Is there any way to do that, any third party tools, or any power shell commands ?
...
- 119
0
votes
1
answer
233
views
Compress and provide single-file access to 10 million files [closed]
I have ~10 million small text files and I would like to solve the following tasks:
compress all the data;
put it all into 1 file to transfer over the Internet;
be able to access each single file ...
- 173
2
votes
0
answers
853
views
Why is my .tar.gz larger than the sum of the separately compressed files in it?
I observed the following situation which is somewhat unexpected to me:
I have a csv file and a corresponding txt file. Uncompressed, their sizes are 375MB and 5KB.
When I compress the csv file ...
- 121
4
votes
2
answers
7k
views
Not enough disk space to unzip 50 GB file on Ubuntu 16.04: Can I extract and reduce zip file size at the same time?
I have a 50 GB zip file, which contains roughly 50.000 jpg images. The jpgs are, unzipped, around 55 GB. I am on Ubuntu 16.04.
But I only have 70 GB disk size on that system. So I get a disk error in ...
- 140
1
vote
1
answer
2k
views
DD image smaller than drive copied
I took a backup of my server's OS disk through a live USB. Using the command:
dd if=/dev/sda bs=512 count=(30 gb worth of sectors) conv=noerror,sync status=progress | gzip -c > /path/to/removable/...
- 13
17
votes
1
answer
39k
views
How do I use 7-Zip GUI for Windows to create .tar.gz files?
How can I use the 7-Zip GUI on Windows to create .tar.gz files?
- 630
1
vote
1
answer
671
views
Why does gzip -c give different results than gzip?
I am writing unit tests for some code and found that a simple gzip is causing a difference in my results. Upon further investigation I found that gzip gives a different .gz file than gzip -c does. Why ...
- 1,454
1
vote
1
answer
2k
views
Gzip -c over pipe matter?
I saw sometime 2 ways of writting: with and without -c
Do the 2 following case output the same and have equal speed ?
mysqldump -u root dbname | gzip > test.sql
mysqldump -u root dbname | gzip -c ...
- 113
0
votes
0
answers
140
views
How to view specific path inside tar.gz?
tar -tf file.tar.gz let's me view inside the whole archive. How do you view a specific path without viewing the whole thing? E.g., I want to view only /path/foo/
Update:
This command almost works ...
- 5,415
0
votes
1
answer
725
views
Can I decompress a .gzip file with tar?
I downloaded a .gz file with anonymized Census data and around 1 GB compressed. I am unable to inspect it with tar:
$ tar -tf census.csv.gz
tar: Unrecognized archive format
tar: Error exit delayed ...
- 1,757
0
votes
1
answer
220
views
MNIST Dataset - .gz files do not contain "raw" bytes
I am trying to use the MNIST dataset. However, when I download the .gz image file and then unzip it, I get a lot of "compressed" bytes. Here's a bit of it:
'Tπüó<$fi˛˛˛˛Ò∆∆∆∆∆∆∆∆™4CrHr£„˛·˛˛˛˙²˛...
- 3
1
vote
1
answer
2k
views
Append to compressed log
There is a couple of z-utilities (zless, zcat), which are able to read contents of zipped file in transparent manner.
Is it possible to append to compressed log file (in either format, not ...
- 261
2
votes
1
answer
1k
views
simple way to extract tar.gz in-place with no hard-drive overhead
I'm working with a tar.gz file (196 GB). This file needs to be extracted (240 GB) in place, and I would like the total disk usage not to go over 240 GB during the process (or close to it as possible). ...
- 121
1
vote
1
answer
880
views
Looking for the best way to backup and restore a directory of sparse files
We have a directory of sparse files (du returns 240M for one of these files; ls returns 960M).
When we copy one of these files over to S3, we end up uploading the entire 960M, and on downloading ...
- 860
0
votes
0
answers
1k
views
AWS Redshift failed to inflateinvalid or incomplete deflate data while inserting a zipped csv file
I have a zipped csv file which is a part of a huge file, and extracted from the original huge file using the following command:
gunzip -c myFile.csv.gz | head -n 500000 > myFile.csv.gz_p1
after ...
- 101
15
votes
2
answers
47k
views
"gzip: stdin has more than one entry--rest ignored" and "gzip: tmp.gz has more than one entry -- unchanged"
I have a certain file named tmp.gz. When I try to decompress it using gzip -d, I get an error-message that it has multiple entries:
$ gzip -d tmp.gz
gzip: tmp.gz has more than one entry -- unchanged
...
- 636
3
votes
2
answers
3k
views
Md5 checksum different after gunzip and gzip
I accidentally deleted a handful of gzipped files from a folder. Thankfully, I had uncompressed them in a different location, and am in the process of restoring them. I had the md5 checksums for the ...
- 133
9
votes
1
answer
10k
views
How to use gzip or gunzip in a pipeline with curl (For binary gz files)
I am using curl to download a file using the following command.
curl -O https://s3.amazonaws.com/dl4j-distribution/GoogleNews-vectors-negative300.bin.gz
I want to unpack it in a pipeline in the ...
- 93
1
vote
2
answers
11k
views
Fast compression in 7z format (like zip or gzip)
In short: can the Deflate compression be used only with the zip format (-tzip) in 7zip?
I want to archive a big directory (hundreds GiB) from a disk to another, while keeping I/O speed the same or ...
- 1,134
10
votes
1
answer
9k
views
Is .gz the same as .gzip?
I'm decompressing a file with a .gzip extension and then re-compressing it using 7Zip. When re-compressed though it has a .gz extension.
Are a .gz and .gzip extension the same thing?
- 203
2
votes
1
answer
14k
views
Create a tar.gz from an image file (Linux or Mac) [closed]
Hi I have an img of a custom linux distro and I would like to convert it to a tar.gz file. (I am using Mac or Linux OS)
What is the most straight forward way to do it?
Thank you
- 151