33

I have a DD image from a 4GB SD card that has two partitions, these two partitions are only using up about 800 MB and as such I wish to reduce the size of the img fie.

Does anyone know of a way to remove the "free space" from the img file?

1
  • you may want to add what the desired outcome is. ie: image back to the card, to store on CD.
    – Wayne
    Jun 22 '13 at 16:41
14

First make sure the free space is actually empty, and doesn't contain leftovers of deleted files.

With recent kernels (3.2 or later), it's easiest to do so by mounting each partition of the loop image, then issuing a discard using fstrim on the mountpoint. This works on loop devices in a similar way to TRIM on SSDs; unused areas are replaced with zeros and the underyling .img file becomes sparse.

# losetup --find --partscan foo.img
# lsblk
NAME      MAJ:MIN RM    SIZE RO TYPE MOUNTPOINT
loop0       7:0    0   4096M  0 loop 
├─loop0p1 259:0    0   2048M  0 loop 
└─loop0p2 259:1    0   2048M  0 loop 
# for part in /dev/loop0p*; do
    mount $part /mnt
    fstrim -v /mnt
    umount /mnt
  done
/mnt: 2xxx MiB trimmed
/mnt: 2xxx MiB trimmed
# losetup --detach /dev/loop0

Otherwise, an easy way to achieve this is to create a huge file on the disk, containing only null bytes, then delete it.

# losetup --find --partscan foo.img
# lsblk
NAME      MAJ:MIN RM    SIZE RO TYPE MOUNTPOINT
loop0       7:0    0   4096M  0 loop 
├─loop0p1 259:0    0   2048M  0 loop 
└─loop0p2 259:1    0   2048M  0 loop 
# for part in /dev/loop0p*; do
    mount $part /mnt
    dd if=/dev/zero of=/mnt/filler conv=fsync bs=1M
    rm /mnt/filler
    umount /mnt
  done
dd: error writing ‘/mnt/filler’: No space left on device
dd: error writing ‘/mnt/filler’: No space left on device
# losetup --detach /dev/loop0

Then compress it with a tool like gzip or xz. Even at lowest compression levels, a long series of zeros will compress well:

# ls -s
4096M foo.img
# gzip foo.img
# ls -s
11M foo.img.gz

Note that you must uncompress the image when writing it back to disk. This will uncompress it 'live':

# cat foo.img.gz | gunzip | dd of=/dev/sda

Note that the output device (sda) must be of sufficient size to fit the original image, otherwise data will be lost or corrupted.


An alternative method, if you want to keep using the image – e.g. with a virtual machine – is to convert the raw image to one of the image formats used by virtualization software; e.g. qcow2 for Qemu, VDI for VirtualBox, or VMDK for VMware.

Note that this still requires you to prepare the image by cleaning the free space using the above method.

# qemu-img convert -f raw -O qcow2 foo.img foo.qcow

# qemu-img convert -f raw -O vmdk foo.img foo.vmdk

But if it's going to be written to a real disk again, you have to convert it back to a raw image.

8
  • 1
    how can you be sure that all of the files that are currently stored on the partitions aren't physically scattered around on the disk? Jun 18 '14 at 13:38
  • because I suppose that files aren't written sequentially on the disk and if a file is at the end of the storage, when I cut the size down, I'll lose the content of that file. Jun 18 '14 at 17:35
  • Because the OP is asking how to shrink a disk image without losing data. If you truncate an image at a certain size you can't be sure the data will be preserved. Jun 19 '14 at 17:05
  • My bad, I thought that the part where you do the cat foo.img.gz | gunzip | dd of=/dev/sda was meant to write back to a smaller storage device. I probably misread it. But my point still stands: if I want to reduce the size of an image to fit on a smaller storage device, how can I do that and be sure that the files aren't being truncated? Jun 19 '14 at 22:42
  • 1
    @ValerioSantinelli: By using the apropriate filesystem-resize tool for your filesystem, e.g. resize2fs or ntfsresize (Linux has no tool for FAT), before writing it to the smaller device, or by creating a fresh filesystem and copying over just the files.
    – user1686
    Jun 20 '14 at 5:39
28

Using resize2fs is much much easier

resize2fs -M xxx.img

you will be asked to e2fsck first - so:

e2fsck -f -y xxx.img

(image must NOT be mounted!)

Note: this will only work if the image is of a single partition, if it's a whole block device with mutiple partitions see above answer...

5
  • Just a note: for multiple partitions, use losetup --find --partscan xxx.img to setup the image file as a loop device. then run lsblk to find the loop device's partitions. Oct 8 '18 at 3:38
  • It looks like it dosen't work for ISO image that contains operating system. After resize system dosen't boot.
    – S.R
    Feb 7 '19 at 10:36
  • @S.R Linux operating system?
    – Rolf
    Feb 10 '19 at 10:09
  • 2
    This is a very nice answer, but resize2fs are only for extN type partitions. Other types of partitions will require other tools.
    – Rolf
    Feb 10 '19 at 10:14
  • -M = "Shrink the file system to minimize its size as much as possible, given the files stored in the file system." Finally found how to do this!
    – endolith
    Jan 17 at 0:30
7

I originally posted the same answer here, on StackExchange Ask Ubuntu. I re-propose the same answer here, it can be useful.

The key information was the use of the command truncate. Following the full solution in order to not lose the answer.

A preliminary step consists in cloning the SD card in your PC:

  1. use lsblk to see which devices are available and if their partitions are mounted

  2. unmount all partitions of the device you want to copy on your pc. For example:

     umount /dev/sdc1
     umount /dev/sdc2
    
  3. create a copy of the whole sd card with all the partitions unmounted

     dd if=/dev/sdc of=/path/to/file/myimage.img
    

Shrinking images on Linux

Context of the problem:

Having a myimage.img bigger then the hardware support (if it is smaller there should be no problem; however, using the same strategy, you can better fit the image in the hardware support).

The secret is to use standard Linux tools and instruments: GParted, fdisk and truncate.

Requirements:

  • A Linux PC
  • The .img you want to shrink (myimage.img in this example)

Creating loopback device:

GParted is an application typically used to manage partition tables and filesystems. In order to shrink the image, GParted is going to be used along the first part of the answer.

GParted operates on devices, not simple files like images. This is why we first need to create a device for the image. We do this using the loopback-functionality of Linux.

Let's enable enable the loopback:

sudo modprobe loop

Let's request a new (free) loopback device:

sudo losetup -f

The command returns the path to a free loopback device:

/dev/loop0

Let's create a device of the image:

sudo losetup /dev/loop0 myimage.img

The device /dev/loop0 represents myimage.img. We want to access the partitions that are on the image, so we need to ask the kernel to load those too:

sudo partprobe /dev/loop0

This should give us the device /dev/loop0p1, which represents the first partition in myimage.img. We do not need this device directly, but GParted requires it.

Resize partition using GParted:

Let's load the new device using GParted:

sudo gparted /dev/loop0

When the GParted application opens, it should appear a window similar to the following:

screenshot

Now notice a few things:

  • There is one partition.
  • The partition allocates the entire disk/device/image.
  • The partition is filled partly.

We want to resize this partition so that is fits its content, but not more than that.

Select the partition and click Resize/Move. A window similar to the following will pop up:

screenshot of dialog

Drag the right bar to the left as much as possible.

Note that sometimes GParted will need a few MB extra to place some filesystem-related data. You can press the up-arrow at the New size-box a few times to do so. For example, I pressed it 10 times (=10MiB) for FAT32 to work. For NTFS you might not need to at all.

Finally press Resize/Move. You will return to the GParted window. This time it will look similar to the following:

unallocated space on right

Notice that there is a part of the disk unallocated. This part of the disk will not be used by the partition, so we can shave this part off of the image later. GParted is a tool for disks, so it doesn't shrink images, only partitions, we have to do the shrinking of the image ourselves.

Press Apply in GParted. It will now move files and finally shrink the partition, so it can take a minute or two, but most of the time it finishes quickly. Afterwards close GParted.

Now we don't need the loopback-device anymore, so unload it:

sudo losetup -d /dev/loop0

Shaving the image:

Now that we have all the important data at the beginning of the image it is time to shave off that unallocated part. We will first need to know where our partition ends and where the unallocated part begins. We do this using fdisk:

fdisk -l myimage.img

Here we will see an output similar to the following:

Disk myimage.img: 6144 MB, 6144000000 bytes, 12000000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000ea37d

      Device Boot      Start         End      Blocks   Id  System
myimage.img1            2048     9181183     4589568    b  W95 FAT32

Note two things in the output:

  • The partition ends on block 9181183 (shown under End)
  • The block-size is 512 bytes (shown as sectors of 1 * 512)

We will use these numbers in the rest of the example. The block-size (512) is often the same, but the ending block (9181183) will differ for you. The numbers mean that the partition ends on byte 9181183512 of the file. After that byte comes the unallocated-part. Only the first 9181183512 bytes will be useful for our image.

Next we shrink the image-file to a size that can just contain the partition. For this we will use the truncate command (thanks uggla!). With the truncate command need to supply the size of the file in bytes. The last block was 9181183 and block-numbers start at 0. That means we need (9181183+1)*512 bytes. This is important, else the partition will not fit the image. So now we use truncate with the calculations:

truncate --size=$[(9181183+1)*512] myimage.img
1
5

I also tried it with qemu-img, and it worked like a charm:

qemu-img resize test.img 2G

We are resizing the test.img to make it 2G (2GB).

Worked flawless for me.

6
  • 1
    Awesome, worked flawless for me too, compressing a Raspbian image. fully working!
    – ffleandro
    Mar 19 '19 at 14:48
  • Well, everything is working, however after booting, running df -h shows the original size. How is this possible?
    – ffleandro
    Mar 19 '19 at 15:06
  • Apparently, qemu-img doesn't the partition table, so how does it compress the .img?
    – ffleandro
    Mar 19 '19 at 15:34
  • 1
    I can resize a .img disk to 1M which definitely breaks the disk image. Any idea to make the resize safer?
    – lesca
    Apr 29 '19 at 9:01
  • 1
    @lesca what do you mean by safer? Why would it break the image?
    – phrogg
    Jul 13 at 9:44
4

I have used the gparted approach with my Ubuntu 16.10 computer:

  1. Map the img file to the next available loop partition with sudo losetup -f --partscan file.img

  2. Check with lsblk which loop drive your image file is mapped to, e.g. /dev/loop0

  3. Execute sudo gparted /dev/loop0

  4. Shrink the loop partition(s) as deemed appropriate; please make sure to have these partitions unmounted.

  5. Execute fdisk /dev/loop0, then enter p, this will show you the block size and end block number of the various partitions.

  6. Execute sudo dd if=/dev/loop0 of=shrunk_image_file.img, apply to that command the options bs=[BlockSize] and count=[EndBlockNumberOfLastLoopPartition+1] and you will have a shrunk and rightsized image file. Optionally add status=progress to see it working.

  7. When done unmap the image file with sudo losetup -d loop0

1
  • This is going to truncate the partition and drop any data in the truncated portion. Need to shrink the fs first.
    – Jeter-work
    Apr 1 at 6:22
0

If you are working on a raspberry pi image, then you can try pishrink: it's very easy to use and minimize your pi image file automatically.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.