Can I expand the size of a file based disk image?

Question

I created an empty disk image using dd, then I used mkfs to make it a real filesystem image. I am mounting and using it fine. What I need is to be able to expand or shrink this file based disk image when needed. Is it possible to increase the size of an disk image that way? Is there a way to make this file based disk image have a dynamic resizing feature like that is found with Virtual machine drives.

Answer 1

First you have to create an image file:

# dd if=/dev/zero of=./binary.img bs=1M count=1000
1000+0 records in
1000+0 records out
1048576000 bytes (1.0 GB) copied, 10.3739 s, 101 MB/s

Then, you have to create a partition on it -- you can use whatever tool you want, fdisk, parted, gparted, I prefer parted, so:

# parted binary.img

You have to create a partition table first and then one big partition:

(parted) mktable                                                          
New disk label type? msdos      

(parted) mkpartfs
WARNING: you are attempting to use parted to operate on (mkpartfs) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs.  We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition type?  primary/extended? primary
File system type?  [ext2]? fat32
Start? 1
End? 1049M

Now let's see:

(parted) print
Model:  (file)
Disk /media/binary.img: 1049MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1049MB  1048MB  primary  fat32        lba

It looks good,

You want to enlarge it, so fist add some zeros to the image using dd:

# dd if=/dev/zero bs=1M count=400 >> ./binary.img
400+0 records in
400+0 records out
419430400 bytes (419 MB) copied, 2.54333 s, 165 MB/s
root:/media# ls -al binary.img 
-rw-r--r-- 1 root root 1.4G Dec 26 06:47 binary.img

That added 400M to the image:

# parted binary.img 
GNU Parted 2.3
Using /media/binary.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print                                                            
Model:  (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1049MB  1048MB  primary  fat32        lba

As you can see, the size of the image is different (1468MB). Parted can also show you free space in the image. If you want to see it just type print free instead of print. Now you have to add the extra space to the filesystem:

(parted) resize 1
WARNING: you are attempting to use parted to operate on (resize) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs.  We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Start?  [1049kB]?
End?  [1049MB]? 1468M

and check it:

(parted) print
Model:  (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1468MB  1467MB  primary  fat32        lba

Pretty nice. If you want to shrink it, just do similar thing:

(parted) resize 1
WARNING: you are attempting to use parted to operate on (resize) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs.  We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Start?  [1049kB]?
End?  [1468MB]? 500M

Now you can check if the partition is smaller:

(parted) print
Model:  (file)
Disk /media/binary.img: 1468MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End    Size   Type     File system  Flags
 1      1049kB  500MB  499MB  primary  fat32        lba

Yes, it is.

If you try to resize the partition when data is on it, you have to pay attention to the size of the data because when you shrink it too much, you will get an error:

Error: Unable to satisfy all constraints on the partition

After shrinking the file system, you also have to cut some of the file off. But this is tricky. You could take the value from parted 500M (END):

# dd if=./binary.img of=./binary.img.new bs=1M count=500

But this leaves some space at the end of the file. I'm not sure why, but the image works.

And there's one thing about mounting such image -- you have to know an offset to pass to the mount command. You can get the offset from, for instance, fdisk:

# fdisk -l binary.img

Disk binary.img: 1468 MB, 1468006400 bytes
4 heads, 32 sectors/track, 22400 cylinders, total 2867200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000f0321

     Device Boot      Start         End      Blocks   Id  System
binary.img1            2048     2867198     1432575+   c  W95 FAT32 (LBA)

2048 (start) x 512 (sector size) = 1048576 , so you have to use the following command in order to mount the image:

# mount -o loop,offset=1048576 binary.img /mnt

Answer 2

Yes, this is possible - it works just like a partition. I tried the following, which worked:

Make the original file, mount it, check, unmount it

dd if=/dev/zero of=test.file count=102400 
mkfs.ext3 test.file 
mount test.file /m4 -o loop
df
umount /m4

Grow it

dd if=/dev/zero count=102400 >> test.file
mount test.file /m4 -o loop
df
resize2fs /dev/loop0
df

There is no reason why shrinking a file would not work similarly, but shrinking a file is always more difficult then growing a file (and, of-course, needs to be done when the block device is not mounted etc)

Have a look at this link which talks about using qemu-nbd to mount qcow2 images

Answer 3

Sparse files are a good choice for dynamic grow/resize disk images.

This will create a 1024M sparse file:

# dd if=/dev/zero of=sparse.img bs=1M count=0 seek=1024
0+0 records in
0+0 records out
0 bytes (0 B) copied, 0.000565999 s, 0.0 kB/s

The image is not using any disk space,

# du -m sparse.img
0   sparse.img

but has the apparent size of 1024M.

# ls -l sparse.img
-rw-rw-r--. 1 root root 1073741824 Sep 22 14:22 sparse.img

# du -m --apparent-size sparse.img
1024    sparse.img

You can format and mount it as a regular disk image:

# parted sparse.img 
GNU Parted 2.1
Using /tmp/sparse.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) mktable                                                          
New disk label type? msdos                                                
(parted) mkpartfs                                                         
WARNING: you are attempting to use parted to operate on (mkpartfs) a file system.
parted's file system manipulation code is not as robust as what you'll find in
dedicated, file-system-specific packages like e2fsprogs.  We recommend
you use parted only to manipulate partition tables, whenever possible.
Support for performing most operations on most types of file systems
will be removed in an upcoming release.
Partition type?  primary/extended? primary                                
File system type?  [ext2]? fat32                                          
Start? 1                                                                  
End? 1024M                                                                
(parted) print                                                            
Model:  (file)
Disk /tmp/sparse.img: 1074MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
 1      1049kB  1024MB  1023MB  primary  fat32        lba

# du -m sparse.img 
2   sparse.img

Now, resize using the same command to create just changing the seek parameter with the new size of image:

dd if=/dev/zero of=sparse.img bs=1M count=0 seek=2048

As you can see, the image is now 2048M and you can enlarge the partition using parted or other tool of your choice.

# du -m --apparent-size sparse.img 
2048    sparse.img


# parted sparse.img 
GNU Parted 2.1
Using /tmp/sparse.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print free                                                       
Model:  (file)
Disk /tmp/sparse.img: 2147MB
Sector size (logical/physical): 512B/512B
Partition Table: msdos

Number  Start   End     Size    Type     File system  Flags
        16.4kB  1049kB  1032kB           Free Space
 1      1049kB  1024MB  1023MB  primary  fat32        lba
        1024MB  2147MB  1123MB           Free Space

(parted)                               

# du -m sparse.img 
2   sparse.img

Now enjoy it!