6

On a Linux system, is it possible to use only user-space tools to create and manipulate partition images (which will probably only have ext2 filesystems on them) and then take those partition images to create a disk image?

I can use genext2fs to generate an image of an ext2 filesystem in userspace, but I can't find any tool that can manipulate disk images (e.g. creating partition tables or create a disk image from existing partition images etc.) without the need for root access. Currently, I'm using kpartx to mount the partitions inside a disk image, but because this requires root, it doesn't really make it suitable for scripting, and if there's an error in the build script which isn't trapped, the mapper devices remain in use.

Basically, I have three directories, and I want each directory to become a partition on a disk image, but I want to script this and I really do not want to use any tools that require root access (it seems like there is nothing in this process that absolutely needs to have root access).

3 Answers 3

4

Partition creation

To make a partition table, you can use your common tools directly on the file instead of the device. Most standard tools should support this usage, like fdisk and parted. parted is great to be integrated in a script as it can take the commands as argument.

parted -s testing.img mklabel gpt
parted -s -a none testing.img mkpart ESP fat32 0 4M
parted -s -a none testing.img mkpart linux ext4 4M 10M

File system creation

I don't have a clean solution to mount or create a file system in a virtual partition. The right thing to do would probably be to have a fuse (Filesystem in Userspace) driver for partition tables. But I haven't found one at the moment.

However, mke2fs can take a -E offset=1234 option to build an ext2, ext3 or ext4 file system at an arbitrary offset within the file. You can give it the offset of your partition. However, be sure to give it the size of the file system as well since the default behavior seems buggy and write a file system with the size of the virtual disk by default, extending the file if needed.

mke2fs -E offset=4000256 testing.img 6316k

Unfortunately, not all the mkfs have such an option. mkfs.vfat doesn't. So another solution can be to create a separate file of the size of the partition, use mkfs on it and then use dd to copy the content into the full virtual disk.

dd if=testing.img of=testing.fat32.img bs=512 skip=34 count=7779
mkfs -t vfat testing.fat32.img
dd if=testing.fat32.img of=testing.img bs=512 seek=34 count=7779 conv=notrunc

Not the cleanest or fastest solution, but it works, it is quite generic and it doesn't require root privileges. The conversion option sparse can also be used to speed things up a bit by not writing longs sequences of zeros.

Mounting file systems

You can mount a file system in a partiton with the help of fuse provided it has a support for your file system. There is a standard support for ext file systems under the name fuse-ext2 and one for FAT under the name fusefat. They unfortunately don't take an offset option at the moment. So I guess you're bound to either using mount or the same dd trick as earlier.

The standard method with partx or kpartx, (or the -o offset option of mount) require root privileges. The dd solution is slow, complex and error-prompt.

2

I use and recommend guestfish, which is a part of libguestfs, library for accessing and modifying VM disk images.

I've found OpenStack's documentation includes a useful description of tools that do not require privileges: Modify images — Virtual Machine Image Guide documentation

Note that if you are making many disjoint changes you may benefit from running a guestfish daemon process and using its remote control mode - this saves on the (not insignificant) start-up time.

See Remote control guestfish over a socket of the guestfish manual.

1

Try using the combination of:

and:

Example:

# Create 1GB sparse file
truncate -s 1G disk1.bin

# Create simple ms-dos partition table an a single partition with maximum free space
echo -e "o\nn\np\n1\n2048\n-1\np\nw\n" | /sbin/fdisk disk1.bin

Welcome to fdisk (util-linux 2.34).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table.
Created a new DOS disklabel with disk identifier 0x4643133c.

Command (m for help): Created a new DOS disklabel with disk identifier 0xff5d1f1e.

Command (m for help): Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)
Select (default p): Partition number (1-4, default 1): First sector (2048-2097151, default 2048): Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-2097151, default 2097151): 
Created a new partition 1 of type 'Linux' and of size 1023 MiB.

Command (m for help): Disk disk1.bin: 1 GiB, 1073741824 bytes, 2097152 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xff5d1f1e

Device     Boot Start     End Sectors  Size Id Type
disk1.bin  1     2048 2097150 2095103 1023M 83 Linux

Command (m for help): The partition table has been altered.
Syncing disks.

# Check out the offset and size in bytes    
$ parted disk1.bin
GNU Parted 3.3
Using /kvm/test2/disk1/disk1.bin
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit b                                                           
(parted) p                                                                
Model:  (file)
Disk /kvm/test2/disk1/disk1.bin: 1073741824B
Sector size (logical/physical): 512B/512B
Partition Table: msdos
Disk Flags: 

Number  Start     End          Size         Type     File system  Flags
 1      1048576B  1073741311B  1072692736B  primary  fat32

(parted) q

# Create a placeholder file for fuseloop to use
touch disk1.bin.part1

# fuseloop - take a raw disk image and create a "loopback block device file" for a partition within the raw disk image file
# fuseloop -O OFFSET -S SIZE DISKFILE.RAW PARTITIONFILE.RAW
$ fuseloop -O  1048576  -S 1072692736    disk1.bin   disk1.bin.part1

$ findmnt $(realpath disk1.bin.part1)
TARGET                           SOURCE    FSTYPE OPTIONS
/kvm/test2/disk1/disk1.bin.part1 /dev/fuse fuse   rw,nosuid,nodev,relatime,user_id=0,group_id=0

# Create the filesystem
$ /sbin/mkfs.vfat -n DATA -v disk1.bin.part1
mkfs.fat 4.1 (2017-01-24)
Auto-selecting FAT32 for large filesystem
disk1.bin.part1 has 255 heads and 63 sectors per track,
hidden sectors 0x0000;
logical sector size is 512,
using 0xf8 media descriptor, with 2095103 sectors;
drive number 0x80;
filesystem has 2 32-bit FATs and 8 sectors per cluster.
FAT size is 2048 sectors, and provides 261371 clusters.
There are 32 reserved sectors.
Volume ID is c86be444, volume label DATA       .

# Make a mountpoint dir (please don't use /tmp)
mkdir /tmp/mnt

# fusefat - take the "loopback block device file" and mount it (-o ro or -o rw required)
# fusefat -o MODE PARTITIONFILE.RAW /path/to/mountpoint/
$ fusefat -o ro disk1.bin.part1  /tmp/mnt/

fat type: FAT32. Fsi at 1
Fsioff: 512, size: 512
--- nxtfree --- :4
--- freecnt --- :261368
dataclusters :261371  
first data byte : 2113536 
1st fat off :  16384 
2nd fat off :  1064960
fat_eoc_value: 268435448
fat_eoc_value is eoc?: 1

$ findmnt /tmp/mnt/ 
TARGET    SOURCE  FSTYPE       OPTIONS
/tmp/mnt/ fusefat fuse.fusefat ro,nosuid,nodev,relatime,user_id=0,group_id=0

# Unmount the filesystem (and verify)
$ fusermount -u /tmp/mnt/
$ findmnt /tmp/mnt/ 

# Unmount the "loopback block device file" (and verify)
$ fusermount -u disk1.bin.part1
$ findmnt $(realpath disk1.bin.part1)

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .