2

Is there a way to easily and arbitrarily increase the byte size of a file without corrupting its content? For instance, by inserting dummy bytes. The file for my use case is of type png.

I am on OS X, but all solutions are welcome.

Improve this question
2

3 Answers 3

Reset to default
19

The literal answer to your question is no, by definition. The size of a file is the length of its content. If you want to change the size, you'll need to change the content.

However, it may be possible to change the content and obtain equivalent content that has a different size. Whether this is possible depends on the file format and what you consider to be equivalent.

For example, if the file is a PNG image, and you consider two files containing PNG images to be equivalent if the displayed images are the same, you can make the file larger by adding (or lengthening) the comment field in the file. For example with ImageMagick:

convert old.png -comment 'This comment will be added to the image file, making it larger' new.png

The maximum portable comment size is 2GB (you might be able to go up to 4GB). In principle you can put multiple comments (or use other field names) but I don't know if common programs support this well.

Continuing the example of a PNG image, many programs accept files containing a valid PNG image plus extra content as if it was just the PNG image without this extra content. So many programs will still accept the image if you append trailing garbage:

cp old.png new.png
echo 'trailing garbage' >>new.png

You can use the truncate command to easily append null bytes to reach a certain size. Just keep in mind that the resulting file is widely accepted, but technically it's invalid so it might not be accepted everywhere. (It seems to work in every program that uses libpng or a derivative, and this library is highly widespread so you have a good chance that the file will work.)

Improve this answer
4
  • The spec explicitly says "Although encoders and decoders should treat the length as unsigned, its value must not exceed 2^31 bytes.". A single comment chunk must not exceed 2 GiB; it can't go up to 4 GiB.
    – Nayuki
    Jan 9, 2021 at 16:49
  • Your last "anywhere" should be "everywhere"
    – OrangeDog
    Jan 9, 2021 at 21:02
  • 1
    "The size of a file is the length of its content." This is not true for "sparse files"; the literal answer is yes. Using the comment field is a good practical solution.
    – Schwern
    Jan 9, 2021 at 22:37
  • 10
    @Schwern it's still true for sparse files. Sparse is just an attribute of the file system where it doesn't store some empty blocks and the user still see the whole file as normal. So the size of the file is still the length of its content, only the size on disk is different, just like transparently compressed files
    – phuclv
    Jan 10, 2021 at 0:32
5

The truncate command has a little-known option to increase the size of files by adding null bytes, which won't be visible in the content unless you're using an editor that explicitly shows these, like vim.

The proper syntax is explained in the man page, but generally, pass the -s option and give either a size you want the file to be like -s 10k, or add a '+' as in -s +10k to add that many null bytes to the file.

The added bytes will be inserted at the end. For your use case, these nulls will not be visible in the image when viewed normally.

Improve this answer
5
  • 3
    Some applications may still complain that the PNG is "invalid", "corrupt" or "has extraneous extra data". Adding extra bytes at the end doesn't really invalidate the content itself, but some software checks if the size as recorded in the PNG header matches the actual file-size. Adobe PhotoShop used to be very obnoxious about this, refusing to load the image at all. It has been a few years since I used PhotoShop. I don't know if current versions still do this. Same goes for most other image-formats by the way.
    – Tonny
    Jan 9, 2021 at 13:47
  • 9
    @Tonny given how faults in data file formats have been exploited over the years, I hesitate to call a program sanity checking files for hidden/unexpected issues "obnoxious".
    – Rob Moir
    Jan 9, 2021 at 14:31
  • 3
    @RobMoir There is a difference between extra data at the end (outside the structure of the actual payload of the file) and malformed data within the payload part of the file. Sanity checks on the latter are a must (to prevent exploits), but checking for data beyond the logical end of file... A properly written file-loader with sanity checks, should never read that extra data anyway, so it can't do any harm. The real danger is in an exploit in the payload that causes the loader to read/execute the extra data AFTER the image. But only if it was read. (Which shouldn't happen in the first place.)
    – Tonny
    Jan 9, 2021 at 16:10
  • truncate -s only appears to insert nulls. It creates a sparse file. The nulls do not actually exist, they only appear when you try to read the file. See my answer for more.
    – Schwern
    Jan 9, 2021 at 22:35
  • 3
    @RobMoir warning and giving the user the chance to cancel before examining the file further would be reasonable. Refusing to do anything at all is obnoxious :)
    – hobbs
    Jan 9, 2021 at 22:51
0

Depending on your purpose, you can create a sparse file. For example...

dd of=sparse bs=5M seek=1 count=0

This creates a file which appears to be 5 megabytes in size, but takes 0 disk space. When read, the "extra space" will appear to be nulls.

You can make an existing file sparse with truncate -s. For example, I've made this animated GIF appear to be 5 megabytes in size but only taking 102 kilobytes on disk.

$ ls -l pikachu-nyan.gif 
-rw-r--r-- 1 schwern staff 100872 Oct 29  2019 pikachu-nyan.gif

$ truncate -s 5M pikachu-nyan.gif 

$ ls -l pikachu-nyan.gif 
-rw-r--r-- 1 schwern staff 5242880 Jan  9 14:32 pikachu-nyan.gif

Finder info on the file says... 5,242,880 bytes (102 KB on disk)

Whether this is valid or not depends on how you're using the file. In the example above, the animated GIF continues to work in Firefox.

Improve this answer
1
  • as said, this actually changes the file content. The file system just doesn't store the blocks that contains zeros in the file content. You can write non zeros to those blocks and they'll be written again to disk. Those parts are not read-only
    – phuclv
    Jan 10, 2021 at 2:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .