I need to create a large sized file that doesn't have repeated stuff in it so that even on zipping, I don't get the size reduced.
1
-
If you're still around, i suggest you check my answer. I believe it's the strongest answer. The one you marked as Answer fails to fulfill your requirement. superuser.com/a/1658098/112542– johny whyCommented Jun 22, 2021 at 9:17
Add a comment
|
4 Answers
The best way to create random dump files is by using 'fsutil' in your command prompt.
C:\>fsutil file createnew another1.txt 32234567000
File C:\another1.txt is created
This creates 32GB file instantly which has random content.
-
2There’s no way to create a file “instantly” with random content. If you mean, “indeterminate” then that is a different story. But, it is far from random. Instead, it just contains bits and bytes of old data left on the drive. That’s not what the OP is asking for. Commented May 12, 2018 at 19:26
-
3It may be even worse. This site says about
createnew: "Creates a file of the specified name and size, with content that consists of zeroes". Commented May 12, 2018 at 19:43 -
2On my PC this command creates a file where all bytes are 0. Even on my hard disk with the file size set to 1TB I still get only 0s. Commented May 12, 2018 at 19:43
-
1@KamilMaciorowski that does appear to be the case, which is, of course, impossible to do “instantly.” On another note, how is this the answer? The question was to create a file with random content. Commented May 12, 2018 at 20:52
-
1@Appleoddity I imagine it could be done instantly with some tricks like sparseness or lazy init etc. It doesn't matter though, there is no randomness at all; for this reason I have downvoted this answer and upvoted the
ddone. Commented May 12, 2018 at 20:58
On Windows, you can create a file containing truly unique content with Powershell. No third-party software needed. No admin rights needed.
As opposed to fsutil, the output will not contain 0's or spaces. Each time you run the command, the output will be unique. On compression, the file will retain it's original size.
The output will look something like this:
䫲藌㦖鐆쩘ꌓ⮅픻씙방醒ম擾酰₵廙⊌弓똌硣浍鐟ઘ⃭佐怱쎜鼋ꄻ윤訟ᦟ맂㱆�㔭ර槗旌閣㏏⯉來殰鲑奣鑍翴㑈Ꮪ嚠昲︇퇇뻕耣珁犒晥䒋酉懒䊿䱵漝벻玓啈�펕戶臓헬珎宇ꄌꖓ萿 etc....
The command
$out = new-object byte[] 100000; (new-object Random).NextBytes($out); [IO.File]::WriteAllBytes('C:\100K.txt', $out)
Replace 100000 with whatever size you want, in bytes. You can use an online converter if you need help defining the size you prefer.
Replace C:\100K.txt with the filepath you want. If you provide a name without a path, it will be saved in your Windows profile folder (or maybe C:, depending on your system configuration). The file extension has no impact on the process.
Breakdown
This is a series of three commands. The first command creates an empty array called $out, of the size you specify. The second command loads the array with random characters. The third command saves the array to a file.
Here's the Microsoft support file for new-object.
This appears to be supported at least as far back as Powershell 3.0. (Currently at version 7.x as of this writing).
Taken from: https://www.digitalcitizen.life/3-ways-create-random-dummy-files-windows-given-size
Limit
I can output files up to 2G. Anything larger hits a limit. For example, 3G gives:
"Cannot convert value "3000000000" to type "System.Int32". Error: "Value was either too large or too small for an Int32.""
This answer shows a way to write large files in chunks. Basically, wrap a do..while around the commands which load the array and write to file. The writes need to append to the file.
Regardless of limits on array-size, I think you'll want to consider your available RAM.
-
2
-
@NedMartin How large? I created 100 MB files no problem with this method. Commented Aug 30, 2021 at 17:18
-
-
1@NedMartin I believe this is a limit on the array-size. In my code, the array is
$out. This answer shows a possible way to write the file in chunks stackoverflow.com/a/66111182/209942 But i bet that code could be much simplified, so we could still achieve a relatively shorter, one-line command. Also, he recommends 4K chunks, but so tiny! I would do 1G chunks, since my command can do that. Commented Sep 2, 2021 at 14:47 -
1Thank you! Your solution worked. I have used the value of 1073741824 to create a 1GB file with pseudorandom data. Commented Oct 6, 2021 at 17:52
Under Linux the easiest way to do this would be the dd command. There is a Windows version at http://www.chrysocome.net/dd
To create a random file with a size of 1GB you can run the command
dd if=/dev/random of=random.file bs=1M count=1000
This means: use a blocksize of 1 MB and read/write 1000 blocks.
btw. on Linux you should use /dev/urandom but for this Windows version it has to be /dev/random
-
1
-
6
Install Veracrypt.
Create an encrypted container that is formatted (not quick format) with whatever size you want.
This will ensure the contents are cryptographically random.
Alternatively, create a file with fsutil as suggested by fossa then encrypt the contents with a one-time pad (for example, with AES using a random passphrase that you throw away).
-
Requires extra software, instead of using built-in Windows utilities. fsutil doesn't produce a file containing unique content. As a result, on compression it will become very tiny (even with Windows Explorer encryption). Commented Jun 21, 2021 at 14:58