11

I have a brand new Silicon Power Marvel M70 64GB USB flash drive has 45MB of unallocated space preceding the 58.89GB of available space. I don’t care about having less than 64GB since I know it’s due to 1024 based math.

What concerns me is within the size of 45MB. Why so much? 1MB is typical for alignment.

So, I loaded up a hexeditor in Linux with this command:

hexdump /dev/sdb -C | less

And I find “strings” of data like the following:

Invalid partition table. Error loading operating system.Missing operating system.

Remove disks or other media. Disk error. Press any key to restart. U.RRaA.

This program cannot be run in DOS mode.

CpaintDC. UserException. CResourceException.

An application has made an attempt to load the C runtime library incorrectly. Please contact the application's support team for more information. Attempt to use MSIL code from this assembly during native code.

That is within the UNALLOCATED 45 MB area... Shouldn't it be zeros straight from the factory? Furthermore, an identical drive (see below) has completely different sizes for the unallocated and partitioned area; it seems executable data is or was stored there

I’ve never encountered this before. Has anyone else? I called the company and they didn’t have an explanation only to tell me that upon opening one and connecting it to the computer theirs also shows 45 MB of unallocated space at the beginning of the drive as well.

That indicates that it’s probably done like this at the factory and not just mine has this configuration.

I am concerned about what is stored there; and is it possibly executable, etc.. especially with seeing the hexdump strings of text above from within the 45 MB of “unallocated space.” If it is executable; I would assume perhaps it is accessible when booting to the drive.

Anyone seen this before? Any explanation you can think of?

EDIT: When I called the company that makes it they also plugged in a 128GB size of the same model line and the unallocated space was 31 MB. Not sure if that has any bearing on what might be going on here, but it's certainly not “proportional” or at the very least the same.

EDIT: I have subsequently tried another brand new unit (same model, batch number, size (64GB) drives. As displayed by GParted:

  • Drive #1: 45.33 MiB Unallocated and 58.89GB FAT32

  • Drive #2: 46.38 MiB Unallocated and 54.70GB FAT32

These are both supposed to be 64GB drives. Identical in size, etc. This is straight from the factory. I have never encountered this kind of discrepancy with other flash drives. I mean, that’s over 4GB less of storage space between the two drives in total sector size.

As displayed by "fdisk -l" as requested (for the #2 drive only since I already cleared out the partition table of drive #1):

Disk /dev/sdc: 58.8 GB, 58787364864 bytes
90 heads, 26 sectors/track, 49067 cylinders, total 114819072 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x75cbf5af

Device Boot: /dev/sdb1
Start: 94976
End: 114819071
Blocks: 57362048
Id: c
System: W95 FAT32 (LBA)

Also, this doesn't explain why there is 45MB or 46MB of unallocated space prior to the first partition, but for 2 identical 64GB drives when I run a "dd if=/dev/sdb of=/flash_drive_dd.img bs=1M" command (which should give me an indication of the entire storage capacity of the drive right?) I get two completely different numbers:

Drive #1 displays 63,283,658,752 file size

Drive #2 displays 58,787,364,864 file size

Those values are for the full drive... not just a partition of supposedly identical model 64GB drives.

Regarding the possibly related drastically different sector count on two identical devices (Perhaps this should be a different question post?) Is Overprovisioning possibly occurring ---> In light of realizing that two idential 64GB drives show wildly different available sector sizes (one coming nowhere near close to 64,000,000,000 MB) Does that mean one is overprovisioning or something while the other isn't?

The way I figure out what I should see in my OS is the following: I take 64,000,000,000 bytes and divide by 1024 ^ 3 and I SHOULD get 59.6 GB being reported by the OS (or very close).

Even if there is some "overprovisioning" (is there such a thing for USB flash drives like for SSD) at least I would expect that to be consistent and not wildly different sizes between two identical model drives.

But, fresh out of the box:

Drive #1 only has 63,283,658,752 sectors for the entire device which is 58.9G. If I've done my math correctly; I'm missing over 700 MB for this drive.

And

Drive #2 only has 58,787,364,864 which is 54.8G And again, if I've done my math correctly, I'm missing 5.2 GB for the second drive.

That's disconcerting.... identical models with WAY less space (for the entire drive) than I would expect... the second drive being way less in size than the other.

Even it was overprovisioning of some sort; I would think it should be consistent no?

That doesn't even explain the 45 or 46 MB worth of unallocated space at the beginning with seemingly executable data.

Things keep getting weirder and weirder unless I'm missing something fundamental about USB flash drives.

Sorry this is a little long... I keep discovering more things and adding as I go forward.

  • 3
    The strings (or rather the files that contained them) may simply be part of an automated test system that checks the drive for errors before releasing it to shipping. – Daniel B Mar 20 '15 at 23:23
  • 1
    Some of those strings (e.g. An application has made an attempt to load the C runtime library incorrectly) are part of MSVCRT. Apparently, the runtime library is statically linked to whatever executable was present there. Earlier strings, such as "Invalid partition table. Error loading operating system. are part of the Windows MBR/bootloader. – bwDraco Mar 20 '15 at 23:33
  • A strings search of the entire drive is meaningless; you will have to show the partition table so we can see what you are talking about: i.e. fdisk -l. – psusi Mar 21 '15 at 1:36
  • What is the brand and model of drive in question? With all the details you are providing you seem to have skipped this one critical piece of info. Also, why not just reparation and reformat the disk before using? – JakeGould Mar 21 '15 at 4:51
  • Silicon Power Marvel M70 64GB. I usually do a ton of research when purchasing products and have leaned towards this brand for a number of reasons and their very high reputation especially based on the Silicon Power A80 external drive. I could simply repartition; but I'd like to know the reason or likely reason to have a bunch of hidden data on a drive and does that potentially indicate any other security concerns. And now with new analysis of a second identical model that is 4,496,293,888 (4.5 Gigs) of less total blocks available on the device. Easy explanation or concerning? – mike2000 Mar 21 '15 at 6:07
1

The unallocated space will be used to replace memory cells which fail. Flash memory devices are manufactured with considerable excess, and when a cell fails, it can no longer be written to, but it can be read from, At that point, the controller on board will copy the memory content to a new, unallocated, cell, and then mark the old cell out of use.

  • This area known as L1 or L2 Overprovisioning in the case of an SSD can be one line of reasoning. However, the area that you speak of is untouchable from the user or OS perspective. Also, no other USB drive has this much "unallocated" space. I can repartition the entire device and reclaim this space if I wanted. My issue is why is it there. What is it's purpose... other drives come with 32 sectors (16K) unallocated, or 8064 sectors (3.9 MiB) unallocated prior.. Never have I found 45MiB OR 46MiB OR 31MiB (seemingly random) amount unallocated prior to main partition. – mike2000 Apr 9 '15 at 5:52
  • Well, then, I suggest going back to Silicon-Power.com and writing them, trying with gentle persistence until an intelligent answer is received. – K7AAY Apr 9 '15 at 23:24
  • Have done so. Will update this thread when I get a response. Thanks. – mike2000 Apr 11 '15 at 0:39
0

Poor quality control practices. Memory is notoriously difficult to manufacture with any level of consistency. Generally ram is made, it is tested to see what size it is, and then the space is formated to appear as that size. As far the strings in the unallocated space, I am guessing that is from their quality control test before the product ships.

  • That kind of explains the variation in the amt of raw storage. However, I'm still puzzled why there is a varying amount of "unallocated" space prior to the main partition from the factory; alignment is one thing.. but 45 MB and 46 MB?. With standard process, I would at least expect to see consistency in the actual amount of unallocated space. That said, there's probably an explanation and very well could be what you said regarding quality control. However, I would imagine quality control diags don't really need to be on the drive itself but, on the testing machine. Thanks for your ideas. – mike2000 Mar 28 '15 at 19:40
  • Note that the testing machine must be writing to and reading from the drive if you want the results to show the quality of the drive. However, this does not explain the large variation in the unallocated space. – milesrf Feb 29 '16 at 5:26
0

[sorry4necro?bumping] Hey mike 2000 ( "Will update this thread when I get a response" ) did you ever get response?

I have Kingston Hyperx Savage 64GB with same thing as you. There (https://www.kingston.com/us/support/technical/products?model=hxs3) is an answer:

The full capacity of my DataTraveler or Flash media is not available for storage. Why?

Some of the flash storage device's listed capacity is used for formatting and other functions and thus is not available for data storage, therefore you will not see the full capacity.

When a flash storage device is manufactured, steps are taken to ensure that the device operates reliably and permits the host device (computer, digital camera, PDA, etc.) to access the memory cells; i.e., to store and retrieve data on the flash storage device. These steps, loosely called "formatting," utilize some of the memory cells within the device and thus reduce the capacity available for data storage by the end-user.

Formatting includes the following operations:

Testing each memory cell in the flash storage device. Identifying all defective cells and taking steps to ensure that no data will be written to or read from a defective cell. Reserving some cells to serve as "spares." Flash memory cells have a long but finite lifetime. Therefore, some cells are held in reserve to replace any memory cells that may fail over time. Creating a File Allocation Table (FAT) or other directory. To enable flash storage devices to conveniently store and access customer files, a file management system must be created to allow any device or computer to identify the files stored in the flash storage device. The most common type of file management system for flash storage devices is the File Allocation Table (FAT), which is also used on hard drives. Reserving some cells for use by the flash storage device's controller, e.g., for storing firmware updates and other controller-specific information. Where applicable, reserving some cells for special features. For example, the specification for Secure Digital (SD) cards requires reserved areas to support special copy protection and security features.

End of quoted answer.

Now, I am interesting: Following that logic, flash drive with 128GB capacity would have 90 (double of 45) MB unalocated space, flash drive with 256GB capacity 180MB... and so on. I don't have that big flash drive, but will be interesting to know...

  • Are you asking questions here or writing an answer? – Pimp Juice IT Sep 30 '17 at 15:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.