Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Quite often when I'm in a rush, I automatically pull out a USB pendrive or USB cable from an external hard drive from my desktop PC or laptop, without right-clicking on the safely remove icon in the system tray and unplugging via this route. So far nothing untoward has happened every time I have "been in a rush".

What is the rationale behind right clicking on the safely remove icon and can I really lose information on USB media if this is not carried out ?

Edit:

Is the vunerability of losing such information greatly increased, if the usb media is still flashing at the time of pulling it out of the computer (as opposed to non-flashing) ?

share|improve this question
4  
@woliveirajr, this is nothing similar to the duplicate? One is asking about ejecting and refers to CD ROMs and Kindles where as this is about understanding if information could be lost if removed unsafely. Happy to be wrong, but I can't see it. –  Dave Rook Sep 18 '13 at 14:51
7  
2  
@Simon : I wanted, indeed, the superuser.com/questions/619076/… question, but it was marked as duplicate of the one that is linked in my comment. And the answer from "Augustus Francis" is about your question –  woliveirajr Sep 18 '13 at 17:09
3  
ahem –  wim Sep 18 '13 at 18:52
1  
Try using left-click on the safely remove icon to safely remove a device instead of right-click. It is faster. –  djhurio Sep 20 '13 at 7:05
show 24 more comments

We're looking for long answers that provide some explanation and context. Don't just give a one-line answer; explain why your answer is right, ideally with citations. Answers that don't include explanations may be removed.

marked as duplicate by Synetech, Kevin Panko, Moses, Scott, kinokijuf Dec 22 '13 at 18:56

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

9 Answers

up vote 101 down vote accepted

Yes it can, it's about what happens if you remove the device when it's in use (reading or writing).

When you plug in a USB drive, you give your PC free rein to write and read data from it; some of which is cached.

Caching occurs by not writing information immediately to the USB device, and instead keeping it in your PC's memory (RAM). If you were to yank the USB drive out of your PC before this information is written, or while its being written, you'll end up with a corrupted file.

However, Windows automatically disables caching on USB devices, unless you specifically say that you want it enabled. For the most part you don't have to click the 'Safely Remove Hardware' button, if you aren't writing or reading anything from the device.

Its there simply as an extra level of security preventing you from destroying your own files.

Doing so causes the files to close "gracefully", preserving data, pointers and file size indicators. When writing to disk the computer doesn't always "flush" a buffer and only part of the data may have been written. Using the proper procedure will assure that the data and pointers are in good shape.

Source

share|improve this answer
40  
I would like to add that while caching is disabled by default on USB pen drives, people have been known to change those defaults. So please do not yank out any drives on a random computer. (The reason to change it is a huge speed boost). –  Hennes Sep 18 '13 at 11:20
    
I lost time (but not data) once like this. The file never made it over to the flash drive because I was in a hurry. Never happened again, though, I guess caching is not very common. –  Thomas Sep 18 '13 at 15:16
    
I've seen "delayed write" errors when this happens. –  TecBrat Sep 18 '13 at 17:51
4  
I think it's more than just cache... Windows will actively deny safe removal if the drive is in use. –  Ben Voigt Sep 18 '13 at 18:49
4  
I'd give this answer a +1 if it also mentioned the fact that Windows makes sure no files are currently open on the drive. Lots of processes like explorer.exe do silly things with files on your drive without you knowing about it, and it's good to make sure none of that Tom Foolery is going on before you unplug it. Not to mention that some people like to do even more silly things like open an Excel spreadsheet directly on the drive, then forget Excel is open. –  Phil Sep 18 '13 at 22:18
show 7 more comments

A second reason is that flash drives need to have stable power for ~0.25 seconds after a write command. This is a fundamental physical problem, due random factors some writes may leave a logical 1 bit in a electrical 0.72 state. The fix is easy: just rewrite the bit, perhaps even a few times. Eventually it will stick.

If you're really unlucky, the bit falling over will be in a filesystem table and corrupt e.g. an entire directory.

share|improve this answer
20  
This is interesting - can I get a reference where I can read more about that happening? –  SqlRyan Sep 18 '13 at 14:28
    
Is there any utility that you can use to check whether directories have been corrupted? –  JFW Sep 18 '13 at 14:54
    
@JFW Yep; just open the flash drive and navigate through the folders. If anything is missing, or it gives you a message like 'directory may be corrupted', you know that there is corruption (or at least that the dirty bit was never cleared, which usually means there is corruption). –  AJMansfield Sep 18 '13 at 15:01
9  
@SqlRyan, here's a paper on the effects of power loss and write errors –  Sean Cheshire Sep 18 '13 at 16:06
1  
@JFW: On Windows, the standard error checking will try this. (Found in the property window of each drive). @ AJMansfield: flash corruption due to power cuts don't work that way. The dirty bit may have been cleared by the last write, ending at a reasonable 0.023, whereas another (metadata) bit ended up as a 0.59 requiring a rewrite. It's not a per-sector property. (Flash doesn't work on sector basis anyway, too finegrained.) –  MSalters Sep 18 '13 at 19:59
show 4 more comments

Few people realize everything that goes on under the hood of a flash drive. Unlike hard drives in which a sector is erased and overwritten in a single step, flash drives can write data to a blank 528-byte page much faster than they can perform an erase cycle. Additionally, since each erase-control circuit would use up a fair amount of silicon, flash chips generally don't have a separate circuit for each page but instead have one per "block", where each block contains hundreds or thousands of pages. This means that every time different data is written to a sector, it will be written to a previously-blank area and the old copy marked somehow as "out-of-date". At some point, the flash will perform a "garbage-collection" cycle where it will identify a block that contains mostly out-of-date pages, copy all pages that aren't out of date from that block to another block which has space, and then erase the block. Because such operations may be very slow, some drives will attempt to, when possible, perform them when nothing else was going on; drives don't tell the computer when they perform such cleanup, because the computer generally won't care. The "gotcha" is that if the computer doesn't let the drive know that it's about to lose power, the drive will have no way of knowing when it's safe to perform such cleanup.

Note that although quality flash drives are designed to ensure that a new copy of data will always be written completely before an old copy is erased, and ensure that they can always figure out which copy of each block should be considered the most current, not all flash-drive designs are perfect. In the absence of any power-fail notification, an algorithm which is designed to be 100% proof against data loss or inconsistency would probably be at least twice as slow as one meeting looser constraints. Adding a hardware power-fail notification which would trip while there was still enough energy to complete one or two flash-write operation would reduce the time penalty, but making power-fail notifications 100% reliable can be tricky, and not all implementations are perfect.

share|improve this answer
    
528-byte? Do you mean 512? –  Alvin Wong Sep 20 '13 at 14:54
1  
@AlvinWong: The pages are 528 bytes. Writing a 512-byte sector requires not only storing the data from it, but also an indication of what was sector that data belongs to. At any moment in time, a drive is apt to know "I've received 5,271,591,371 write requests so far, and I'll use page 15 of block 57 for the next one.". If it receives a request to write 512 bytes of data to sector 147, then page 15 of block 57 will receive those 512 bytes of data plus an indication that they represent the 5,271,591,372nd write request, which was to sector 147. –  supercat Sep 20 '13 at 16:41
    
If the system receives a request to read sector 147, there may be many pages on disk which hold a sector 147, but (absent another request to write sector 147), none of them will have a sequence number higher than 5,271,591,372; thus, page 15 of block 57 will be recognizable as the "current" version of that sector, and any lower-numbered copies will be recognizable as obsolete. Note that in theory, the system could respond to a "read sector 147" request by scanning all pages to find which one held the most current version, but that would be hideously slow. –  supercat Sep 20 '13 at 16:48
    
More typically, a system will use some rather interesting data structures to keep in flash a mostly-up-to-date table of where everything is located, as well as an indicator for which physical pages in the flash may have been written since the table was last updated. If the hardware design is such that any attempted page write will complete fully using stored energy even if power is removed, the fact that the drive writes a page that says "the 5,271,591,372nd write request was sector 147, and here is its data" will be considered to update that page. –  supercat Sep 20 '13 at 16:54
add comment

Windows itself tells you that you don’t need to use the Safely Remove Hardware option if you use certain settings – the default settings.

Quick Removal vs. Better Performance

Windows allows you to optimize your USB device for quick removal or improved performance. By default, Windows optimizes USB devices for quick removal. You can access this setting from the device manager – open the Start menu, type Device Manager, and press Enter to launch it.

Expand the Disk drives section in the Device Manager, right-click your device, and select Properties.

Select the Policies tab in the Properties window. You’ll notice that Windows says you can disconnect your USB device safely without using the Safely Remove Hardware notification icon, so this means you can unplug your USB device without ever safely removing it, right? Not so fast.

Data Corruption Danger

The Windows dialog shown above is misleading. If you unplug your USB device while data is being written to it – for example, while you’re moving files to it or while you’re saving a file to it – this can result in data corruption. No matter which option you use, you should ensure that your USB device isn’t in-use before unplugging it – some USB sticks may have lights on them that blink while they’re being used.

However, even if the USB device doesn’t appear to be in-use, it may still be in-use. A program in the background may be writing to the drive – so data corruption could result if you unplugged the drive. If your USB stick doesn’t appear to be in-use, you can probably unplug it without any data corruption occurring – however, to be safe, it’s still a good idea to use the Safely Remove Hardware option. When you eject a device, Windows will tell you when it’s safe to remove – ensuring all programs are done with it. Write Caching

If you select the Better Performance option, Windows will cache data instead of writing it to the USB device immediately. This will improve your device’s performance – however, data corruption is much more likely to occur if you unplug the USB device without using the Safely Remove Hardware option. If caching is enabled, Windows won’t write the data to your USB device immediately – even if the data appears to have been written to the device and all file progress dialogs are closed, the data may just be cached on your system.

When you eject a device, Windows will flush the write cache to the disk, ensuring all necessary changes are made before notifying you when it’s safe to remove the drive. Ultimately, no matter which option you use, you should use the Safely Remove Hardware icon and eject your device before unplugging it. You can also right-click it in the Computer window and select Eject. Windows will tell you when it’s safe to remove the device, eliminating any changes of data corruption.

Note This advice doesn’t just apply to Windows – if you’re using Linux, you should use the Eject option in your file manager before unplugging a USB device, too. The same goes for Mac OS X.

share|improve this answer
1  
In the Linux command line, you could also use the umount /path/to/device. :D –  Justin Pearce Sep 18 '13 at 15:31
1  
@JustinPearce Which is distinct from eject /dev/ice. –  Michael Kjörling Sep 18 '13 at 15:36
1  
@AugustusFrancis Thankyou for mentioning that this "problem" is not just applicable to Windows but affects other OSes as well. –  Simon Sep 21 '13 at 9:46
add comment

Failing to safely remove your devices causes phantom devices to remain in Windows. This isn't usually an immediate problem, but once you've amassed a few hundred phantom devices things start to go wrong.

EDIT FOR CLARIFICATION: Phantom devices are, in short, devices that have drivers installed but are no longer connected to the USB port. Windows keeps track of devices that have been installed, but because the devices hasn't been "safely removed" the driver is still loaded for that USB port. This can cause a variety of problems ranging from assigning a new drive letter when you plug in your flash drive to causing video games to detect controllers that aren't there. When you safely remove a device from Windows you're not just making sure it isn't writing data anymore, you're telling Windows to unload the driver from that USB port. Unplugging your drive on the fly not only risks data corruption, but also risks creating a phantom device that Windows doesn't really handle well.

By default, Windows doesn't show phantom devices in Device Manager, which is why most people are unfamiliar with the concept. To find these devices manually run Command Prompt as an Administrator and enter set devmgr_show_nonpresent_devices=1. This will allow Device Manager to show the phantom devices for the current session.

It is quite possible that there are many too many phantom devices to easily remove them all by hand. GhostBuster is a lightweight GUI designed to easily display and remove phantom devices.

Reference: https://windowssecrets.com/newsletter/how-to-prevent-and-remove-phantom-devices/

share|improve this answer
    
I have never come across the term "phantom devices", thankyou for bringing it to the table..so to speak :) –  Simon Sep 18 '13 at 21:34
    
I hadn't heard about them until just the other day when one of my friends posted this nifty little GUI. GhostBuster –  acarbonaro Sep 19 '13 at 13:09
    
Is it possible to explain in your answer above, more about "phantom device effect", as no one else has mentioned this. Just in case the link in your answer goes dead at some point, if you don't mind. –  Simon Sep 19 '13 at 15:23
    
Hopefully I've added enough info, but if you think that it needs more I'll gladly add whatever's necessary. –  acarbonaro Sep 19 '13 at 16:02
    
Thats fine. Thanks very much –  Simon Sep 19 '13 at 16:22
show 1 more comment

In general, when the computer is writing to the USB drive, you run the risk of corrupting the file system or files when you pull it out without safely removing it.

FAT32

Then, when you have a FAT32 disk using Windows' default settings, Windows won't cache the writes so you can yank the drive out when it is not being written to.

However, you can improve FAT32 performance by enabling write caching. However, then you must safely remove the drive or otherwise not all data is written to the disk.

enter image description here

NTFS

And when you use NTFS, since it is a journaling file system, you can't break the file system by yanking out the drive. However, you will lose or corrupt files if you do that.

Surprise removal doesn't result in corruption, at least from NTFS's point of view. The file system data structures remain intact (or at least can be repaired from the change journal) regardless of when you yank the drive out of the computer. So from the file system's point of view, the answer is "Go ahead, yank the drive any time you want!"

Since NTFS is a journaling file system, it can auto-repair the internal data structures that are used to keep track of files, so the drive itself remains logically consistent. The file is correctly set to the final size, and its directory entry is properly linked in. But the data you wrote to the file? It never made it.

Even though NTFS is robust and can recover from the surprise removal, that robustness does not extend to the internal consistency of the data you lost.

-- Raymond Chen

share|improve this answer
    
I have NTFS corruption cases on the not very often (thankfully!), but regular basis. –  Free Consulting Dec 19 '13 at 22:52
add comment

Have you ever noticed that sometimes, when "safely" ejecting a USB Flash Drive, it takes several seconds until you're told it's safe to remove it? And that during this time, the disk keeps flashing its activity light?

I do see this often, and that alone is indication enough for me not to risk yanking out a disk even when it appears never to lose any data when I yank it out accidentally sometimes.

A bit more background on what may be going on:

Possible disk corruption.

Others have already explained them pretty well, especially the one by supercat is pretty important for flash drives.

Possible program issues.

This is where programs that have files open on the disk are not well prepared to handle the case where the files suddenly become inaccessible. These programs might crash or freeze up then, depending on how well they were developed.

Therefore, by using the "remove hardware safely" command, you start a process in the operating system that asks these programs to let go of the disk. Once all programs have let go if it, you are told that you can now safely disconnect the disk.

However, if there's a program running that does not let go of the disk, then you will be told so as well, and then you have the option to close that program so that it stops using the disk, preventing both data corruption on the disk and program issues on your computer.

Even more background on caching and disk corruption

A disk that stores files works like a library with indexes. The challenge is that the index needs to be kept in sync with the books (files) it contains. When you add/remove a file on a disk, its index (i.e. the directory) needs to be updated as well. All this happens in several steps, and yanking the disk out in the middle of it leads to corrupted disk structures (to identify and repair them, tools such as "chkdisk" come to the rescue).

Howevever, keeping the directories up-to-date all the time requires a lot of writing and time. So, if you write 100 files to the disk, the operating system (e.g. Windows) does not write each file followed by an update of the (often same) directory, but optimizes this operation by first writing the data of the 100 files to disk, and only then write the updated directory in one go.

Such an optimization makes copying (and deleting) files much faster, and modern operating systems tend to delay the update of the directory as much as possible to speed the computer up even more.

I.e, the operating system keeps the correct directory only in its local memory (RAM or internal hard disk, which is usually still faster then the USB connection). This is the caching that others speak of: Keeping data in a separate location for faster access.

But what if the entire computer has a power failure? Well, even then it has ways to recover from it as soon as it gets powered on again, keeping the directory intact (using "journalling", for example). Of course, that only works if the disk whose directory was cached is still present after power comes back - which is usually the case, so nothing to worry here.

But all these safeguards can't work if you simply remove the disk and do not give the operating system to write its cached directory back to disk, obviously.

Therefore, you announce to the operating system (and the programs it runs) that you intend to remove the disk. This process, also called "unmounting", leads to writing all cached data to the disk and then stopping further access to it.

share|improve this answer
add comment

Flash is only able to write one's and no zeros. It's able to erase a whole block (setting it to zeros). Because most flash media uses Wear Leveling it only adds a block to a "to be deleted" list and uses another block instead. The controller on your usb media begins erasing blocks on the "to be deleted" list whenever nothing else has to be done. Plugging the power while blocks are being erased may let them in a undefined state.

share|improve this answer
    
If this was the answer, then it would imply that you could never safely remove a USB drive because wear leveling is done by the drive's controller, not by the operating system. –  Michael Kjörling Sep 18 '13 at 15:38
    
But there is a difference between eject and unmount. In some operating systems (Mac OS X) it's possible to unmount a drive but not eject it. If you eject a memory stick you should not be able to mount it again because the device shuts down. –  frog32 Sep 18 '13 at 15:47
    
Anything at this level should be transactional - the block would not be removed from the "to be deleted" queue until it was actually deleted. Its state would not be undefined - if the system didn't complete the erase, it should just pick up (from the beginning of the block) next time. –  GalacticCowboy Sep 18 '13 at 16:34
    
@GalacticCowboy: Drives are supposed to be transactional, but not all implementations are equally robust. If write operations are robustly transactional, making erase operations (which take longer) robustly transactional too isn't too hard, but it may be hard to ensure that it's impossible for power loss during an write cannot possibly result in a bit which reads as written 99% of the time but occasionally reads as blank. One may be able to reduce the probability of such an event below that of the board being struck by a meteor, but it's hard to eliminate it entirely. –  supercat Sep 18 '13 at 16:58
    
@supercat - That doesn't change what I said, however - if the drive lost power during the erase, the queue would not be modified and the block would just be erased later. If power is lost while rewriting the queue, yeah, that could cause issues, but as you noted in your answer, the write is significantly faster (so less likely to be interrupted). Still not impossible, just not something you probably need to put any significant worry into. –  GalacticCowboy Sep 18 '13 at 17:04
show 1 more comment

What is the rationale behind safely removing USB media from windows PCs?

This function is for removable media. That is, media that is connected via USB, FireWire, and as such, can be disconnected at any given moment.

Accessing the drive to write every piece of data to it as requested would put a lot more burden on the drive which would reduce performance and increase wear. To increase performance and reduce wear and tear, Windows can cache data in order to write a bunch of data at a time. It’s like delaying a trip to the other end of town until you have enough errands to do there to make the trip worthwhile.

The problem with caching is that if the drive happens to be unplugged between the time that the write request was made and the time that Windows “flushes” the cache to disk (actually performs the write), then that data would not get written. This leads to data loss and possibly even corruption.

By “ejecting” a removable volume, you tell Windows to flush the cache and write any data it has waiting. That way you ensure that it is safe to unplug the drive.

Quite often when I'm in a rush, I automatically pull out a USB pendrive or USB cable from an external hard drive from my desktop PC or laptop, without right-clicking on the safely remove icon in the system tray and unplugging via this route. So far nothing untoward has happened every time I have "been in a rush".

That’s mostly because Microsoft is not stupid (lets not argue here…) They know that people don’t like doing extra work. Therefore, when you plug a removable media into the system for the first time, when Windows installs drivers for it, part of the configuration it does is to set it for “quick removal”.

In the Device Manager, in the Policies tab of the Properties dialog for your drives, depending on the type of drive, you can choose to configure the drive for either quick-removal or better-performance or enable or disable write caching.

Enabling write-caching or selecting better-performance will cause Windows to queue write requests which increases performance and reduces wear, but also increases chances of data-loss and requires ejecting the volume before removing it. Disabling write-caching or selecting quick-removal causes Windows to write everything to the disk immediately which reduces performance and increases wear, but makes it okay to simply pull the drive out.

Because Windows defaults removable media to quick-removal, most people will rarely experience a problem from just unplugging it without ejecting first.


The screenshots below show the default drive policies for a variety of media. There are few interesting points to notice.

The internal hard-drive is set to use write-caching (figure 1). This makes sense because it is assumed that the drive is a permanent fixture, so it will never normally become unavailable in the short period between a write request and a cache flushing.

The USB flash-drive is set for quick-removal (figure 2). This too makes sense because it makes it convenient to remove it easily.

The USB memory-card is configured for quick-removal (figure 3). This is logical because it is essentially no different than a USB flash-drive (or any other USB drive). What’s interesting is that the internal memory-card reader is also set to quick-removal (figure 4). The reader itself is built into the laptop, and therefore always present, but the card can be removed, and so it defaults to quick-removal. (This is the same for floppy drives; there’s no caching because the disk in the drive could be removed.)

Even more interesting are the external USB hard-drive (figure 5), and iPod mini (figure 6) which has a little, spinning hard-drive in it. These are actual hard-drives, not flash-drives, but because they are removable, they still default to quick-removal. Again, this makes sense because they can be unplugged before the data is written to the disk.

Can I really lose information on USB media if this is not carried out ?

Is the vunerability [sic] of losing such information greatly increased, if the usb media is still flashing at the time of pulling it out of the computer (as opposed to non-flashing) ?

Yes. The problem is that even if a drive is set to quick-removal does not make it safe to unplug it right away. Just because Windows performs the writes immediately, does not mean that they occur immediately.

Most removable media such as flash-drives and memory-cards are much slower than hard-drives. Therefore, even though Windows initiates the write as soon as it is requested, it will usually take a few moments for the data to actually get written. Even if the write/copy/etc. dialog box that shows the write progress goes away, the drive may still be writing for a few more moments, and if it has its own drive-activity LED, you can see this. It is not safe to remove the media right away; watch the LED and wait for it completely stop flashing.


To be extra safe, you can use a program like Sync tell Windows to flush the cache. Essentially, it is exactly the same as using the safely-eject function, but can be a little more convenient if you make a shortcut on the desktop to run it with the specific drive-letter of your removable drive. It will save about 5-10 seconds which doesn’t sound like much, but in HCI, is a huge barrier. What’s even better is that it only flushes the cache, it does not actually eject the drive. Therefore, the drive is still mounted and available for use if you suddenly remember something you forgot to do with it. This makes it even more useful because you can even assign it a hotkey so that you can flush the cache and make it safe for unplugging with a keystroke.


Figure 1: Internal HDD policy

Internal HDD policy

Figure 2: USB flash-drive policy

USB flash-drive policy

Figure 3: Internal memory-card reader policy

Internal memory-card reader policy

Figure 4: USB memory-card reader policy

USB memory-card reader policy

Figure 5: External HDD policy

External HDD policy

Figure 6: iPod (mini) HDD

iPod mini HDD policy

share|improve this answer
add comment

protected by Diogo Sep 20 '13 at 13:53

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?

Not the answer you're looking for? Browse other questions tagged or ask your own question.