Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

When a file is deleted, its contents may still be left in the filesystem, unless explicitly overwritten with something else. The wipe command can securely erase files, but does not seem to allow erasing free disk space not used by any files.

What should I use to achieve this?

share|improve this question
    
The only safe solution may be to save your files elsewhere, wipe the whole partition, recreate the filesystem, and then restore your files. I've run photorec and was shocked by how much stuff could be retrieved even after 'wiping' free space. A compromise solution is to move the left boundary of your partition by 6% of its size after having wiped the apparently free space. –  user39559 Sep 7 '10 at 12:12

11 Answers 11

up vote 58 down vote accepted

Warning: Modern disk/SSD hardware and modern filesystems may squirrel away data in places where you cannot delete them, so this process may still leave data on the disk. The only safe ways of wiping data are the ATA Secure Erase command (if implemented correctly), or physical destruction. Also see How can I reliably erase all information on a hard drive?

You can use a suite of tools called secure-delete.

sudo apt-get install secure-delete

This has four tools:

srm - securely delete an existing file
smem - securely delete traces of a file from ram
sfill - wipe all the space marked as empty on your hard drive
sswap - wipe all the data from you swap space.

From the man page of srm

srm is designed to delete data on mediums in a secure manner which can not be recovered by thiefs, law enforcement or other threats. The wipe algorithm is based on the paper "Secure Deletion of Data from Magnetic and Solid-State Memory" presented at the 6th Usenix Security Symposium by Peter Gutmann, one of the leading civilian cryptographers.

The secure data deletion process of srm goes like this:

  • 1 pass with 0xff
  • 5 random passes. /dev/urandom is used for a secure RNG if available.
  • 27 passes with special values defined by Peter Gutmann.
  • 5 random passes. /dev/urandom is used for a secure RNG if available.
  • Rename the file to a random value
  • Truncate the file

As an additional measure of security, the file is opened in O_SYNC mode and after each pass an fsync() call is done. srm writes 32k blocks for the purpose of speed, filling buffers of disk caches to force them to flush and overwriting old data which belonged to the file.

share|improve this answer
3  
It's hard to locate the current "official" homepage of secure-delete. A perhaps older version claims there are no bug reports, but at the same time there is no open bugtracking system where I could report a bug that I have found. The secure-delete homepage also points out that it may not wipe all the unused blocks of data, depending on the filesystem that you use, which is true. –  user39559 Sep 7 '10 at 12:10
7  
With modern hard disks (bigger than around 20 GB), it is totally useless to do several passes and wait for ages. So installing specialized tools has also become useless (which may explain why secure-delete has no more home page). Just do this from the appropriate partition: cat /dev/zero >nosuchfile; rm nosuchfile. –  mivk Nov 4 '11 at 11:47
1  
@mivk: Why is it useless to do more than one pass? And why use /dev/zero instead of /dev/random? Is that due to speed concerns? –  naught101 Jan 5 '13 at 12:14
2  
Using /dev/zero is much faster. If you write free space from /dev/random, the kernel has to generate all that random data on the fly. It's an entertaining way to watch your load average jump up to the maximum... –  dafydd May 26 '13 at 2:38
    
The question of whether multiple wipes are necessary is answered here: Why is writing zeros (or random data) over a hard drive multiple times better than just doing it once? –  sleske Jan 28 at 10:25

The quickest way, if you only need a single pass and just want to replace everything with zeros, is:

cat /dev/zero > zero.file
rm zero.file

(run from a directory on the filesystem you want to wipe)

There will be a time during this operation when there will be no free space at all on the filesystem, which can be tens of seconds if the resulting file is large and fragmented so takes a while to delete. To reduce the time when freespace is completely zero:

dd if=/dev/zero of=zero.small.file bs=1024 count=102400
cat /dev/zero > zero.file
rm zero.small.file
rm zero.file

This should be enough to stop someone reading the old file contents without an expensive forensic operation. For a slightly more secure, but slower, variant replace /dev/zero with /dev/urandom. For more paranoia run multiple steps with /dev/urandom, though if you need that much effort the shred utility from the coreutils package is the way to go:

dd if=/dev/zero of=zero.small.file bs=1024 count=102400
shred -z zero.small.file
cat /dev/zero > zero.file
rm zero.small.file
shred -z zero.file
rm zero.file

Note that in the above the small file is shredded before creating the larger, so it can be removed as soon as the larger is complete instead of having to wait for it to be shredded leaving the filesystem with zero free space for the time that takes. The shred process with take a long time over a large file and unless you are trying to hide something from the NSA isn't really necessary IMO.

All of the above should work on any filesystem.

File Size Limits:

As DanMoulding points out in a comment below, this may have problems with file size limis on some filesystems.

For FAT32 it would definitely be a concern due to the 2GiB file limit: most volumes are larger than this these days (8TiB is the volume size limit IIRC). You can work around this by piping the large cat /dev/zero output output through split to generate multiple smaller files and adjust the shred and delete stages accordingly.

With ext2/3/4 it is less of a concern: with the default/common 4K block the file size limit is 2TiB so you'd have to have a huge volume for this to be an issue (the maximum volume size under these conditions is 16TiB).

With the (still experimental) btrfs both the maximum file and volume sizes are a massive 16EiB.

Under NTFS the max file length is larger than max volume length in some cases even.

Starting points for more info:
http://en.wikipedia.org/wiki/Ext3#Size_limits
http://en.wikipedia.org/wiki/Btrfs
http://en.wikipedia.org/wiki/Ntfs#Scalability

share|improve this answer
2  
The simple zeroing can apparently also be done with the secure-delete tools: using sfill -llz reduces the whole procedure to one pass which only writes '0's. –  foraidt Oct 3 '10 at 14:38
    
This takes a while. Is it really the quickest way? I guess writing GB of data will always take a while... –  endolith Jun 15 '11 at 2:51
2  
@endolith: if you want to blank the free space on an active filesystem then you can't get around the need to write that much data via the filesystem overhead. The secure-delete tools suggested by fnord_ix may be faster, because they are optimised for this type of task. –  David Spillett Jun 15 '11 at 12:04
    
Why 'dd' and not 'pv'? –  pbies Dec 5 '13 at 13:53
    
@pmbiesiada: where dd is being used above it is only going to be running for a very short time so the progress indicator of pv is of little use. Also pv is not as universally available as dd and cat. You could replaces the instances of cat with pv though it will only tell you what it has done, not what it has left to do and an ETA. shred has an option to display %done as it progresses too, if you want to monitor the process more closely than just setting it off and waiting for it to finish. –  David Spillett Dec 5 '13 at 17:47

WARNING

I was shocked by how many files photorec could retrieve from my disk, even after wiping.

Whether there is more security in filling the "free space" only 1 time with 0x00 or 38 times with different cabalistic standards is more of an academic discussion. The author of the seminal 1996 paper on shredding wrote himself an epilogue saying that this is obsolete and unecessary for modern hardware. There is no documented case of data being physically replaced zeroes and recovered afterwards.

The true fragile link in this procedure is the filesystem. Some filesystems reserve space for special use, and it is not made available as "free space". But your data may be there. That includes photos, personal plain-text emails, whatever. I have just googled reserved+space+ext4 and learned that 5% of my home partition was reserved. I guess this is where photorec found so much of my stuff. Conclusion: the shredding method is not the most important, even the multi-pass method still leaves data in place.

You can try # tune2fs -m 0 /dev/sdn0 before mounting it. (If this will be the root partition after rebooting, make sure run -m 5 or -m 1 after unmounting it).

But still, one way or another, there may be some space left.

The only truly safe way is to wipe the whole partition, create a filesystem again, and then restore your files from a backup.


Fast way (recommended)

Run from a directory on the filesystem you want to wipe:

dd if=/dev/zero of=zero.small.file bs=1024 count=102400
dd if=/dev/zero of=zero.file bs=1024
sync ; sleep 60 ; sync
rm zero.small.file
rm zero.file

Notes: the purpose of the small file is to reduce the time when free space is completely zero; the purpose of sync is to make sure the data is actually written.

This should be good enough for most people.

Slow way (paranoid)

There is no documented case of data being recovered after the above cleaning. It would be expensive and resource demanding, if possible at all.

Yet, if you have a reason to think that secret agencies would spend a lot of resources to recover your files, this should be enough:

dd if=/dev/urandom of=random.small.file bs=1024 count=102400
dd if=/dev/urandom of=random.file bs=1024
sync ; sleep 60 ; sync
rm random.small.file
rm random.file

It takes much longer time.

Warning. If you have chosen the paranoid way, after this you would still want to do the fast wipe, and that's not paranoia. The presence of purely random data is easy and cheap to detect, and raises the suspicion that it is actually encrypted data. You may die under torture for not revealing the decryption key.

Very slow way (crazy paranoid)

Even the author of the seminal 1996 paper on shredding wrote an epilogue saying that this is obsolete and unecessary for modern hardware.

But if yet you have a lot of free time and you don't mind wasting your disk with a lot of overwritting, there it goes:

dd if=/dev/zero of=zero.small.file bs=1024 count=102400
sync ; sleep 60 ; sync
shred -z zero.small.file
dd if=/dev/zero of=zero.file bs=1024
sync ; sleep 60 ; sync
rm zero.small.file
shred -z zero.file
sync ; sleep 60 ; sync
rm zero.file

Note: this is essentially equivalent to using the secure-delete tool.


Before the edit, this post was a rewrite of David Spillett's. The "cat" command produces an error message, but I can't write comments on other people's posts.

share|improve this answer
    
You can comment under other people posts with 50 reputation. –  Gnoupi Aug 18 '10 at 9:44
1  
The cat command is expected to give a "no space left" error in my examples, at the end of its run. You can hide this by redirecting stderr to /dev/null if it is a problem. I usually use pv rather than cat or dd for this sort of thing, in order to get the useful progress indication. –  David Spillett Jun 15 '11 at 12:09
    
...raises the suspicion that it is actually encrypted data. You may die under torture for not revealing the decryption key. Heh, that's exactly what I was thinking. I guess that means I am paranoid... –  Navin Dec 14 '13 at 23:37

There is zerofree utility at least in Ubuntu:

http://manpages.ubuntu.com/manpages/natty/man8/zerofree.8.html

   zerofree — zero free blocks from ext2/3 file-systems

   zerofree  finds  the  unallocated, non-zeroed blocks in an ext2 or ext3
   filesystem (e.g. /dev/hda1) and fills them with zeroes. This is  useful
   if  the  device  on  which this file-system resides is a disk image. In
   this case, depending on the type of disk image, a secondary utility may
   be  able  to  reduce the size of the disk image after zerofree has been
   run.

   The usual way to achieve  the  same  result  (zeroing  the  unallocated
   blocks)  is to run dd (1) to create a file full of zeroes that takes up
   the entire free space on the drive, and then delete this file. This has
   many disadvantages, which zerofree alleviates:

      ·  it is slow;

      ·  it makes the disk image (temporarily) grow to its maximal extent;

      ·  it  (temporarily)  uses  all  free  space  on  the disk, so other
         concurrent write actions may fail.

   filesystem has to be unmounted or mounted  read-only  for  zerofree  to
   work.  It  will exit with an error message if the filesystem is mounted
   writable. To remount the  root  file-system  readonly,  you  can  first
   switch to single user runlevel (telinit 1) then use mount -o remount,ro
   filesystem.

Also check this link about zerofree: Keeping filesystem images sparse - it is from its author - Ron Yorston (9th August 2012)

share|improve this answer

Here's how to do it with a GUI.

  1. Install BleachBit
  2. Run as root by clicking Applications - System Tools - BleachBit as Administrator.
  3. In the preferences, tell it which paths you want. Generally it guesses them well. You want to include one writeable path for each partition. Generally that is /home/username and /tmp, unless they are the same partition, in which case just pick one.
  4. Check the box System - Wipe Free Disk Space.
  5. Click Delete.

The advance of BleachBit over dd (which otherwise is very nice) is when the disk is finally full, BleachBit creates small files to wipe the inodes (which contains metadata like filenames, etc).

share|improve this answer
    
Inspect Bleachbit's opensource python code for wiping freespace from a drive for your self. –  shadowbq Jan 14 '13 at 15:18

You probably already have the GNU coreutils package installed on your system. It provides the command shred.

share|improve this answer
1  
Shred won't clean up unused disk space without making it into files first... –  dmckee Aug 7 '09 at 15:22

I use dd to allocate one or more big files to fill up the free space, then use a secure deletion utility.

To allocate files with dd try:

dd if=/dev/zero of=delete_me bs=1024 count=102400

This will generate a file named delete_me that is 100 MB in size. (Here bs is the "block size" set to 1k, and count is the number of blocks to allocate.)

Then use your favorite secure deletion utility (I've been using shred) on the files so created.

But NOTE THIS: buffering means even if you do the whole disk, you may not get absolutely everything!


This link recommends scrub for free space wiping. Haven't tried it.

share|improve this answer
    
Oh, if memory serves me, I tried scrub once and it corrupted the whole file-system. Fortunately I had the good sense of first experimenting on a testing file-system, NOT on my real data. –  landroni Aug 30 at 16:33

Easier is to use scrub:

scrub -X dump

This will create a dump folder in the current location and create file until the disk is full. You can choose a pattern with the -p option (nnsa|dod|bsi|old|fastold|gutmann).

It's not easy to get scrub installed (see the Ubuntu Forums on this), but once the installation is done, you've a really SIMPLE and efficient tool in your hand.

share|improve this answer
    
If memory serves me, I tried scrub once and it corrupted the whole file-system. Fortunately I had the good sense of first experimenting on a testing file-system, NOT on my real data. –  landroni Aug 30 at 16:35
    
Don't know what you did or what happen, but scrub basically create new file up until it fill the filesystem. It does not play with existing file, neither does it delete any of them ( at least not the command I gave )... –  FMaz008 Aug 31 at 19:33
1  
Indeed. Tried scrub -X dump_dir and it seems to have worked nicely. BTW, installing on Ubuntu 14.04 is very straightforward: apt-get install scrub. –  landroni Sep 12 at 21:02

use dd and just zero out the free space. it is a myth data needs to be over written multiple times (just ask peter guntmann) and random data , as opposed to 1's then 0's implies unnatural activity. then end result is a clean drive with way less time spent writing. besides, secure deletion programs cant guarentee they even overwrite the real file on modern file systems(journaled). do yourself a favor and get photorec, scan your drive to see the mess, wipe it with 1's and optionally with zeroes to make it look untouched. if photorec still finds stuff, remember it is scanning everything available so do this carefully again with root user.

remember, the cia/fbi/nsa doesnt have a fancy machine that can read the actual state of your magnetic media bits. that was all just a paper written a long time ago. a "what-if". you only need to wipe 1 time.

share|improve this answer
1  
There are few interesting things you've said, but do you actually have any sources to back this information? It's hard to believe that all that overwriting is useless. Also, please improve your post, it's hard to read with punctuation like that. –  gronostaj May 25 '13 at 21:14
    
@gronostaj: The "it is a myth data needs to be over written multiple times" claim for modern drives at least has been proven by multiple studies. All those 30+ passes recommended by Gutmann are no longer required, as acknowledged by the author himself. –  Karan May 25 '13 at 23:47

You can wipe your free space by using secure deletion package.

In that package you can find sfill tool, which is designed to delete data which lies on available diskspace on mediums in a secure manner which can not be recovered by thiefs, law enforcement or other threats.

To install secure deletion package in Linux (Ubuntu), install it by the following command:

$ sudo apt-get install secure-delete

Then to erase your data no free space, try the following command:

sfill -f -v -ll /YOUR_MOUNTPOINT/OR_DIRECTORY

Where /YOUR_MOUNTPOINT/OR_DIRECTORY is your mount point (df -h, mount) or directory to wipe the free space.

Read the manua:

http://manpages.ubuntu.com/manpages/hardy/man1/sfill.1.html

share|improve this answer

Once the file is gone off the file system's record, the data that is left on the hard disk is meaningless sequence of 1's and 0's. If you are looking to replace that meaningless sequence with another meaningless sequence, I can advice some commercial products for safely erasing drives, like arconis.

share|improve this answer
18  
Contiguous chunks of former file contents still remain on disk, and are far from meaningless if raw disk data is examined directly. –  Alex B Aug 7 '09 at 0:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.