139

I have a directory, that contains ~ 3 million files in certain subdirectories on a Windows 2008 server. Manually deleting the files via SHIFT+DEL on the root dir takes ages. Is there any other way to do the deletion in a faster manner?

migrated from stackoverflow.com Aug 7 '09 at 21:37

This question came from our site for professional and enthusiast programmers.

14 Answers 14

178

I regularly need to delete lots of files and directories from a WinXP encrypted drive, typically around 22 GB of 500,000 files in 45,000 folders.

Deleting with Windows Explorer is rubbish because it wastes lots of time enumerating the files. I usually move the stuff I need to delete to C:\stufftodelete and have a deletestuff.bat batch file to rmdir /s/q C:\stufftodelete. This is scheduled to run at night, but sometimes I need to run it during the day so the quicker the better.

Here's the results of a quick time test of a small 5.85 MB sample of 960 files in 303 folders. I ran method 1 followed by method 2, then reset the test directories.

Method 1 removes the files and directory structure in one pass:

rmdir /s/q foldername

Method 2 has a first pass to delete files and outputs to nul to avoid the overhead of writing to screen for every singe file. A second pass then cleans up the remaining directory structure:

del /f/s/q foldername > nul
rmdir /s/q foldername
  • Method 1: 17.5s, 14.9s, 13.9s, 14.8s, 13.8s: average 14.98 seconds
  • Method 2: 14.3s, 12.1s, 11.7s, 14.2s, 11.8s: average 12.82 seconds

Here's results of another test using 404 MB of 19,521 files in 3,243 folders:

  • Method 1: 2 minutes 20 seconds
  • Method 2: 2 minutes 33 seconds

So there's not much in it, probably too close to judge on a single test.


Edit: I've retested with much more data, this is a typical case for me: 28.3 GB of 1,159,211 files in 146,918 folders:

  • Method 1: 2h 15m, 2h 34m: average: 2 hours 25 minutes
  • Method 2: 49m, 57m: average: 53 minutes

Wow, method 2 is nearly three times faster than method 1! I'll be updating my deletestuff.bat!

  • RD /S /Q works great if you specify an absolute path but it does not when working with relative paths. Is there a solution? – Michael S. Jan 31 '12 at 7:25
  • 1
    @Hugo : Hugo, a question: In the above timed tests, for either or both methods, did you count the files immediately before you ran the method? I'm asking because the folder contents might already be in the OS file cache. Thanks! – William C Apr 10 '12 at 15:19
  • @WilliamC: I don't remember exactly, but for the repeated deletes with the same contents I will have counted immediately before only one delete, and won't have repeated it for the other deletes (as I already knew the numbers). – Hugo Apr 10 '12 at 17:21
  • Does your timing include moving the 1,159,211 files in 146,918 folders to the C:\stufftodelete folder? – martineau Jan 19 '14 at 19:55
  • @martineau: No, the test timings were (if I remember correctly) deleting the folders in place. It's usually quick to move a folder to C:\stufftodelete`. Copying is slooow, but you don't want that. If you have another drive (eg E:), also create E:\stufftodelete` and always move files to the same drive's folder and include it in deletestuff.bat (which also recreates the folders at the end for next time). – Hugo Jan 20 '14 at 7:22
18

If you have to delete large directory trees regularly, consider storing the root of that directory tree on a separate partition, then simply quick-format it whenever you need to delete everything. If you need to automate this, you can use this DOS command:

echo Y | format Z: /FS:NTFS /X /Q

where Z: is your 'volatile' partition. Note: the partition must have no label. I blogged about this here.

  • And use a hardlink to create connection between the disk & directory where files stored if exact directory path needed. – Nime Cloud Nov 22 '11 at 12:07
  • 1
    @NimeCloud: Hardlinks cannot cross filesystems or link directories. – grawity Dec 12 '11 at 9:34
  • 5
    However, symbolic links can cross filesystems and link directories, so they might be appropriate here. See mklink command (Vista and later; otherwise use junction from Mark Russinovich/SysInternals.) – Robert Calhoun Jul 19 '12 at 15:37
9

In command prompt (Start -> Run -> cmd):

del /f /s /q foldername 
  • This is great because it lists what its deleting.. but its the same as rd in terms of speed. – bobobobo Aug 8 '09 at 16:10
  • 1
    Oh woah woah woah. Not as good as I thought. It deletes the files quickly but leaves the directory tree intact. – bobobobo Aug 8 '09 at 16:13
  • 1
    Which I might add takes quite a long time to clear out using rd. – bobobobo Aug 8 '09 at 16:15
  • 3
    @bobobobo I did some tests with ~30GB/1,000,000 files/15,000 folders and del+rd is nearly three times faster than just rd: superuser.com/questions/19762/mass-deleting-files-in-windows/… – Hugo Jun 1 '11 at 21:25
  • I think rmdir is faster is some situations. del . /f /s /q is nice because you can also delete the contents of all files within a directory. After you are finished, it is just a matter of deleting the folders which is fast enough. Also, remember attrib -s -h -r . /s if you can not delete folders because of hidden files. – Sun Dec 23 '13 at 17:40
6

In addition to copying/moving files very fast (using its own API), TeraCopy can delete files and it's very fast too. Ever since finding out TeraCopy I don't use a computer without it installed (if I'm gonna copy/move/delete).

The same installer installs x64 edition but to use it you have to manually force it.

The beta which I recommend over the stable versions: http://blog.codesector.com/2010/09/22/teracopy-2-2-beta-3/

  • Have you ever done or seen some like-with-like time comparisons of TeraCopy vs rmdir or del or del+rmdir? – Hugo Jun 1 '11 at 21:29
  • 1
    @Hugo: Stupid question. I wouldn't recommend TeraCopy if it wasn't seenable by plain eye it's faster ! – rautamiekka Jun 20 '11 at 20:50
  • 5
    So do you estimate it's 10% faster? Twice as fast? Ten times as fast? A million times faster? It would great if you could time it against rmdir with two large and identical test sets. – Hugo Jun 20 '11 at 21:05
  • 1
    Thank you! I've been using TeraCopy for years, but never knew about the mass-delete function. And you're right, it's much, MUCH faster! – AzDayton May 23 '17 at 20:11
  • 1
    @Amalgovinus You don't. You either open the program and add files/folders through drag-drop or right-click, or right-click your files/folders in Explorer/on desktop. Once it's done walking them, just click 'Delete' and 'Move to Recycle Bin' if you want, but I favor 'Delete from disk'. However, you'll need to boot TeraCopy as Admin to conf it to show up in desktop/Explorer right-click if it's not done by default. – rautamiekka Jun 6 at 22:48
6

I did a bat file that do the same.

@echo off
echo --------------------WARNING--------------------
echo folder "%~1" will be deleted
echo --------------------WARNING--------------------
pause
echo Deleting folder: "%~1".
time /T
del /f/s/q "%~1" >nul
rmdir /s/q "%~1" >nul
echo Done.
time /T
echo --- Taking ownership.
takeown /f "%~1" /r /d y >nul
icacls "%~1" /reset /t >nul
icacls "%~1" /setowner "%username%" /t >nul
echo Done all.
time /T

it do the work in two step, one, it try to delete the files. And the second is to try to take ownership of the files. y should be changed according your local (stand for yes in english). If fail the task (for example if fail because permission) then you must run it again. However, the second round will not try to delete the files deleted in the first round, so it could be pretty quick.


How to use it. Save as delfolder.bat in a path route (for example c:\windows), then run it as

 delfolder "foldername"   

where foldername is the name of the folder

In my test, deleting 123'000 files took 3 minutes (sata 7200rpm). YAY!

  • 1
    Usually you should not use %1 in a script, but "%~1". This way, if the argument has been passed with quotes, it is dequoted and requoted. If it has not been passed with quotes, it is quoted. – Benoit Jan 23 '18 at 7:28
4

Using the code below usually works well for me.

mkdir empty_dir
robocopy empty_dir dir_to_wipe /mir /r:0 /w:0 /e
rmdir empty_dir dir_to_wipe
  • Add a ` > nul` to the end of that robocopy call so you don't have to see it onscreen. – JoshDM Nov 18 '13 at 18:13
  • It is similar to the use of rsync in Linux. It's even here faster then rmdir? – Hastur Mar 24 '16 at 15:06
  • FYI, this robocopy is very different from del /f/q/s + rmdir /s/q. If your dir_to_wipe has junktions to c:\some_important_dir you will end up losing your data. – Pavel Jan 16 '18 at 18:54
  • @Pavel then use /XJ to exclude junction points and symbolic links? – Bernhard May 22 at 9:48
3

Use the rd /s command from the command prompt.

  • 1
    No. This is as slow as deleting from explorer. – bobobobo Aug 8 '09 at 16:08
  • Well, its alright. Works better than del /f /s /q, anyway. – bobobobo Aug 8 '09 at 16:15
  • 3
    Well, this is MUCH better than using Explorer to delete, actually. It continues to delete, even if a delete or two will fail (where explorer chokes and aborts) – bobobobo Aug 8 '09 at 20:14
2

The best practical solution is probably to move the folder out of the way somewhere (e.g. the Recycle Bin) and then start deleting it. It'll take ages, but at least it'll be out of the way.

I'm pretty sure the time required to delete all those files is an inherent requirement of the task, not an inefficiency in the implementation of deletion.

1

Install Cygwin and use rm -r. But that's likely to be overkill.

  • That's exactly the same as rd /q /s ... – Joey Mar 17 '10 at 9:28
  • Too much task, and almost equivalent speed! – adeelx Apr 4 '10 at 3:51
  • 1
    It's no worse than the several other install-tool-X-and-use-it answers here, and shouldn't be voted down if they are not being. – JdeBP Jun 1 '11 at 22:02
  • 2
    Is cygwin rm really as fast as cmd.exe builtin rd? Because the cygwin emulation of posix interface is rather slow as it requires doing some extra work, that rm does not actually appreciate. – Jan Hudec Jun 18 '13 at 11:59
  • While waiting for a large collection of files to delete I had enough time to install Cygwin and try rm -rf. About 10 minutes in I aborted and went back to del /f/s/q because under Cygwin, disk usage was only 50%. Under del, disk usage is pegged at 100%. So I figure it must be deleting faster. CPU usage is <1% for both methods. I'm not sure on the file number because I closed the Windows File Deleter after it spent 10 minutes calculating. – Derek Ziemba Apr 18 '18 at 4:18
1

Do you have short file name generation enabled? If so, do you really need it? Removing a file is only a metadata opeartion. So if you've got twice the number of names to remove, the amount of work is significantly higher.

0

Have you tried either of these two apps?

Be sure to set the number of overwrites to 0 if you want fastest performance. Do this by clicking options then change the value at the bottom of the dialog.

Enter the path to delete in the Source field and then click Delete.

These apps do not put the files in the recycle bin. Use with care.

0

This is provided to compliment and augment the steps taken in the previously mentioned answers.

The given two methods above seem quite effective, but to determine the performance would be difficult unless they are benched against the exact same conditions:

  • Physical Hard Drive Device & Model
    • Preferably, ATTO/ Crystal Disk benchmarked before cloning the data on them
  • Same Partition/ Volume Location & Geometry
    • The way to do this would be defrag the sample partition/ folders and clone them exactly as they are 2 exactly the same types of Hard Drives (from previous step)
  • Then execute the configurations via Batch File and use Powershell Cmdlet to log and measure their performance.

    • Ideally, add some mechanisms to prevent any caching biases due to recency of similar activity.
  • The following are some sample folders (named as total size) that I experimented on but was not able to get any conclusive performance / results.

r.bat

rmdir /s/q 3.2G

rd.bat

del /f/s/q 3.3G > nul
rmdir /s/q 3.3G
  • I'd have posted the output Log files, but I believe the variance is huge due to sizes & distribution of folders files etc.

Powershell:

PS S:\T> Measure-Command { S:\T\rd.bat } > rdlog.txt

PS S:\T> Measure-Command { S:\T\r.bat } > rlog.txt

PS S:\T> Measure-Command { S:\T\rd.bat } > rd_1.7G_log.txt

PS S:\T> Measure-Command { S:\T\r.bat } > r_1.8G_log.txt

PS S:\T> Measure-Command { S:\T\r.bat } > r_2.4G_log.txt

PS S:\T> Measure-Command { S:\T\rd.bat } > rd_2.7G_log.txt

PS S:\T> Measure-Command { S:\T\r.bat } > r_3.2G_log.txt

PS S:\T> Measure-Command { S:\T\rd.bat } > rd_3.3G_log.txt
0

The answer from Hugo is the fastest but it does not remove hidden or system files in the first pass, so if you want a more complete solution use the parameter /a:

del /f /s /q /a foldername > nul
rmdir /s /q foldername

About the BAT file by Magallanes, be very carefull with takeown and icacls, cause hardlinks share the same permissions and owners, so if there are unsaturated hardlinks in the folder you want to remove, using takeown and icacls will change permissions and owners of files outside the folder.

Using the tool ln.exe you can list all the files hardlinked inside the folder and know if they are hardlinked to files outside the folder (unsaturated hardlinks):

ln.exe --enum foldername > HardlinksList.txt
-1

Deleting folder is faster than deleting multiple files.

So if you prefer to use mouse instead of command prompt, you can create directory, move files there and delete it (with Shift+Del as you said).

  • 2
    The directory should be created on the same partition where the files reside. This will make the move action instantaneous. If the new folder and the files are on separate partitions, then the whole copy operation will last a lot. – lmsasu Aug 7 '09 at 7:15
  • 5
    Where do you get the idea that deleting a folder is faster than deleting the files within? Deleting a folder contains as its first step to delete all files in that folder, so it can't be faster. Moving to the recycle bin is an entirely different matter, though, but that's not the question here. – Joey Aug 7 '09 at 22:40