1

We have a cache folder, that accidentally grow large enough that it break the server. We have 8 GB RAM on server, and when I run simple rm command to delete all files within it consume all of RAM and still do not delete it after 5 hrs.

So, we try to use find but it fails too after 12 hrs of operation. Now from last 24hrs a find with perl statement is running, and the folder is still not delete in fact not a single file is deleted.

When we ls the parent folder, it shows folder size around 1GB, I just wonder how many million files are there.

So, my question, Is there any way we can delete files without listing them, so it just delete folder or files within without making a list (aka do not call getdir() like system calls)

I am really considering formatting the server to get rid of it now.

EDIT:

I have used find with -delete and with -exec rm -f {}

EDIT2:

Based on this article we are running the perl command right now (well it is about 24+ hrs that command is still running), but nothing is been done in folder size visible to us.

7
  • Ops I was posting the same page :-) Some questions and notes: which filesistem is (ext?), with which options is mounted? Is the server up? Did you try to move the directory (rename) and recreate it (even on another filesystem)? If the system is up this should avoid some bottlenecks due to the requests to write or modify new files, and to the traffic on the disk. If the server is in use with a nice -n 18 (or whatever) before you can decrease the CPU requests. Note that In one of the comment it is stated rm -rf directory/ it also works faster for billion of files in one folder...
    – Hastur
    Jan 21, 2016 at 10:32
  • Note If you have knowledge of the file names (I mean which are the expected names, maybe a common prefix and a random part) inside you can use the perl expressions, and the other solution, on a subset of files and test who work faster on that disk. If the system is down, you have the possibility to copy all but that directory somewhere else and it is not so long... you can really think about to format again the disks. Let us know...
    – Hastur
    Jan 21, 2016 at 10:41
  • @Hastur We get the folder rename and recreate blank folder to avoid fresh write or modification. We have plesk and it host websites and database for us. Jan 21, 2016 at 10:41
  • System is up and yes formatting actually mean, dumping this server and transferring all file to other machine (which is last choice we have). We cannot take system offline. We actually run all command in that article, give most of them about 4-5 hrs but nothing touch those files. So now we stick to perl's command to wait and see what it does, 24 maybe 36 hrs passed and that command is still doing something in background. Jan 21, 2016 at 10:44
  • How many temp files have been created in the new directory in those 24+ hrs? From this you can have an idea about the access request to the old directory, without asking to the system. If you move (in the same filesystem) or delete a files with handles still open the file can still be used. Doing the cache dir on a new FS/disk will decrease the requests sent to the old one. For a future planning of a nested structure of this cache directory you may find interesting to read this answer from Gilles.
    – Hastur
    Jan 21, 2016 at 11:11

2 Answers 2

3

One option would be to use the -delete switch of find. I know you said you've already used find, but haven't said exactly how you've used it. Normally this will try and enumerate every file, but I believe using the following command it should delete things as it finds them:

find . -type f -print -delete (You can remove -print if you don't want an output, and of course cd to your folder or enter the correct folder name)

Worth a try at least.

Edit-- here's a suggestion from another question over @ unix

mkdir empty_dir
rsync -a --delete empty_dir/    yourdirectory/
2
  • well I did try that based on other stackoverflow answer, but again to print it has to list through directory files and thus not worth. Jan 21, 2016 at 7:49
  • @SumitGupta Added another suggestion - you can also try the find command without -print, it's optional if you don't mind not seeing the progress.
    – Jonno
    Jan 21, 2016 at 7:55
1

After trying various suggestions the one that worked for me was to backup the files I needed, format the mount and restore them back. Of course this depends on how large the files you want to backup are. However, one can only assume you are doing backups of your filesystems anyway - in which case you can simply format and restore.

2
  • One way that worked for us was to delete the folder structure above and then replace the structure...
    – Solar Mike
    Feb 3, 2018 at 19:06
  • That is nice suggestion if you can. But on Large Physical server, it is not always an option. You have to wait for process to complete. :). Mar 5, 2018 at 12:49

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .