Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I want to find the folders which sizes are over 1GB and then if they are over then I want to erase them.

I found some commands like

find /some/path -type d -size +1G -exec ls {} \;

or

du -h /some/path | grep ^[0-9.]*G

or (over 600M)

du -h /some/path/ | grep ^[6-9][0-9][0-9][0-9.]*M | sort

But these two commands are not really helping to me because the find command is not finding any folders although there are folders over 1GB but the linux thinks they are some small KB. Is there any command to achieve that?

share|improve this question

5 Answers 5

up vote 2 down vote accepted

A common problem when dealing with file/directory names is when they contain whitespace. *nix filepaths can even contain \n newlines. To get around all whitespace issues, you need to work with a null delimiter \x00.

#!/bin/bash
#
# Parameter 1 ("$1"):  Remove sub-directories from this directory
# Parameter 2 ("$2"):  Remove sub-directories larger than this many bytes 
#
# Example, To remove sub-directories bigger than 1 GB from your HOME directory
#   
#    script "$HOME"  $((2**30))     
#        
dir="$1"; shopt -s extglob; dir="${dir%%+(/)}"  # remove trailing / from directory path
[[ -d "$dir" ]] || { echo "\$1: directory NOT found: $1"; exit 1; }

size=$2  # size in bytes
[[ -z $2 || -n ${2//[0-9]} ]] && { echo "\$2: size-threshold must be numeric: $2"; exit 2; }

du -0b "$dir" |                        # output with \x00 as end-of-path
 sort -zrn  |                          # sort dirs,largest first
  awk -vRS="\x00" -vORS="\x00" -v"size=$size" -v"dir=$dir" -v"prev=\x00" '{
     if( $1<=size ) next               # filter by size; skip small dirs
     match( $0, "\x09" )               # find du TAB-delimiter           
     path = substr( $0, RSTART+1 )     # get directory path 
     if( path ~ "^"dir"/*$" ) next     # filter base dir; do not kill it! 
     match( path, "^" prev ".+" )      # print (ie. process) parent dirs only
     if( RSTART == 0 ) { print path }
     prev = path }' |
   xargs -0 -I{} echo rm -vr {}        # remove the `echo` to run live!!!!
share|improve this answer
    
where should i write the path in this script? –  xmux Jul 26 '12 at 8:24
    
If you save the script as rm-dirs-gt, the you can just call it from the command-line as: rm-dirs-gt "$HOME" (or whatever path you choose) ... The size threshold , which must be specified in bytes, is currently hard-coded as $((2**30)), which is 1GB ... It would be quite simple to have it as a second parameter ... I'll add the size-threshold parameter to my answer. –  Peter.O Jul 26 '12 at 9:24
    
i got du illegal option -- 0 and awk: invalid -v option error from this script –  xmux Jul 26 '12 at 10:02
    
Which version of du and awk are you using? Mine is du (GNU coreutils) 7.4, and GNU Awk 3.1.6.. With awk the variables can also be passed as args, but the -v method it the preferred way. –  Peter.O Jul 26 '12 at 10:10
    
sorry, my mistake, the code works! can you exclude the main folder from removing? –  xmux Jul 26 '12 at 10:20

-size only looks at the directory inode size which is not what you want. I think a combination of the two commands is the way to go, something like this (run in bash):

find /some/path -type d -print0 \
| while read -rd $'\0' dir; do
  SIZE=$(du -k "$dir" | cut -f1)
  if [[ $SIZE > $(( 1024**2 )) ]]; then
    rm -vrf "$dir"
  fi
done

Note be careful as this has the potential to delete directory structures. I usually test commands such as this with an echo in front of the rm.

share|improve this answer
    
thanks a lot! what if the folder size over 500M, what should i change in the script? –  xmux Jul 25 '12 at 10:32
    
The part between $(( )), which is in KiB, so 1024 squared is a mega kilo byte aka giga byte. If you don't care about the MiB/MB difference just change it to 500000, otherwise use 512 * 1024. –  Thor Jul 25 '12 at 10:53
    
when i tried your script on mac, i got this error : ./check.sh: line 2: syntax error near unexpected token do' ./check.sh: line 2: while read -rd $'\0' dir; do' –  xmux Jul 25 '12 at 10:55
    
Oops, there was a pipe | missing. Also make sure no space is after the backslash. –  Thor Jul 25 '12 at 10:59
    
uppps! this script wanted to erase everything under /some/path, it did not check if the folder's sizes are bigger than 1GB –  xmux Jul 25 '12 at 11:05

use du -h to get a list of the files and directories with their sizes. Then you can use sed to extract the folder names you want to erase and pass them to rm. This can be implemented as a bash script or a command on any shell you like. If you need more help in writing the command, let me know. Though reading the man pages of the above commands should help you accomplish your task.

share|improve this answer
    
can u give me an example script for this situation? thanks –  xmux Jul 25 '12 at 10:52
    
See if someone can help you with the sed script. I have just started learning sed. Will try and cook up a script if I can. –  darnir Jul 25 '12 at 10:58

What you're asking for is a terrible idea. This is mostly because of how what you asked for works: If a folder foo contains more than 1GB, every parent folder of foo also contains more than that (because it contains the folder foo).

Thus, if you scan /home/myuser/myfolder/ for things larger, and /home/myuser/myfolder/bar/quz/baz/foo is, /home/myuser/myfolder/bar/quz/baz, /home/myuser/myfolder/bar/quz/ /home/myuser/myfolder/bar/, and /home/myuser/myfolder/ will all be marked for deletion.

You can get around this with the -S option to du.

This gives a result (THAT I DO NOT RECOMMEND RUNNING)

du -Sb $DIR | grep '^[0-9]\{10\}' | cut -f 2- | xargs -d "\n" rm -rf

This will fail on directories whose names contain newline characters. Fixing it to not have that flaw is left as an exercise to the reader.

If you want another size make up a regex to match it. du -b returns sizes in bytes, so work from there. HINT: 365MB or more would be '^\([0-9]\{10\}\|[4-9][0-9]\{8\}\|3[7-9][0-9]\{7\}\|36[6-9][0-9]\{6\}\)'.

share|improve this answer
    
yes i know i want to delete all the folders because if they are over 1GB i dont need them any more.. –  xmux Jul 26 '12 at 9:07
    
do u know how can i exlude the main folder from remove? –  xmux Jul 26 '12 at 9:29
    
i think i found out, but the problem is the /path/ is getting removed also du -h /some/path/ | grep ^[0-9.]*G | cut -f 2- | xargs echo rm -rf –  xmux Jul 26 '12 at 9:36
    
is there any way remove the main folder from selection? for example /home/myuser/myfolder/ will be not removed with that command –  xmux Jul 27 '12 at 7:52
    
"You can get around this with the -S option to du."-- though that will still get flagged if that directory itself has too many large files. Solution I guess would be du -Sb $DIR/*, so you're not ever running it on the $DIR in question. –  zebediah49 Jul 27 '12 at 20:05

To find folders larger than 10G: du -h /mnt/backup/ |awk '$1 ~ /[0-9]*G/ {print}' |sort -nr|sed 's/G//g' |awk '{ if ( $1 > 10.0 ) print }'

You can change the 10.0 to any number and /mnt/backup to any path, it will print out folders that match with their size in GBs.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.