5

I want to find the folders which sizes are over 1GB and then if they are over then I want to erase them.

I found some commands like

find /some/path -type d -size +1G -exec ls {} \;

or

du -h /some/path | grep ^[0-9.]*G

or (over 600M)

du -h /some/path/ | grep ^[6-9][0-9][0-9][0-9.]*M | sort

But these two commands are not really helping to me because the find command is not finding any folders although there are folders over 1GB but the linux thinks they are some small KB. Is there any command to achieve that?

7 Answers 7

4

A common problem when dealing with file/directory names is when they contain whitespace. *nix filepaths can even contain \n newlines. To get around all whitespace issues, you need to work with a null delimiter \x00.

#!/bin/bash
#
# Parameter 1 ("$1"):  Remove sub-directories from this directory
# Parameter 2 ("$2"):  Remove sub-directories larger than this many bytes 
#
# Example, To remove sub-directories bigger than 1 GB from your HOME directory
#   
#    script "$HOME"  $((2**30))     
#        
dir="$1"; shopt -s extglob; dir="${dir%%+(/)}"  # remove trailing / from directory path
[[ -d "$dir" ]] || { echo "\$1: directory NOT found: $1"; exit 1; }

size=$2  # size in bytes
[[ -z $2 || -n ${2//[0-9]} ]] && { echo "\$2: size-threshold must be numeric: $2"; exit 2; }

du -0b "$dir" |                        # output with \x00 as end-of-path
 sort -zrn  |                          # sort dirs,largest first
  awk -vRS="\x00" -vORS="\x00" -v"size=$size" -v"dir=$dir" -v"prev=\x00" '{
     if( $1<=size ) next               # filter by size; skip small dirs
     match( $0, "\x09" )               # find du TAB-delimiter           
     path = substr( $0, RSTART+1 )     # get directory path 
     if( path ~ "^"dir"/*$" ) next     # filter base dir; do not kill it! 
     match( path, "^" prev ".+" )      # print (ie. process) parent dirs only
     if( RSTART == 0 ) { print path }
     prev = path }' |
   xargs -0 -I{} echo rm -vr {}        # remove the `echo` to run live!!!!
11
  • where should i write the path in this script?
    – xmux
    Jul 26, 2012 at 8:24
  • If you save the script as rm-dirs-gt, the you can just call it from the command-line as: rm-dirs-gt "$HOME" (or whatever path you choose) ... The size threshold , which must be specified in bytes, is currently hard-coded as $((2**30)), which is 1GB ... It would be quite simple to have it as a second parameter ... I'll add the size-threshold parameter to my answer.
    – Peter.O
    Jul 26, 2012 at 9:24
  • i got du illegal option -- 0 and awk: invalid -v option error from this script
    – xmux
    Jul 26, 2012 at 10:02
  • Which version of du and awk are you using? Mine is du (GNU coreutils) 7.4, and GNU Awk 3.1.6.. With awk the variables can also be passed as args, but the -v method it the preferred way.
    – Peter.O
    Jul 26, 2012 at 10:10
  • sorry, my mistake, the code works! can you exclude the main folder from removing?
    – xmux
    Jul 26, 2012 at 10:20
1

What you're asking for is a terrible idea. This is mostly because of how what you asked for works: If a folder foo contains more than 1GB, every parent folder of foo also contains more than that (because it contains the folder foo).

Thus, if you scan /home/myuser/myfolder/ for things larger, and /home/myuser/myfolder/bar/quz/baz/foo is, /home/myuser/myfolder/bar/quz/baz, /home/myuser/myfolder/bar/quz/ /home/myuser/myfolder/bar/, and /home/myuser/myfolder/ will all be marked for deletion.

You can get around this with the -S option to du.

This gives a result (THAT I DO NOT RECOMMEND RUNNING)

du -Sb $DIR | grep '^[0-9]\{10\}' | cut -f 2- | xargs -d "\n" rm -rf

This will fail on directories whose names contain newline characters. Fixing it to not have that flaw is left as an exercise to the reader.

If you want another size make up a regex to match it. du -b returns sizes in bytes, so work from there. HINT: 365MB or more would be '^\([0-9]\{10\}\|[4-9][0-9]\{8\}\|3[7-9][0-9]\{7\}\|36[6-9][0-9]\{6\}\)'.

5
  • yes i know i want to delete all the folders because if they are over 1GB i dont need them any more..
    – xmux
    Jul 26, 2012 at 9:07
  • do u know how can i exlude the main folder from remove?
    – xmux
    Jul 26, 2012 at 9:29
  • i think i found out, but the problem is the /path/ is getting removed also du -h /some/path/ | grep ^[0-9.]*G | cut -f 2- | xargs echo rm -rf
    – xmux
    Jul 26, 2012 at 9:36
  • is there any way remove the main folder from selection? for example /home/myuser/myfolder/ will be not removed with that command
    – xmux
    Jul 27, 2012 at 7:52
  • "You can get around this with the -S option to du."-- though that will still get flagged if that directory itself has too many large files. Solution I guess would be du -Sb $DIR/*, so you're not ever running it on the $DIR in question.
    – zebediah49
    Jul 27, 2012 at 20:05
1

To find folders larger than 10G: du -h /mnt/backup/ |awk '$1 ~ /[0-9]*G/ {print}' |sort -nr|sed 's/G//g' |awk '{ if ( $1 > 10.0 ) print }'

You can change the 10.0 to any number and /mnt/backup to any path, it will print out folders that match with their size in GBs.

1

Caution deletes all the Files & Directories above 1GB in the given path

du -sh -t1000000000 /some/path/* | awk -F" " '{print $2}' | xargs rm -rf

0

use du -h to get a list of the files and directories with their sizes. Then you can use sed to extract the folder names you want to erase and pass them to rm. This can be implemented as a bash script or a command on any shell you like. If you need more help in writing the command, let me know. Though reading the man pages of the above commands should help you accomplish your task.

2
  • can u give me an example script for this situation? thanks
    – xmux
    Jul 25, 2012 at 10:52
  • See if someone can help you with the sed script. I have just started learning sed. Will try and cook up a script if I can.
    – darnir
    Jul 25, 2012 at 10:58
0

you can find the folders using du -sm * | awk '$1 > 1000'

1
  • 1
    This appears to be very similar to other answers. Can you go into a little more detail on what is different with your answer?
    – Burgi
    Aug 24, 2017 at 7:55
0

This command finds all folders bigger than 1GB:

du -h --max-depth=1 |grep G |sed s/G//g |awk '$1>1'

Since we work actually in TB: The following command finds all folders bigger than 150 GB:

du -h --max-depth=1 |grep G |sed s/G//g |awk '$1>150' && du -h --max-depth=1 |grep T |sed s/T//g |awk '$1>1'| grep -v "."

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .