list folders by the number of files recursively

Question

Is there any Linux application for finding the folders with the most number of files?

baobab sorts folders by their total size, I'm looking for a tool that lists folders by the total number of files in it.

The reason I'm looking is because copying tens of thousands of small files is excruciatingly slow (much slower than copying a few large files of the same size), so I want to archive or delete those folders with high file counts that that will be slowing down the copying (it won't speed things up now, but it would be faster when I need to move/copy it again in the future).

Answer 1

Try out JDiskReport it may be workable for you. FileLight is another one if you run KDE.

Screenshot of JDiskReport

& 

Answer 2

From Shell: list directories ordered by file count (see article for explanations) :

A one-liner (for the home-directory) :

find ~ -type d -exec sh -c "fc=\$(find '{}' -type f | wc -l); echo -e \"\$fc\t{}\"" \; | sort -nr

A script :

countFiles () {
    # call the recursive function, throw away stdout and send stderr to stdout
    # then sort numerically
    countFiles_rec "$1" 2>&1 >/dev/null | sort -nr
}

countFiles_rec () {
    local -i nfiles 
    dir="$1"

    # count the number of files in this directory only
    nfiles=$(find "$dir" -mindepth 1 -maxdepth 1 -type f -print | wc -l)

    # loop over the subdirectories of this directory
    while IFS= read -r subdir; do

        # invoke the recursive function for each one 
        # save the output in the positional parameters
        set -- $(countFiles_rec "$subdir")

        # accumulate the number of files found under the subdirectory
        (( nfiles += $1 ))

    done < <(find "$dir" -mindepth 1 -maxdepth 1 -type d -print)

    # print the number of files here, to both stdout and stderr
    printf "%d %s\n" $nfiles "$dir" | tee /dev/stderr
}


countFiles Home

Answer 3

I was sure there is a way to do this with a script, so I went and figured it out.

If you make a bash script like this (say we called it 'countfiles'):

#!/bin/bash
find . -type d | while read DIR; do
ls -A $DIR | echo $DIR $(wc -w);done

then run it and pipe the output like this:

./countfiles | sort -n -k 2,2 > output

Then your output file will have all the subdirectories listed with the number of files just after it (highest number of files at the end).

eg. running this script as above on my /usr folder shows this when I do 'tail output'

./lib/gconv 249
./share/doc 273
./share/i18n/locales 289
./share/mime/application 325
./share/man/man8 328
./share/perl/5.10.1/unicore/lib/gc_sc 393
./lib/python2.6 424
./share/vim/vim72/syntax 529
./bin 533
./share/man/man1 711

There is probably a better way to do it; I am not very good at bash scripting :(

Answer 4

Try this:

find . -type d | while read dir; do; echo "$dir" : $(find "$dir" -type f | wc -l); done | sort -k2 -t ':' -n 

Here's what it does:

1. Get a list of all directories (and any sub-directories) within the current directory.
2. For each directory found in (1), find how many files are within that directory (and again, any sub-directories), by listing the files then doing a count of the number of lines outputted.
3. For each directory found in (1), print its path (relative to where you are now), along with the number of files found within it, determined in (2). Put a colon between them.
4. Sort the list of directories, sorting the second field (fields split by a colon) numerically. Thus, present the directories with the most files last in the output.

The current directory, represented by ., will appear last, as it's the root node in the tree.

The algorithm is poor, but it does the job I think, and it runs very quickly anyway, so I think it's acceptable as a quick hack for real world usage.

Answer 5

Try these two alternatives -

1) For a detailed output of the tree -

 for i in $(ls -d */); do tree  $i ; done > results.txt

Output -

c++/
|-- 4.4
|   |-- algorithm
|   |-- array
|   |-- backward
|   |   |-- auto_ptr.h
|   |   |-- backward_warning.h
|   |   |-- binders.h
|   |   |-- hash_fun.h
|   |   |-- hash_map
|   |   |-- hash_set
|   |   |-- hashtable.h
|   |   `-- strstream
|   |-- bits
|   |   |-- algorithmfwd.h
...
38 directories, 662 files

2) For a summary of the tree use -

for i in $(ls -d */); do tree $i | grep -v \\-\\-\  ; done

Output -

arpa/

0 directories, 6 files

asm/

0 directories, 56 files

asm-generic/

0 directories, 34 files

bits/

0 directories, 103 files

c++/

38 directories, 662 files