Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Is there any Linux application for finding the folders with the most number of files?

baobab sorts folders by their total size, I'm looking for a tool that lists folders by the total number of files in it.

The reason I'm looking is because copying tens of thousands of small files is excruciatingly slow (much slower than copying a few large files of the same size), so I want to archive or delete those folders with high file counts that that will be slowing down the copying (it won't speed things up now, but it would be faster when I need to move/copy it again in the future).

share|improve this question
    
Note to those answering this question: Lie is really looking for a file manager/browser, a GUI app, not a script as most people would think reading the question. –  CarlF Aug 19 '11 at 15:04
    
A common trick to copy lot's of small files is to not copy them as many, but as one. For example, using tar or cpio, like this: tar cf - |ssh myotherbox tar xvf - –  Konrads Aug 21 '11 at 10:56
    
@Konrads: I'm copying into/from an external harddisk to my local drive. –  Lie Ryan Aug 21 '11 at 11:32
add comment

5 Answers

up vote 1 down vote accepted
+200

Try out JDiscReport it may be workable for you. FileLight is another one if you run KDE.

share|improve this answer
    
JDiscReport does the job, the default setting lists file size, but if you do "View > Show number of files" then it lists by file count. –  Lie Ryan Aug 24 '11 at 19:07
add comment

I was sure there is a way to do this with a script, so I went and figured it out.

If you make a bash script like this (say we called it 'countfiles'):

#!/bin/bash
find . -type d | while read DIR; do
ls -A $DIR | echo $DIR $(wc -w);done

then run it and pipe the output like this:

./countfiles | sort -n -k 2,2 > output

Then your output file will have all the subdirectories listed with the number of files just after it (highest number of files at the end).

eg. running this script as above on my /usr folder shows this when I do 'tail output'

./lib/gconv 249
./share/doc 273
./share/i18n/locales 289
./share/mime/application 325
./share/man/man8 328
./share/perl/5.10.1/unicore/lib/gc_sc 393
./lib/python2.6 424
./share/vim/vim72/syntax 529
./bin 533
./share/man/man1 711

There is probably a better way to do it; I am not very good at bash scripting :(

share|improve this answer
    
I don't mind command line tool, but I would prefer if the tool would allow me to browse the directory tree, so if I start from / that has in total 10000 folder/files, then I can drill down to /usr/ 1000 folder/files, etc. –  Lie Ryan Aug 19 '11 at 9:52
1  
The substitutions should be quoted, like the following. If not, you will end up with problems for directories containing spaces etc.: ls -A "$DIR" | echo "$DIR" $(wc -w);done –  jonathan Dec 6 '13 at 12:02
add comment

From Shell: list directories ordered by file count (see article for explanations) :

A one-liner (for the home-directory) :

find ~ -type d -exec sh -c "fc=\$(find '{}' -type f | wc -l); echo -e \"\$fc\t{}\"" \; | sort -nr

A script :

countFiles () {
    # call the recursive function, throw away stdout and send stderr to stdout
    # then sort numerically
    countFiles_rec "$1" 2>&1 >/dev/null | sort -nr
}

countFiles_rec () {
    local -i nfiles 
    dir="$1"

    # count the number of files in this directory only
    nfiles=$(find "$dir" -mindepth 1 -maxdepth 1 -type f -print | wc -l)

    # loop over the subdirectories of this directory
    while IFS= read -r subdir; do

        # invoke the recursive function for each one 
        # save the output in the positional parameters
        set -- $(countFiles_rec "$subdir")

        # accumulate the number of files found under the subdirectory
        (( nfiles += $1 ))

    done < <(find "$dir" -mindepth 1 -maxdepth 1 -type d -print)

    # print the number of files here, to both stdout and stderr
    printf "%d %s\n" $nfiles "$dir" | tee /dev/stderr
}


countFiles Home
share|improve this answer
    
I don't mind command line tool, but I would prefer if the tool would allow me to browse the directory tree, so if I start from / that has in total 10000 folder/files, then I can drill down to /usr/ 1000 folder/files, etc without recounting the subdirectories. –  Lie Ryan Aug 19 '11 at 9:52
    
Then you are looking for a tool, rather than a script. –  harrymc Aug 19 '11 at 10:38
add comment

Try this:

find . -type d | while read dir; do; echo "$dir" : $(find "$dir" -type f | wc -l); done | sort -k2 -t ':' -n 

Here's what it does:

  1. Get a list of all directories (and any sub-directories) within the current directory.
  2. For each directory found in (1), find how many files are within that directory (and again, any sub-directories), by listing the files then doing a count of the number of lines outputted.
  3. For each directory found in (1), print its path (relative to where you are now), along with the number of files found within it, determined in (2). Put a colon between them.
  4. Sort the list of directories, sorting the second field (fields split by a colon) numerically. Thus, present the directories with the most files last in the output.

The current directory, represented by ., will appear last, as it's the root node in the tree.

The algorithm is poor, but it does the job I think, and it runs very quickly anyway, so I think it's acceptable as a quick hack for real world usage.

share|improve this answer
    
Can you tell us how this command works? –  Tom Wijsman Aug 19 '11 at 12:41
    
@Tom: yes, that'd be a good idea. I've edited my answer. –  Chris Poole Aug 19 '11 at 15:00
    
I get bash: syntax error near unexpected token ;'` –  Adam Parkin Dec 12 '12 at 0:27
add comment

Try these two alternatives -

1) For a detailed output of the tree -

 for i in $(ls -d */); do tree  $i ; done > results.txt

Output -

c++/
|-- 4.4
|   |-- algorithm
|   |-- array
|   |-- backward
|   |   |-- auto_ptr.h
|   |   |-- backward_warning.h
|   |   |-- binders.h
|   |   |-- hash_fun.h
|   |   |-- hash_map
|   |   |-- hash_set
|   |   |-- hashtable.h
|   |   `-- strstream
|   |-- bits
|   |   |-- algorithmfwd.h
...
38 directories, 662 files


2) For a summary of the tree use -

for i in $(ls -d */); do tree $i | grep -v \\-\\-\  ; done

Output -

arpa/

0 directories, 6 files

asm/

0 directories, 56 files

asm-generic/

0 directories, 34 files

bits/

0 directories, 103 files

c++/

38 directories, 662 files
share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.