I have an application that creates 16^5 files (1048576). I can configure it to create all them in the same folder or separate them in whatever way I want (having 1, 2, 3 or 4 subfolders on each folder). Example:
/a/* . . . /f/*
/a/a/* /a/b/* . . . /f/f/*
/a/a/a/* . . . /f/f/f/*
/a/a/a/a/* /a/a/a/b/* . . . /f/f/f/f/*
All the files have less than 4KB. I am using Ubuntu 12.10 64bits and an ext4 partition to store this. What folder structure would be the best structure for this case? Maybe other filesystem would be best suitable for this case, any ideas?
Anyway I am trying to run this algorithm, I should be able to open 9999999 files:
user@pc$ ulimit unlimited user@pc$ cat /proc/sys/fs/file-max 9999999 user@pc$ cat /etc/sysctl.conf fs.file-max = 9999999
However, when I run it saving everything in one single folder the fopen call fails around 999999 files:
user@pc$ ls database/ | wc -l 999958
Strangely this 999999 was my previous value for file-max at system files. I have of course rebooted my machine before updated the value, maybe it is too big and then it keeps the last one. What can be wrong?