My Personal Information Management problem
I have gathered terabytes of data but it has became a habit to store files and folders to the same folder, that folder could be kind of seen as a Inbox
where most files (non-installations) enter my system. This way I end up with a big collections of files that are hard to organize properly, I mostly end up making folders that match their file type but then I still have several gigabytes of data per folder which doesn't make it efficient such that I can productively use the folder.
I'd rather do a few clicks than having to search through the files, whether that's by some software product or by looking through the folder. Often the file names themselves are not proper so it would be easier to recognize them if there were few in a folder, rather than thousands of them.
Scaling in the structure of directory trees in a computer cluster summarizes this problem as following:
The processes of storing and retrieving information are rapidly gaining importance in science as well as society as a whole [1, 2, 3, 4]. A considerable effort is being undertaken, firstly to characterize and describe how publicly available information, for example in the world wide web, is actually organized, and secondly, to design efficient methods to access this information.
[1] R. M. Shiffrin and K. B¨orner, Proc. Natl. Acad. Sci. USA 101, 5183 (2004).
[2] S. Lawrence, C.L. Giles, Nature 400, 107–109 (1999).
[3] R.F.I. Cancho and R.V. Sol, Proc. R. Soc. London, Ser. B 268, 2261 (2001).
[4] M. Sigman and G. A. Cecchi, Proc. Natl. Acad. Sci. USA 99, 1742 (2002).
It goes further on explaining how the data is usually organized by taking general looks at it, but by looking at the abstract and conclusion it doesn't come with a conclusion or approach which results in a productive organization of a directory hierarchy. So, in essence, this is a problem for which I haven't found a solution yet; and I would love to see a scientific solution to this problem.
Looking for an empirical solution to this problem
Upon searching further, I don't seem to find anything useful or free papers that approach this problem so it might be that I'm looking in the wrong place. I've also noted that there are different ways to term this problem, which leads out to different results of papers. Perhaps a paper is out there, but I'm not just using the same terms as that paper uses? They often use more scientific terms.
I've once heard a story about an advocate with a laptop which has simply outperformed an advocate with had tons of papers, which shows how proper organization leads to productivity; but that story didn't share details on how the advocate used the laptop or how he had organized his data. But in any case, it was way more useful than how most of us organize our data these days...
Advice me how I should organize my data, I'm not looking for suggestions here. I would love to see statistics or scientific measurement approaches that help me confirm that it does help me reach my goal.
Search does not solve this problem
Search is not a solution to this because it disallows access to relevant files, which is often required as you are working in the context of a file (which often requires other files). This is why I mentioned before that bad file names are a mayor problem. And no, putting all directory names in the file name is not the right approach to this. Full text data search engine searches only work for textual files, and the directory names are often not within the file. That makes Search unreliable and uncertain when searching...
Also note that I am looking for structural organization/accessing, because semantic approaches (like "this is how I divide it and works for me and my friends" or "search works for me") are not applicable to everyone, while structural approaches are applicable to the majority of users that navigate their storage.
researching the evidence behind claims you encounter
as per the FAQ. Apparently, you are not really respecting your community. I'd wish to know why this computer data question is off-topic, because otherwise its just you that wanted to see it closed for no clear reason.