3

I have two options:

  1. Either: To have a database with 100,000 records, Or -
  2. To have these records as separate (and very small) text files in the single directory.

What are the disadvantages of the second approach, except the copying will take a long time?

I ask about the disadvantages from the performance point of view.

  • Thanks. I saw that question and several similar ones yesterday. Those questions are about "how many files can I have in the single directory". They are about technical limits, not about the issues that can occur even when you are far from these limits. – john c. j. Mar 5 '20 at 9:06
4

It's not only copy, access the filelist in general, search file will also take significant amount of time. Moreover database engines use indexes to speedup location of the record(s). Also database engines use other techniques to speedup the access like partitioning of tables. Think about Data warehouse where you can have huge tables with billion of rows.

1

You may suffer performance issues when having so many files in a directory.

Example:

I had 70000 internet recorded mp3 files in a folder on a NTFS file system under Windows XP3.

Just opening this folder to look inside took me more than 30 seconds until the file list appeared.

This may be caused by Shell Extensions.

I recommend you to test the situation.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.