I have a large remote directory with 600GB of content, that is split over 6 million MP3 files, without any sub directories.
Will the large number of files under a single directory, cause any performance issue with SSHFS?
Would it be better to shard the files into multiple sub directories based on the first two characters of the filename, similar to how git stores objects (Filenames are MD5 hashes). For example,
098f6bcd4621d373cade4e832627b4f6.mp3will be renamed to
09/8f6bcd4621d373cade4e832627b4f6.mp3Or would it further slow down file reads, since SSHFS needs a round-trip for each directory.
Update: The directory will be used as a cached for generated audio files. Files will be written to / read from the directory. There is a monthly cron job to delete less frequently used files. There wont be any directory scans. Files will be accessed / deleted using exact file path, which is stored in a local database.