Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I have a set of files that I wish to be allocated contiguously in the filesystem. I will be accessing all these files sequentially in a single read one after the other and I want to optimize reading them. I can't combine them into a single file , and I am looking for a solution which will allow them to be individual files.

I am using an ext4 file-system and I was wondering if there is some existing tool which might do this for me as I learnt that ext4 supports online block exchange and defragmentation. I tried using e4defrag on my directory of files , although it ensured each individual file was defragmented, each file itself was in a separate block not necessarily adjacent to the other files. (I used filefrag -v file_name to verify if they were being allocated next to each other or not)

EDIT: Just to clarify on file access patterns, These files will be written exactly once and never modified again. They will be read frequently, but in such a manner that if any one of them is read all the other files in the set will also likely be needed to be read. What I intend to do is prefetch all these files together into the filesystem buffer/cache in one go, so that subsequent random reads of any of these files will be really fast. (The total file size is small enough 100~200MB to fit in cache). What I am trying to improve right now is the read performance when I try to load these files into cache. Right now read performance takes a hit while trying to prefetch them into cache because there are multiple disk seeks as they are located in disjoint segments.

share|improve this question
    
Would you elaborate your scenario a bit more? E.g., are these files written once and then read once? ... Read/write a lot, and often? Do you wish to enhance read performance or read/write performance? How does "random" file placement prevent achieving the performance requirements? –  rickhg12hs Nov 11 '13 at 14:24
    
@rickhg12hs The files are static and won't change so write performance need not be considered. I'm only concerned about improving read performance and infact I will always attempt to prefetch the entire set of files to transparently improve performance of several other processes that will immediately use several of them. –  phininity Nov 12 '13 at 5:49
    
Easiest way I can think of off the top of my head is to just create a separate partition that is just big enough for the files. As an added bonus, if you create the partition at the beginning of the disk, it'll be even faster. –  Lawrence Nov 12 '13 at 7:51
    
Just wondering ... is this premature optimization? Could writing/duping these files to a ramdisk help? Or your prefetching will work great if it's completed before the data is needed. –  rickhg12hs Nov 12 '13 at 9:56

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.