In a Linux system, I have a set of very big files, containing a long list of entries, one per line, that are read one by one to create certain indexing files for each of them, that are then used for another process later on. For example:
indexprogram file1.txt # creates file1.txt.idxA file1.txt.idxB file.txt.idxC file.txt.idxD analysisprogram file1.txt # reads file1.txt as well as the index files and produces some output
I would like to combine the files I have by creating something equivalent to a symlink of concatenated files, so that I can reuse the files and save some space, and only produce the index files from the combined input. For example:
create symlink of "cat file1.txt file2.txt file3.txt > file1.2.3.txt" indexprogram file1.2.3.txt # creates file1.2.3.txt.idxA file1.2.3.txt.idxB file1.2.3.txt.idxC file1.2.3.txt.idxD analysisprogram file1.2.3.txt # reads file1.2.3.txt as if it were a real file, reads index files, produces output
Is there a way of creating this symbolic concatenation of files? Maybe using named pipes or some similar trickery?