1

In following-up a question I asked on StackOverflow, do any file systems exist wherein data is written "end to front" or "bottom to top", rather than top to bottom?

Specifically, I am looking for (perhaps a purpose-built) a way of storing log files in a most-recent-first form (a la how blogs and news sites are organized, with the most recent on top).

Does such a beast exist? If so, what is it, and where is it to be found?

9
  • To what end would you do this?
    – Pylsa
    Oct 25, 2010 at 17:31
  • Do you really want the whole file system to be like that, just to prepend the latest entry on top of a file? Is the whole file system just for log files then?
    – Arjan
    Oct 25, 2010 at 17:32
  • Painful result when such file system would exist: out.write("!egassem gol desrever yM")
    – Arjan
    Oct 25, 2010 at 17:35
  • As per the accepted answer on the StackOverflow question, I'd have suggested using tac stackoverflow.com/questions/3970241/… Oct 25, 2010 at 18:49
  • @Arjan - no, the individual lines would be right-left (since I'm thinking in English), but in gestalt, the lines would be written "bottom-up"
    – warren
    Oct 25, 2010 at 19:43

6 Answers 6

3

What you're asking for is not just a “reversed” filesystem. You want a record-structured, “reversed” filesystem, i.e. a record filesystem where the record added last appears first in the file. In fact the reversed aspect would probably be implemented as “you can insert a record before the first existing record”.

Filesystem interfaces found in operating system typically found on PCs (Unix, Windows, and even more exotic ones) are byte-structured only — they have no notion of record. So you're out of luck.

One possible approach would be to make each log entry a separate file in the directory. Then traverse the directory in reverse order of file creation time, or in reverse order of names if you give monotonically increasing names to the log entries. Since you're likely to have a large number of log entries, either make sure to use a filesystem that supports large directories well (e.g. on Linux reiserfs and ext3 with the dir_index feature are ok but ext2 is not), or else use subdirectories (one for the first 1000 entries, one for the next 1000 and so on).

Another approach would be to use a more sophisticated database, for example one that you can query in SQL, and just select the records in reverse order from their creation (SELECT message FROM logs ORDER BY date DESC).

2

I'm not entirely sure none exist, but I've certainly never heard of one. If they can be done, I should think there would be some major disadvantages.

Prepending to a file generally requires a full copy of the existing data. In a file system, you might be able to handle it to adding a block to the beginning of the file, but it still causes a few minor problems. Blocks with free space would have to keep the free space at the beginning, so it would very likely require extra seeking by the drive to find the proper location.

Handling free space on the drive, when working backward, would become a major pain. It would contradict most programming techniques, since you would have to find the max index and then work back from there.

I can imagine it would slow down on large files, and would definitely be a ridiculous thing to program.

Instead of finding a reverse filesystem, why can't you simply write the file as usual and parse it in reverse? Work out a basic message formatting scheme, read the file and parse messages from it, then display them last-to-first. If you only need the last messages, seek to the end of the file, then back n messages. It would have a similar outcome, but with far less work and comparable or better performance.

1

You need to separate the ideas of storage and retrieval. Even in the blogs you mention, the entries are likely stored in forward chronological order, but displayed in reverse chronological order (ignoring the fact that it's made easier by using structured storage).

One could conceivably create a simplistic structured storage system that would store entries in the familiar forward order with "records" of a free-form and variable length with byte-offset pointers stored in a resource file in a fixed-length format (64 bits would support files of over 18 million terabytes). Seeking the last record or the nth record or the last - n record in the pointer file, then the byte it points to in the main file would be trivial and quick. The trick that a special filesystem or driver would allow would be to make this atomic and make the resource file transparent.

0

Are you looking for endianness which is the byte order? Why would you want the log files to be organized on filesystem level rather than just ordering it through lets say ls?

If it is about Endianness, there are several filesystems available.

0

Two thoughts come to mind:

Some version control systems store the first version of the controlled file in full and all subsequent versions as changes, where others store the current version of the controlled file in full, and all prior versions as changes.

If you record runtime events in a database rather than a flat file, it may be opaque to you whether the database is storing events sequentially, reverse-sequentially, or haphazardly.

0

Unfortunately there isn't an easy way to do what you want. It would require a re-write of the entire file every time an entry is added. It would be SLOW and gets slower as the file grows. I think the best you can do is a sequence keyed log that's reversed on display, but kept in "normal" order on disk. Any SQL db could do this easily but it may be more overhead than you want.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .