Super User is a question and answer site for computer enthusiasts and power users. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have just finished reading the Google File System (GFS) paper. The paper says that GFS is

optimized for appending operations rather than random writes. 

Seeing that this characteristic is emphasized throughout the paper, I take it that it must be very important.

As a student who has had no working experience at all, what are some real-life examples of such Appending Operations that Google speaks of? It sounds pretty intense.

(I suppose logs are append-heavy. Maybe e-mail as well?)

share|improve this question
up vote 1 down vote accepted

Yes, logs would be a classic example of an append operation. Email is implementation dependent. For example, Maildir format email stores are one email per file, so no appending needs to take place. With mbox format, all the emails are in a single file, so new ones get added on the end - they are appended.

A database update would be an example of a random write type operation, for example if you decided to update the Firstname of everyone in a database, this would result in a series of writes to the middle of the file that contains the database.

What Google are saying here is, to keep performance optimal and if you can get away with it, then add new information to the end of existing files rather than modifying the content already in them. If you need to delete data from a file, then just flag it somehow and delete it later when performance isn't important, like in a cleanup batch job. An example of this would be with mbox format email stores again. If you delete an email, this is just tracked in a separate file. Then later you can "compact" the file, which effectively rewrites the file without the deleted emails.

share|improve this answer
A bad choice of examples. Flagging the deleted content in an mbox-family file is a "random write" operation; and writing a message to a Maildir file is a series of append operations from an operating system's perspective. – JdeBP Jan 6 '12 at 10:46
@JdeBP Maildir does not have a file, it has folders in the filesystem. Each email is a single file, so nothing is appended. – Paul Jan 6 '12 at 12:51
@JdeBP and with mbox, I was referring to an implementation of mbox where the flagged emails are specifically not written to the mbox file and are actioned later. – Paul Jan 6 '12 at 12:55
That you can utter "Maildir does not have a file" and "Each email is a single file" in the same breath without figuring things out, is somewhat depressing. Read what you are writing. And also think about what your vague "flag it somehow" handwave actually is in the real world, when programs like Thunderbird do it. I repeat: A bad choice of examples. – JdeBP Jan 6 '12 at 13:08
@JdeBP Writing a single file can hardly be referred as an "append operation" can it? What is it that is being appended to? – Paul Jan 6 '12 at 13:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .