up vote 2 down vote favorite
1
share [g+] share [fb]

I've been thinking about using temporary GIT repositories on RAMDisks (using DataRAM's ramdisk) as a kind of object store. I was interested to see whether it was worthwhile, so I ran a simple benchmark of writing 1000 x 30K files to the drive, and then compared the average write times. I was surprised when the performance came out consistently in favour of the HDD.

For the HDD I got:

average: 0.47ms/write

For the RAM disk I got:

average: 0.76ms/write

Can anyone explain why this is the opposite of what's expected?

link|improve this question
NB don't confuse this with solid state hard drives (RamDrives)! I'm looking at in-memory drives... – Andrew Matthews Jun 23 '10 at 6:02
2  
It kind of spooks me that code I committed is one power outage or kernel panic away from oblivion ;-). Given the speed of git I wonder about the use case behind this question... – Peter Tillemans Jun 23 '10 at 6:09
these experiments may lead to a distributed (P2P) caching network that is intended to allow me to avoid using an RDBMS. the data is purely transitory state data for controlling softphones. – Andrew Matthews Jun 23 '10 at 6:21
1  
strange, have you tried another RamDisk? Here's a site that is using a well established disk benchmarking tool. See raymond.cc/blog/archives/2009/12/08/…. You might want to check it out. – Evan Plaice Jun 23 '10 at 9:17
1  
There's no logical reason why RamDisk would be slower than HardDisk access unless the RamDisk driver is doing a really poor job in Win7 or MS did a really poor job of implementing virtual memory in Win7 – Evan Plaice Jun 23 '10 at 9:20
feedback

migrated from stackoverflow.com Jun 25 '10 at 1:53

This question came from our site for professional and enthusiast programmers.

4 Answers

up vote 3 down vote accepted

When using DMA, the CPU copies the data into memory and tells the harddrive to get it and write it to disk. While the harddrive does this, the CPU can do something else. With 30K files, the harddrive can keep up with reading and writing. Those files fit in the cache and do not have to be written to disk for writing to be completed. The spinning of the disk and seeking with the arm is the really limiting factor.

I don't know how the ramdisk is implemented, but it may have the CPU do the copying twice.

link|improve this answer
Why twice? Any idea how I can verify whether this is indeed what's happening? – Andrew Matthews Jun 23 '10 at 6:09
1  
@Andrew: It may copy the data into a kernel buffer for the driver's processing and then again from the kernel buffer into the driver's internal allocation for the "drive". It's hard to know for sure without knowing the internal workings of the driver though. – Dean Harding Jun 23 '10 at 6:11
1  
A ramdisk may still perform better for your project, because it has much faster random-seek times than a hard disk. – Sjoerd Jun 23 '10 at 6:38
feedback
up vote 3 down vote

Maybe the HDD operations have been cached in memory by your friendly OS and the RAMDisk was (partially) on virtual RAM / swap space?

link|improve this answer
Yeah, that thought occurred to me - so why aren't they just roughly the same instead of twice as fast? – Andrew Matthews Jun 23 '10 at 6:05
also, assuming you're right - how do I stop win 7 swapping the ramdisk pages out? – Andrew Matthews Jun 23 '10 at 6:06
feedback
up vote 2 down vote

This is a hard question to answer, but in general you should understand that artificial benchmarks are rarely reliable. You should try testing actual git repositories.

My guess is that writing files to disk is "fast" because the CPU just issues a command to the drive "take this block of memory and write it to this section of the drive" and the drive processes it asynchronously (i.e. once the command is issued, the operation is "complete"). But copying to a RAM disk, you actually wait until the data is physically copied and since you're basically doing a RAM-to-RAM copy, you've got contention.

Again, I would suggest you try actual operations on actual Git repositories, if that's what you're planning to use it for.

link|improve this answer
Hi Dean, I did! The performance with GIT# was poor, so I went back to just testing the underlying write operations, to try to get a feel for what was just GIT# inefficiency and what was 'unavoidable'. – Andrew Matthews Jun 23 '10 at 6:13
So, what is the ideal usage scenario for a RAMDisk? presumably in subsequent read operations? My expected usage is divided, some of these will get almost even reads/writes, others will get almost 100% reads. The data rate will be very high... – Andrew Matthews Jun 23 '10 at 6:16
feedback
up vote 1 down vote

i think the problem here not in kernel<>mem speed but realisation of dataram software. for unixes there is tmpfs for ages suited for such purposes (and there is no need in additional second filesystem layer), however dataram product rather old unmodified software with (possible) bugs or using old non-optimized program architecture.

Im also got performance hit when try to run game client from dataram ramdisk actually speed was worse than running from IDE drive... it even BSOD on some moments (12gb RAM total 2.5gb ramdisk). with 4gb ramdisk performace worse (???).

With same system FreeBSD without problems handling 4gb+ filled tmpfs

however also has been noted by ImDisk author on forums that windows doing something weird in memory management regarding all ramdisks so possibly there also windows subsystems drawbacks

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.