Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

Does Linux have a mechanism to "scrub" memory? e.g. testing the memory and marking areas as dirty if they fail so that the system can continue to operate "safely" even with bad ram chips installed?!

share|improve this question

3 Answers 3

up vote 1 down vote accepted

The answer is yes, and it is done transparently (provided your kernel version is at least 2.6.30).

Basically, your memory is scrubbed* periodically and checked at every read from the processor, to check for consistency with the Error Correcting Codes (ECC). If an error happened, you get a Machine Check Exception, which is then logged and grabbed by mcelog (http://www.mcelog.org/).

If your error was correctable, it increments a "leaky bucket" counter, which causes a physical DIMM that fails too often to be transparently replaced by another one. Thus your memory page is copied to a new location, your virtual memory address is updated to point to the new page, and the old page is marked by the OS as not to be used anymore.

This is called "soft-offlining" on Linux (and memory page retirement on Solaris, I don't know about other OSs).

If your error was not correctable however, what is called "hard-offlining" happens, that is your memory page gets removed from the normal operating system memory management, and your application gets killed (NB : by some catchable SIGBUS signal that tells you where the error happened, but it's rare enough not to care and try to catch it). If your memory page is mapped from a file and clean, the OS can also reload it transparently at another physical location instead of killing the process.

You can read more on mcelog, there are plenty configuration options, you can get other behaviours to be triggered, options, and other leads on what to read and how to make sure mcelog is running on your system.


* Scrubbing, or "Patrol Scrubbing" consists in reading memory, checking it against ECC for errors, and overwriting with the corrected memory words when an error is discovered. The term patrol scrubbing is used by opposition to overwriting incorrect data on errors in memory reads, which is sometimes called "Demand Scrubbing".

share|improve this answer
    
This only applies if you have the more expensive ECC memory. –  psusi 9 hours ago
    
This applies to all memories with ECC. Be it parity (but then you can't correct), SECDED, the more expensive Chipkill or any newer ones. DDR1 could already implement ECC, but it would all depend on which actual model you use. The "home" market has traditionally less need for resilience, but supercomputers have been equipped with them for over 20 years -- servers are in between. –  Cimbali 8 hours ago
    
I meant ECC memory is more expensive ( than non ECC ) and so most people don't have it. –  psusi 8 hours ago
    
Well "most people" is pretty vague. Whether it is common to pay the price in investment and power depends on the market, as I said. My average Dell laptop, that's 2 years old now, is equipped with it (standard, no special options asked). It's getting more and more common, because miniaturization of features makes DIMMs more sensible to various radiations. –  Cimbali 8 hours ago

This is actually a bad idea. Memory cannot be reliably tested in a quick sweep. This is why software like memtest86 uses multiple passes with different bit patters to test memory. Solution:

  1. Test memory with memtest86, preferably long test, leave it running overnight, it will take a long time.

  2. If bad memory is detected, use memmap kernel parameter to force kernel not to use that memory:

   memmap=nn[KMG]$ss[KMG]
            [KNL,ACPI] Mark specific memory as reserved.
            Region of memory to be used, from ss to ss+nn.
            Example: Exclude memory from 0x18690000-0x1869ffff
                     memmap=64K$0x18690000
                     or
                     memmap=0x10000$0x18690000

In addition, you can use ECC memory which will correct 1-bit errors and detect 2-bit errors in your memory automatically (and you'll get log messages from kernel about uncorrectable memory problems if they happen)

share|improve this answer
    
Thanks for the tip on those kernel parameters. Do you think you please can clarify why this is a such a bad idea and why you can't check a chunk of memory using the same methods as memtest86(+)? I am aware that more reliable testing requires more CPU time (and probably bigger chunks of ram in one go as well) but why would this have to be a show stopper? CPU time may not a problem if spread over a long enough period and besides multi cpu is becomming more and more mainstream. –  Waxhead Dec 28 '11 at 21:32
    
Well technically, if done over long enough period of time, this may be possible. But the bottleneck here is not CPU(s), but memory bus, and of course you "poison" your CPU's memory cache. I'd not aware of such kernel module, and the idea looks very fragile to me (orchestrating repeated pattern writing to an arbitrary region of memory on a live system, etc.) –  haimg Dec 28 '11 at 21:50
    
haimg : question : will the VFS manage paging for this reserved memory ? i think it can not as it won't be visible to it. –  Jay D Sep 19 '12 at 22:30
    
@JayD: That reserved physical memory off-limits to kernel and not touched by the kernel or anything else. –  haimg Sep 20 '12 at 3:35
    
@Waxhead Memory scrubbing is usually done at the BIOS level using hardware. If enabled you should find options for patrol scrubbing and demand scrubbing. If memory integrity is important to you, which it surely is if you're using ECC memory, then the smallish performance hit gained by enabling these options is worthwhile. –  Ian Worthington Sep 23 '13 at 16:49

The post and answer misunderstand the issue. Memory scrubbing is intended to keep correctible single bit errors from turning into uncorrectible double errors. The scrubber merely all physical memory (forcing cache misses to do so) occasionally. If there are any single bit errors, they will be corrected (and the correction must rewrite the correct value using a compare-and-swap), thus clearing the error.

Otherwise, if a second error occurs in a word which already has one error, the entire word will be uncorrectible and the OS will have to do something drastic.

Scrubbing is important because without it, memory which is read but not written (like code pages) may accumulate errors over time.

share|improve this answer
    
Why do you think the answer miss understood the issue when it has been marked as the answer? –  Dave Jun 13 '13 at 14:35
1  
Notwithstanding Dave's reply, Larry is quite correct, the answer /does/ misunderstand the question. The question asks if linux can do a memory scrub, used, as Larry carefully explains, to prevent single bit errors detected and corrected by ECC h/w from turning into uncorrectable 2 bit errors. The answer talks about how to detect those errors in the first place using a software application. –  Ian Worthington Sep 23 '13 at 16:49
    
I think you misunderstand the purpose here. You are of course correct in your description about scrubbing however if you for example run a (non-critical) file server on non-ecc ram and have CPU cycles to spare it sounds like a good idea to sooner or later be able to detect corrupt memory and flag it as bad and know about it than to be unaware of a bad memory chip. Perhaps a better wording would be memory validation / verification. Not technically scrubbing perhaps, but still a viable way of reducing the damage done by potentially bad memory. –  Waxhead Feb 19 at 22:31

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.