43

TL;DR:

I've noticed that if I make a registry change and then hard-shutdown my Windows 10 system, the registry change does not appear after reboot.

I've also noticed that the deletion of a hibernation file can impact the ability for a Linux recovery tool to make changes to the Windows registry in an offline state. The tool seems unable to make persistent changes after the deletion of a hibernation file. I will list specific examples below.

Example 1:

  • I add a key called "1111" into "HKLM\SOFTWARE". I then hard power down my box by holding the power button for 5 seconds.
  • Test Key
  • When I pull the registry back up, that key and its values are gone:
  • Test Key Gone
  • (These images were edited for formatting purposes)

Example 2:

  • Make a change in the registry (using regedit)
  • Hibernate the system
  • Boot into a Linux recovery tool
  • Delete the hibernation file in order to mount the disk.
  • Read the windows registry (from the recovery tool)
  • The registry change is gone.

This seems equivalent to a hard shutdown on the box.

Example 3:

I see stranger behavior when I:

  • Hibernate the box
  • Boot to a Linux recovery tool and delete the hibernation file
  • Make changes to the registry (from the recovery tool)
  • Reboot the box

Those changes are not reflected in the registry either.

So what's going on here?

  • When does Windows 10 write the registry changes to the disk?
  • Why does the deletion of a hibernation file (in Example 3) prevent registry changes made from the recovery tool from being reflected on the next boot?

Hoping to get some clarification!

  • 2
    "he registry change does not appear after reboot." - The registry change does not go into effect until the reboot, and typically, only File Explorer has to be restarted. It all depends on what behavior the registry change is intended to do if a reboot is actually required. The actual key is modified, when it's modified, which answers your question. – Ramhound Jun 14 '18 at 18:09
  • 8
    What do you mean by a hard shutdown? Holding the power button until it shuts down? That process bypasses writing out any pending disk cache writes. – DavidPostill Jun 14 '18 at 18:10
  • 2
    @Ramhound I get that it doesn't go into effect, but why hasn't it been written to the disk? I make the edit, hard power down and then change isn't there any more. Whether or not it went into effect in memory doesn't matter at that point – Shrout1 Jun 14 '18 at 18:10
  • 2
    @Shrout1 - Why are you forcing the machine to turn off instead of safely shutting the machine done? – Ramhound Jun 14 '18 at 18:14
  • 6
    @Ramhound I'm running tests to see what happens if the system doesn't shut down gracefully. Random, I know, but I need to know exactly how this is committed to memory so that we don't lose anything on our systems if it isn't handled correctly. – Shrout1 Jun 14 '18 at 18:15
54

TL;DR: Shut your system down properly.


Hibernation is nothing to do with shutting down, it is closely related to suspend-to-RAM (sleep) except with the contents of RAM pushed to disk in order for it to be read back in and operation resumed from exactly where the system left off.

If you want the changes to persist then you need to disable hibernation and Windows Fastboot (which is a subset of hibernation). Or you can actually reboot rather than hibernate and restart.

The reason changes are not persisted is because they are not written to disk yet except in the hibernation file. Which you are deleting, meaning that the filesystem may well have to repair itself and go back to a "last known good" state.

While the system is hibernated there are going to be several key filesystem structures that may not have been written out to disk and are instead in RAM. The system will, upon resuming from hibernation, expect the disk to be in a very particular state and it is possible that disk caches and important system files get saved to the hibernation file rather than to the actual disk.

If you do a proper shutdown then Windows will properly flush working memory to disk, and then unmount the disk cleanly before powering down.

To force a proper shutdown open a command prompt and type

shutdown /s /f /t 0

/s is "shutdown", /f to force, and /t 0 to mean "now" (time = 0 seconds)

Or you can just disable fastboot and hibernation.

Read more at HowtoGeek: Shutting Down Doesn’t Fully Shut Down Windows 10 (But Restarting Does)


Related to you doing a hard shutdown the problem there is that Windows is not guaranteed to write any changes out to disk the same millisecond (or even minute) that you make the change. It will almost certainly be written within a few minutes but the probablility of it having actually been written will increase over time. It's unlikely to be written immediately, then near to the time you make the change the probability increases sharply, and will almost certainly have been written within an hour.

The thing is though, that by forcing a hard shutdown you are not giving the system a chance to safely write changes to disk.

Most modern filesystems are written to make changes in the safest way possible. In the past they have been referred to as "atomic", as in the change has either happened or it has not.

Today we know them as Journalled file systems because they keep a log of operations that will happen that can be either reverted or rolled forwards in the event of system failure and reboot. Upon startup from a power failure the system checks the journal and for each transaction it checks if the actual file data is written to disk and is "good". If it is then the transation rolls forwards and is completed, if not then it rolls back to the old data.

By using this order the disk is almost always in an easy to repair state.

But by forcing your system to power down unexpectedly you cannot guarantee whether the transaction has progressed far enough to roll forwards upon repair, and chances are that an operating system such as Linux will not care as much as Windows about the transaction history and is more likely than not to just make changes that roll everything backwards rather than forwards.

If you rebooted into Windows it might try or be able to repair the disk properly as it has a more intimate knowledge of the filesystem.

  • Thank you! I appreciate your response :) I turned off hibernation, rebooted and re-ran the same test. Example 1 has the same result, no hibernation file involved. It appears that registry changes are only written at some point during logoff / shutdown... I also agree that the system might be looking for a "last good" state when waking from a broken hibernation - it tends to run integrity checks. – Shrout1 Jun 14 '18 at 18:29
  • 1
    Hmmm... So I downloaded sysinternals "sync" which forces the write cache to disk and that didn't do anything (still lost the keys / values on hard shut off). I also ran process explorer and looked at the Disk I/O in the properties for Regedit. It had read I/O but no write I/O. This makes me think that it might be waiting for shutdown... Do you know of any exceedingly dry M$ documentation that might lay this out plainly? Thanks again for all your help!! – Shrout1 Jun 14 '18 at 19:24
  • 11
    The NTFS journal makes no guarantees about file data. It's merely to keep the NTFS metadata consistent, so you don't end up with a broken filesystem. Individual files can still have partial changes recorded to them, breaking the file. There is Transactional NTFS but that's not recommended and very rarely used. This is also why database engines keep their own journal for data consistency. See also blogs.msdn.microsoft.com/oldnewthing/20130101-00/?p=5673 – Bob Jun 15 '18 at 3:40
  • 2
    @Shrout1 That's assuming the registry changes are pending in the write-back cache. It's entirely possible that they are instead managed separately and that this has nothing to do with the filesystem, caching or otherwise. For example, Last Known Good Configuration is only updated on successful startup. (I don't know enough about registry internals to be certain when changes are saved. And then TxR might be involved.) – Bob Jun 15 '18 at 3:58
  • 1
    Is there a reason you're forcing shutdown? AFAIK that only forces running applications to close, which doesn't do much other than kill applications that you can't otherwise close and makes it more likely that you'd lose some unsaved changes somewhere if you don't know what you're doing or that you'd put some application in an unrecoverable state - I wouldn't really call that a "proper" shutdown. – NotThatGuy Jun 16 '18 at 17:15
39

As documented on the MSDN page for RegFlushKey:

Calling RegFlushKey is an expensive operation that significantly affects system-wide performance as it consumes disk bandwidth and blocks modifications to all keys by all processes in the registry hive that is being flushed until the flush operation completes. RegFlushKey should only be called explicitly when an application must guarantee that registry changes are persisted to disk immediately after modification. All modifications made to keys are visible to other processes without the need to flush them to disk.

Alternatively, the registry has a 'lazy flush' mechanism that flushes registry modifications to disk at regular intervals of time. In addition to this regular flush operation, registry changes are also flushed to disk at system shutdown. Allowing the 'lazy flush' to flush registry changes is the most efficient way to manage registry writes to the registry store on disk.

This suggests that apart from flushing a specific key to disk immediately (which locks everyone else out of the registry until the flush is complete), the registry is automatically flushed periodically: a time is not given, but presumably it is at least more than the time you waited between writing the key and hard shutdown. In addition, it is flushed at shutdown, as you had already figured out.

You can use the RegFlushKey function in the software that manipulates said key, or create an additional tool with it to force writing a registry key to disk immediately, if this is crucial to your usage case.

  • Hey! I'll give you this one if you throw a link in to the following MSDN article: Saving application registry changes on Windows 8 or Windows Server 2012 and add a brief blockquote. Your answer led me down the rabbit hole to that article and it basically says the same thing you did. Thanks a whole lot! – Shrout1 Jun 15 '18 at 14:16
  • But does anybody know how a user can call RegFlushKey? – Scott Jun 15 '18 at 18:19
  • @Scott It appears that it has to be done in code... The MSDN article listed in the post has a C++ example and I've seen it implemented in C# elsewhere. Not sure it can be done from the command line. – Shrout1 Jun 15 '18 at 19:53
  • 3
    @Scott: I would be shocked if sync did flush the registry to the disk. It just wouldn't make any sense. Why should flushing the file system cache (which is used by the configuration manager, not the other way around) suddenly flush the configuration manager's own cache? It doesn't even know what the structure of the higher-level cache is, if one even exists at all. – Mehrdad Jun 16 '18 at 4:21
  • 1
    @Scott There is a way. The API has already been linked. There are tools out there that give you a GUI to call arbitrary Win32 functions. Thing is... It's an implementation detail. If you expose it, people rely on it and then you can't change it. Also worth noting that the system disk is a totally different beast to removable media. The system disk is used for many things that require a handle to be always open (eg page file). Windows doesn't support unmounting the system partition, so has no need for this "feature". Plus... try to keep the comments neutral. You hate Windows, we get it. – Basic Jun 17 '18 at 0:57
14

TL;DR: It is very careful to do this at the right moment.

The documentation on FSCTL_MARK_AS_SYSTEM_HIVE has this to say:

The FSCTL_MARK_AS_SYSTEM_HIVE control code informs the file system that the specified file contains the registry's system hive. The file system must flush system hive data to disk at just the right moment to avoid deadlocks and to ensure data integrity.

I don't believe there is any more detail available publicly than this.

Bear in mind that flushing the file system does not imply flushing the registry, because the registry can perform caching on top of the file system. To flush the registry first, you need to somehow cause NtFlushKey or ZwFlushKey to be called on your key of interest.

  • Oh good catch! That's my new favorite MSDN-ism: just the right moment. My previous favorite was "Use caution when specifying the TOP clause in a query that contains a UNION, UNION ALL, EXCEPT, or INTERSECT operator. It is possible to write a query that returns unexpected results" which was an euphemism for "this feature is badly broken and doesn't do what any reasonable person would expect and instead of fixing it we're going to document it". – davidbak Jun 17 '18 at 2:13
  • @davidbak: lol, well in MSDN's defense, I don't really see what more detail they could provide that a user should be relying on. – Mehrdad Jun 17 '18 at 2:17
  • Oh you're right; clearly this was just something documented as a fallout of the long-ago EU settlement where Microsoft was required to document all APIs - no matter how internal they are. This page you linked even says "Only kernel-level components can use this filesystem control code" which is a strong hint to keep your hands off. Still, "just the right moment" - one day I'll document some API of mine with "call this method only at just the right moment" and see what happens.... – davidbak Jun 17 '18 at 2:20
  • @davidbak: I think you're a little... excited. :-P My guess is that this was documented in order to let file system drivers know what file contains the SYSTEM registry hive, which may be important in that they wouldn't attempt to write anything to the registry at the same time that that file is being flushed to disk. I have no proof of this though; it's just one reason I find conceivable. One doubt I have about it, though, is why a disk driver wouldn't need to know this too. – Mehrdad Jun 17 '18 at 2:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.