The erroneous assumption that you're making here is that moving subfolder items in an Explorer folder is a simple operation that involves merely modifying a couple of directory entries. It isn't. The low-level system call to move a file may be. But Microsoft Windows Explorer places layers on top of that. A "move" of a folder item may end up as being a "copy to new name then delete original" at the low-level rather than "rename this directory entry to that parent directory". Of course, the former is a much more expensive operation than the latter.
Why does Explorer do this? For starters, it's because a folder is not a directory. Not all "folders" that Explorer presents to the user are honest-to-goodness bona fide directories on a disc volume. Explorer has to, for example, work out what it means to "move" a directory from/to a virtual folder denoting remote files on a smart 'phone viewed over a Bluetooth FTP connection. That's not a simple rename operation at the raw system API level. The same is true of the Recycle Bin, whose actual expression at the raw file and directory level is filesystem-format-specific.
Explorer has an "engine" that works out what the "copy", "move", "rename", and "delete" actions for shell objects actually mean at the raw system API level. In the Explorer that comes in Windows NT 6.0 and later, the "move" operation is the
IFileOperation::MoveItem() method. (It used to be the
SHFileOperation() function.) There's a whole load of non-obvious stuff that goes on, even in what one might erroneously think could map be a simple "rename" system call, to provide the end-user-visible semantics of "moving" a subfolder item in Explorer. This includes all of the processing for all of the operation flags that can be set, which can include undo information recording, collision detection and autorenaming, checking last modification datestamps, overriding the old security descriptor, and updating progress dialogues.
If you want to test how long it takes to change a few directory entries around, use an application that does only that, and doesn't do all of the Explorer layers on top of it, such as the