What worked for me was:
Shift + right click Command Prompt and Run as Administrator.
dir says "File Not Found", but autocompletion still works, so we can find the names of the directories here by typing, for example
dir then pressing
Thanks to this answer,
rd /s (remove directory recursively) on each of the folders in the
C:\$Recycle.Bin. For me, they have names starting with "S" and followed by numbers and hyphens, like "S-1-2-3-1234567-12344-121212".
After that, the recycle bin is back to normal and the stuck folder is gone.
This may be related to a path of a file or folder in the Recycle Bin being longer than some limit. Some alternatives are suggested in this answer: (a) dealing with long file names using UNC
\\.\... path notation and (b) using RoboCopy.
What was also useful is enabling Windows Explorer to show hidden system files and folders, like
C:\$Recycle.Bin (may be called different on different systems, such as "Recycler"), by: pressing Alt to show the menu in Windows Explorer,
Tools | Folder Options,
View tab, uncheck Hide protected operating system files (Recommended).