I have been searching for the answer, but couldn't find any that would explain it completely. It is more complicate to google because somehow it ignores those $$ marks even when entered with quotes.
The only thing clear is that $$PendingFiles directory is part of installation or update process and it most likely appears if one has moved the Program Files or ProgramData directory to another drive and linked it with junction.
Update: The resources I found are: http://www.runsmall.com/2011/08/index.html and http://www.sevenforums.com/crashes-debugging/168933-error-has-occurred-not-all-features-were-successfully-changed.html, but none of them explains why the directory is created and what exactly creates it.