If you're a member of Administrators, you should be able to take ownership of an old Windows directory, apply appropriate permissions, then delete it.
Right-click on the directory and choose Properties, then go to the security tab.
Click Advanced, then the Owner tab. Change the owner to yourself and flag "Replace owner on subcontainers and objects". Click OK and let this run.
When this finishes, go back into Properties → Security and add yourself as a user and allow full control. Click OK and let this run.
At this point you should be able to delete the folder, or at least most of it.
If this still doesn't work, you may need to do the deletion as SYSTEM. To do this, download psexec and run it as follows:
psexec -sid cmd
This will open a command prompt running as the SYSTEM user, which is a higher-privileged account meant for the system's use. From this command prompt:
rd /q /s C:\OldWindowsFolder
If that doesn't work, cd
into the old Windows folder and start deleting sub-folders and files until you find out which one is preventing you from continuing.