I have created a new user as admin. Then I changed the new user's account into a limited account. It cannot install new software, however the account was able to install Mozilla Thunderbird and a few other programs.
Is there a way to prevent this?
You can set almost every kind of permission using the Group Policy and Local Security Policy.
(Yes, you can set installation too.)
... but this will not protect against applications not needing any installation, such as Thunderbird or Seamonkey which can be "installed" just by extracting a ZIP file into some random location the limited user is allowed to write to. Yes, there are clever developers not at all needing the "registry" stuff other windows programs can't live without...
For more professional protection, I think you're thus reduced to more strict tools, like M$'s AppLocker or comparable.