Let's say I want a specific folder X in My Documents to only be accessible by a specific application.
The Windows permissions don't allow me to do this easily, is there any solution which can do this?
Although a bit extreme, the way I implement this in my home is to have virtual machines for various purposes.
This way I can have a virtual machine for various purposes, finance, programming, day to day, gaming, etc. And on the host you can secure the VMs appropriately and even give accounts on host/guest VMs in various capacities if you are looking to address a multiple user situation.
Like I said, I know extreme but for me, it does the job exactly how I want it to be since I use a handful of environments.
You can use a Host-based Intrusion Prevention System (HIPS), which wikipedia defines as :
an installed software package which monitors a single host for suspicious activity by analysing events occurring within that host.
Such a system will have to be taught : It will ask permission for each program and for each folder and will let you permit or block the access or permit unrestricted access to all folders. Many HIPS suites also come bundled with a firewall.
Some free HIPS programs are listed in the article (and its comments):
Best Free Intrusion Prevention and Detection Utility for Home Use (HIPS).
My own favorite firewall+HIPS product is Online Armor Free.
Another very good one is Comodo Internet Security (CIS).
I think this might be possible with Sandboxie, but I'm not sure.
You should make a user with restricted permissions and "run as" the program using that user. I did that with Filezilla FTP server recently.