I recommend you look intop Manuel's answer, Synctoy, since it is very simple to use and straightforward. Personally, I use the free and open-source FreeFileSync for it's more advanced settings, fast speed and batch file associations.
However, neither of those tool starts syncing automatically upon inserting a USB key.
1) Simple workaround technique:
My preferred way of dealing with external disk/key sync is to put a batch file on the root folder. Whenever I plug it in, the files appear and I double-click "sync.bat" or whatever. Very user-friendly compared to having to load up your syncing tool and run it - you'll never forget to start it!
Here's how to do it with SyncToy.
Run Synctoy and create a folder pair. I called mine "test"
Create a new file on your USB key and call it "sync.bat". Edit it in notepad and type the following:
"c:\Program Files\SyncToy 2.1\SyncToyCmd.exe" -R test
PAUSE
Save and try it.
With FreeFileSync:
- Use the Create batch function, create a profile and save it directly on your usb key. No need for a .bat file.
2) Automated technique:
I had looked into the problem a while ago, and managed to make it work by running a Windows Powershell script. If it's critical for you (syncing very often), I posted all my details in another answer HERE.
Since you asked for a "simple" solution, it might not be worth the trouble.