Every time I open a folder and double-click on another folder inside, another folder opens up. If I am finding a file burried in a folder, many folders will open before I can reach the file and then I will have to close them all. Can I somehow configure my machine so that it uses the same folder like in Windows?
In GNOME 2.6, the Nautilus file manager uses "Spatial" file management mode by default. What you are looking for is the "Browser" mode.
Go to Edit → Preferences, click the Behaviour tab, and enable the Always open in browser windows option.
(As for "like in Windows" – Windows 95 used spatial mode by default too.)