This is not possible using any native Windows 7 solution.
However, there are multiple third-party utilities that can achieve this effect. They are -
For all three options to get the full path (and not just the folder name) make sure "Display the full path in the title bar (Classic theme only)" is selected under 'Tools, Folder Options, View' of any Explorer window.
The simplest of the three, its sole mission is to display the folder name / path in Explorer windows.
Simply download the executable from the link, and run it. The folder path would immediately appear. Run it again (or kill the executable from the task manager) to disable.
To get an icon as well as a a folder name / path - rename AeroBar.exe to AeroBarIco.exe before executing it.
Aims to remember Explorer windows positions and sizes (similar to Windows XP behavior). It supports saving and restoring positions explicitly as well.
As a side feature it also has "Show window titles under Aero" and "Show window title icons under Aero" options.
When enabled, Explorer windows will have titles and icons.
The provided link has both a "portable" version and an installer. Take your pick.
Classic Explorer from Classic Shell
An application designed to revive various features that were lost over the years in Vista and Windows 7.
Among other useful features (like adding an "Up" icon and disabling the breadcrumbs address bar) it has two options to "Show caption in the title bar" and "Show icon in the title bar".
Enable these two to get the desired effect.
For details on using Classic Explorer, see the provided link.