Under Windows, particularly older versions, it was common for programs to store configuration files and non-constant data in their
C:\Program Files directory. This is derived from how programs were usually installed and ran under single-user, non-networked, non-file-permission DOS.
From a security standpoint, this is a bad idea. Places where executable code lives should be separated from modifiable data. That way it's easier to apply appropriate file permissions to prevent modification of installed binaries by unauthorized users. Similarly library directories which may be updated separately from main executables should also be in a separate directory.
With the advent of Vista and UAC annoyances, this tradition is finally starting to seriously lose traction.
UNIX, and Linux, being a multiuser system from much earlier on, had the tendency to separate executable directories from other directories much earlier, since there was a need to prevent users other than root from modifying installed binaries. It's also why
/usr and even
/sbin are sometimes separate partitions - a particularly security conscious admin can mount those partitions readonly and remount them read/write when an install/uninstall needs to happen.
Packages are usually installed from a package manager. There's various package managers, such as
aptitude (Debian and derived distributions),
yum (Redhat and derived distributions),
pacman (forget which distro this is...), and others.
The package manager lets you browse repositories, download, install, query, and remove software, much like a sophisticated (and free) "app store." It assumes responsibility for ensuring dependencies are taken care of and tracking what is currently installed.
Usually the package manager will also allow the same operations on a package you downloaded manually outside of any repositories. Tools are also available if you want to create your own from software you made or compiled yourself.
Since the package itself is NOT an executable file, you don't have to run an untrusted executable which you don't really know what it does. (Windows is finally coming around with updates by distributing
.msu's instead of
.exe's - but
.msi's have been around a while...)