DMG stands for Apple Disk Image. These are treated like a Volume of their own, but one that's contained in that file. Volumes on a Mac are basically any physical or virtual disk that can be mounted permanently or temporarily. The hard drive icon you see on your desktop (probably Macintosh HD unless you renamed it) is a volume. You can clone one volume to another easily, whether it be physical or virtual. This is one of the features that makes a Mac so powerful.
These volumes are "mounted" or "unmounted" on the Mac OS. This is similar to Windows in that when a removable drive is plugged it, it's automatically assigned a drive letter (E:)... in other words, it's mounted. When you "safely remove" it, you're unmounting it. You must always unmount a drive on the Mac OS before removing it, whether it be physical or virtual.
A DMG is similar to any other compressed file, like ZIP in Windows, but more powerful on the Mac OS. The DMG is a self contained volume formated HFS+ that retains file system attributes that are important. These are called resource forks and are usually invisible to Mac users. If you've ever copied a file from your Mac HD onto a FAT32 thumb drive, then plugged that drive into a Windows box and seen those pesky "._name" files, those are resource forks. They're important to a Mac as they contain metadata pertaining to the file itself.
When you download a disk image containing an Application, the app should be copied to your Applications folder before running it. This is because most disk images are read-only, and running it from inside the disk image can produce undesirable results.
Some Applications come in a package which are handled by the Macintosh Installer. These usually need to write system setting files that require administrative access and will prompt you for your password. Make sure you know where a package came from before giving it your administrative password.
The application, when launched, will create a few files that it needs in your Library folder in your Users directory (your username... looks like a Home icon in Finder). Settings you change while using the application are stored into these files, so if you "uninstall" the application by moving it into the trash, then re-install it later, those settings are retained. This can be a good or bad thing, depending on what you want. If you want to completely remove an application along with all of it's settings pertaining to you and/or the system, first see if the Application came with an uninstaller. This might be in the application folder itself or in Utilities (Adobe is notorious for having it's own uninstallers). Usually, if an application was installed via Macintosh Installer, it will have it's own uninstaller or you can visit the company website for instructions on how to remove it. If it was a standalone application and you want to completely remove everything related to it, you can use a great free program such as AppCleaner to do this.
Hope that helps some. Enjoy Mac!