gpg will stay for embedded and server usage, as it brings less dependencies and smaller binaries.
This is the standalone version of gpg. For desktop use you should consider using gpg2.
gpg2 is a redesigned version of
gpg -- but changes are mostly on internal level. The newer version is split into multiple modules, for example there are also modules for X.509 (used by S/MIME).
In contrast to the standalone version gpg, which is more suited for
server and embedded platforms, this version is commonly installed under
the name gpg2 and more targeted to the desktop as it requires several
other modules to be installed.
An important change came with GnuPG 2.1, which combines the formerly separated public and private keyrings (
secring.gpg) into the public keyring. This has been implemented in a manner keeping things compatible, so you can still use GnuPG 1 when GnuPG 2.1 integrated the private keyring, but changes to the private keys will not show up for the respective other implementation. From the changelog:
[...] allows co-existence of older GnuPG versions with GnuPG 2.1. However, any change to the private keys using the new gpg will not show up when using pre-2.1 versions of GnuPG and vice versa.
To directly answer your question:
Should I install both, or just one is enough?
Does it matter which one I install first matter at all?
Just install both. They don't interfere anyway. Install
gpg (if not installed anyway) for other applications that access it (like package managers, mail clients, ...), and
gpg2 for "direct use" on the command line.
If I installed both, are both compatible with each other?
Both implement the OpenPGP protocol, so they're compatible to each other regarding data shared among them. Additionally, they're (mostly) using the same commands and options, so most of the time you could switch between them arbitrarily.
GnuPG 2.1 makes changes to the private keyring invisible to pre-GnuPG 2.1 implementations (see above in the GnuPG 2.1 section).