2017 Update
Newer versions of Adobe Acrobat Reader DC have changed both the wording and the function within Preferences. Instead of unchecking the option to open the Tools Pane by default, the application will now remember whether it was shown or hidden previously.
Before you change your preferences, open Acrobat Reader and from the Menu Bar and navigate to View > Show/Hide > Tools Pane to hide it accordingly.
More recent iterations of of Acrobat Reader DC will still display a set of icons near the right side of your PDF document after you hide the Tools Pane. However, you can easily hide them by clicking on the icon at the bottom-right that allows you to Hide Task Pane.
Once that is done, use the Menu Bar (or press Ctrl+K) to select Edit > Preferences and in the subsequent window highlight Documents on the upper-left. Check the box to Remember current state of Tools Pane and then select the OK button.
2024 Update
With the latest version of Acrobat Reader, the GUI has been changed again. If you prefer to utilize the previous interface, you can do the following:
In the upper-left corner select Menu to expand the options and then choose Disable new Acrobat Reader.
After an application restart, Acrobat Reader will be returned to the legacy GUI.
If you eventually decide to use new interface, you can re-enable it from the Menu Bar by navigating to View > Enable new Acrobat Reader.
The new Acrobat Reader GUI features an All tools menu that can be hidden in a similar manner to its predecessor. Simply click on the "X" in the upper-right corner of the All tools pane to collapse it.
After that, click on Menu (or press Ctrl+K) to select Preferences and in the subsequent window highlight Documents on the upper-left. Check the box to Remember last state of the All tools pane when opening documents and then select the OK button.