Disable "fast start-up" under:
Control Panel > Power Options > "Change what the power buttons do"
If you always use the start menu, you can also hold shift while clicking the power icon and then "Shutdown".
So what is this "fast start-up" thing anyway?
To quote Microsoft:
The Fast Startup feature in Windows 10 allows your computer start up faster after a shutdown. When you shut down your computer, Fast Startup will put your computer into a hibernation state instead of a full shutdown. Fast Startup is enabled by default if your computer is capable of hibernation.
Edit with addition from Modelmat in the comments:
Fast Startup results in a "Hybrid Shutdown", where system/kernel level processes are hibernated, but user level programs are closed as normal. This creates the illusion of a full shutdown, but enables the system to be ready for use faster.