Since kicken's answer to this question only seems to work on Windows 10, making me unable to test it, and since I got tired of trying to make MPC-HC work for this purpose (it does, but it's glitchy, its playlist view is plain ugly, and development on it has also been stalled for a while), I decided to spend the last 24 or so hours looking for and experimenting with an alternative music player.
I quickly realised I was no longer looking for just a music player - whereas previously I'd used my music player almost exclusively to listen to music, it was now overdue for me to move with the times and start using something that had more powerful music organisation abilities.
A much better alternative
I began by looking into foobar2000 and Clementine, based on recommendations from answers to this question and elsewhere, and although I found both okay, both were still lacking in comparison to the solution I tried last.
MusicBee is something I've been looking for for a long time without realising it - a modern music player and manager with an interface and user experience as beautiful as it is powerful. Ironically, I'd already had it installed because I'd checked it when last running Ninite, but I'd never started it up or thought to use it as my regular music player.
The one thing it is missing is a context menu item to add files in a folder to the Library, which I found odd, but clicking the Menu, Add Files to Library
, and then navigating to the Playlists tab gives me the view of my files that I need:

From here, it allows me to play files and delete them as they're playing - both to the Recycle Bin and permanently (with built-in hotkeys for both).
Another thing that sets MusicBee apart from the competitors I tried is just how customisable it is - the Options panel is chock-full of settings to be enabled/ disabled and panels to be added/removed, from changing the program's icon (with four included options):


...to changing virtually every component of the layout:

It also comes with visualisations, as well as built-in transcoders and skins (many of them dark, for those who are into that sort of thing), and many other powerful features.
Then there's the Theater mode, which is a big sell for me since I hadn't until today found a full-screen music player view that rivals that of (Windows 7's version of) Windows Media Player:

Finally, MusicBee is complete freeware. That's worth mentioning because at this point this answer is starting to sound like a product plug, which I apologise for, but it's very rare that I come across a product that I know will massively transform the way I do things and make me ten times more productive as a result.
PS. Please don't judge me for my music tastes, this is just one of many collections of music that I've inherited and am trying to separate the wheat from
wmplayer file.mp3
) then you can Up, Home, Ctrl+Del,del
, Enter to delete them.