Ok guys, here's what I did based on Harry's answer and comments.
- I first renamed the folder with the app closed, to play this safe and be able to recover if something breaks. Apparently nothing was broken on my actual Google Drive account after restarting the desktop app.
- Then I went ahead and moved the ".tmp.drivedownload" folder to a different (not-synced) location on my SSD and again nothing was broken.
- Last step, i deleted the ".tmp.drivedownload" folder, restarted Windows and finally, again nothing was broken, so there we go. It can just be deleted.
More so, a new clean and empty ".tmp.drivedownload" folder was re-created by the app after being removed, so I believe this was ultimately just full of junk files.
I made a test, copied a file to my GD, this file was the duplicated into the newly re-created ".tmp.drivedownload" folder. Manually deleted the file from my GD, the duplicate was automatically deleted from the folder. So i think it really was full of "oh we forgot to delete these at some point" files.