I can't undelete my post apparently... So here is a more extended answer :
So... I manged to fix my problem by doing this : https://www.tenforums.com/drivers-hardware/87515-pny-ssd-stuck-raw-file-system.html#post1062176
Because it wasn't obvious enough, the above steps does this :
Diskpart - Open the disk partition software of Microsoft
list disk - List all current disks (including my bugged USB)
select disk # - Allow me to select my bugged USB (# refer to the ID displayed with the previous command)
clean - remove everything from the USB (not really needed, since I can't write / remove files from it)
convert gpt - Convert the partition table, apparently, Windows like it that way.
create part pri - Create a primary partition (required)
format fs=ntfs quick - Quickly format the parition to NTFS format.
assign - Assign a letter, so it can be seen in Explorer.exe
exit - obvious
exit - obvious
NTFS
orFAT32
file system.diskmgmt.msc
. You can also manually format your USB drive from there and if your USB drive is not listed there at all then it is different story.