The temp folder is for temporary things. If the files are in use your delete of them will fail. If they are not in use, then delete them.
I've been deleting the temp folder contents since the days of Windows 95 with no ill-effects, in fact back then I used to be a domain admin and I had it set up so that the domain logon script would erase the temp folder every day when a user logged in.
The reason for the accumulation of junk is that there are many, many badly written / badly behaved programs that don't do a good enough job of cleaning up their crap.
Delete away. Remember to use SHIFT-DELETE key when deleting so they are really deleted instead of going to the recycle bin.
(It used to be the case the MS Word in particular was a bit sensitive to the number of temp files kicking around and would run slower and with less stability. That's probably no longer the case with more modern versions, but was one of the reasons I used to religiously clean out the temp folder.)