What worked for my Windows 8.1 PC:
First of all close Windows Explorer by:
Press Ctrl + Alt + Del for opening Task Manager, go to processes, find Windows Explorer, right click, and end task. Don't worry if start bar is gone and something else. Use Alt + Tab for traversing between files. Then in Task Manager click File and Run new task, and type
cmd. This will open you a Command Line.
Inside the Terminal:
If your version is 32 bit, and it is in my case, it's located in Program Files (x86), so you should type:
cd "C:\Program Files (x86)\Git\git-cheetah"
regsvr32 /u git_shell_ext64.dll
This was for unregistering.
You can traverse between directories using
cd directoryname and you can get back one directory by
cd ... For viewing what is in a directory you can type
dir in when you are in that directory. For deleting the file type:
del /F filename
After that again open the Task Manager, go to File, then run
This way you will get to usual display. Go to the Git folder which we wanted to get rid of and delete it simply.
I hope this works for other people who are using Windows 8.1.