clip.exe should be in
C:\Windows\SysWOW64\. You can check if those folders are in your path by doing
echo $PATH. If they aren't (which would surprise me), you can add them:
$ export PATH="$PATH:/c/Windows/System32:/c/Windows\SysWOW64"
It's probably more likely
clip.exe doesn't exist on your system at all.
However, there's a much simpler way you can manipulate the Windows clipboard from Git Bash (or, for that matter, Cygwin): the clipboard device file,
To write to it:
$ echo "test" >/dev/clipboard
To read from it:
$ cat </dev/clipboard
The equivalent of your command would be:
$ cat ~/.ssh/id_rsa.pub >/dev/clipboard