Do protocols are installed by operating systems or they come with the hardware?
It depends what you mean by "protocols"; TCP/IP and HTTP are very different, for example. Since you name TCP/IP, my answer focuses only on that:
They are opten part of the operating system. They may be built in to the core, or they may be added as modules after the fact; the later is typical on most modern operating systems.
They are often layered; almost all protocols at the TCP level, today, are built on top of IP - IPv4 or IPv6 - which reduces the overhead of getting started because most network routers understand IP routing.
It is also possible to implement a new protocol entirely in "user space", though that can be harder - it usually needs raw IP access, which is a "privileged" operation, so requires the equivalent of root or administrator privileges.
If a new protocol is developed, how it will be implemented on machines?
You write a driver for it. Beyond that, it depends on the operating system, and where you chose to implement it.
Note, this includes, for example, Cisco IOS or other "operating systems" built into routers - including the your cable or ADSL modem. They have drivers for IP and TCP just as much as a Mac, Linux, or Windows system does.
Is it possible to implement a protocol over the internet?
I assume you mean "can I design a protocol like TCP, and have it work over the Internet without having to do anything but install it at the two communicating machines?"
Yes, you can - if you layer your protocol over IP. Most of the routing is done at the IP layer, and the "protocol" field in the IP packet is used to route the content into the right protocol handler - which then breaks it out in an OS specific way.