If I make a C program (a server and a client) and I run the server on one PC and the client on another host on the same network and start communicating (suppose I know the IP of both of them), how exactly will the NIC of the first computer will know the NIC of the computer with the destination IP?
Lets assume IPv4. For IPv6 the principles are similar but the details are different (neighbour discovery instead of ARP).
First your computer looks up the destiniation IP address in it's IP routing table. This will tell it the "interface" and the "next hop IP address". If the destination is in a local subnet then the next hop IP address will be the same as the destination IP address (if the destination is not in a local subnet then the next hop IP address will be the address of a gateway).
Next your computer looks up the next hop IP address in the ARP table for the interface. If it finds a match with a valid destination MAC address then the packet can be sent out immediately. Otherwise the packet will be queued until a MAC address is available.
To discover the MAC address your computer sends out an ARP request. The ARP request is sent to the broadcast MAC address. If everything is working the next hop (destination in your case) computer will reply to the ARP request, your computer will create an entry in it's arp table and the queued packets will be sent.
If there is no response to the ARP request then your system will likely retry a finite number of times before giving up on sending the packets. Depending on the OS it may or may not generate ICMP destination host unreachable packets when it gives up (linux does, BSD apparently doesn't, not sure about windows).
I know more about how routing works. Basically (? if the computer does not have the ip address in the ARP cache and no host on the network responds to the arp request package) than the host that wants to send the data to some IP will forward the packet to the default gateway (which should be a router) and that router will than check the routing table to see on what port it is connected the NETWORK in which the destination IP is part of. If the network is not in that table then router forwards to its own default route and so on until some router knows the network. Is this correct?
This is wrong, the default gateway is used when there is no other entry in the IP routing table that matches the destination IP. It is not used if ARP fails. Using the default gateway if ARP failed would be very likely to create routing loops.