A web server such as Apache is a piece of software for transferring files (such as web pages) between two computers using a web browser.
Every server on the internet has a public IP address such as 22.214.171.124. This number is how computers locate each other on a network (including the internet) to communicate. You can think of this as something similar to a telephone number.
A DNS server is a server which uses the Domain Name System to provide a way of connecting a domain name such as coolsite.com to a specific IP address. Think of this like calling information. You don't have to know or remember the phone number (the IP address) - you simply call an operator (contact the correct DNS server), give a name (domain) and you are connected to whomever you wish to speak to (the web server).
DNS servers hold DNS records which actually map a domain name to an IP. In BIND specifically, a DNS record entry may look like e.g.:
coolsite.com IN A 126.96.36.199
If I have BIND setup as DNS server, what can I do with that? What does it do if I'm setting it up on my Windows PC?
It allows that computer to be the "operator" in the example above (acting as an Authoritative Nameserver). The advantage mainly is in the fact you can then direct "calls" (requests for a website) however you wish, without relying on a third party (which can have a variety of benefits, including saving money). Often this is used to connect outside visitors to specific computer, but you can also do special things on your local network as well, like redirecting requests for public sites (e.g. Google) or creating custom network-only domain extensions (e.g. mysite.lan).
Godaddy is a famous domain name registrar. What does that do and how does that work?
For every Top Level Domain (TLD) such as .com, .net, .org, etc. there is an organization (or organizations) of some sort that manage the DNS server information tied to a domain with a corresponding extension. So, for instance, a company called Verisign handles DNS server information associated with .com domains. There are also (more technically) groups of special DNS servers called the root nameservers run by different organizations where this information is actually stored.
These primary organizations often do not directly deal with the public (or may need assistance), so there are frequently registrars to handle this portion of it. In the case of GoDaddy, they have a contract with Verisign to help people register domain names (e.g. coolsite.com), ask for DNS server information (such as your Windows PC running BIND) and then pass that information along to Verisign. Verisign then helps ensure that if anyone asks for coolsite.com, they get put in contact with the correct DNS server (which returns the IP address of the web server associated with that domain).
[W]hat is a DNS resolver?
DNS resolvers are DNS servers that help find the IP of a website if that IP address is otherwise unknown to your computer (they help "resolve" e.g. coolsite.com to 188.8.131.52 by making a series of requests to other DNS servers, often including to the root nameservers, to find out that IP).