From the man(1) man page:
SEARCH PATH FOR MANUAL PAGES
man uses a sophisticated method of finding manual page files, based on
the invocation options and environment variables, the /etc/man.config
configuration file, and some built in conventions and heuristics.
First of all, when the name argument to man contains a slash (/), man
assumes it is a file specification itself, and there is no searching
involved.
But in the normal case where name doesn’t contain a slash, man searches
a variety of directories for a file that could be a manual page for the
topic named.
There's a lot more text in there though. I'll let you read it at your leisure.