If you type in
mount without arguments, it will tell you what directory is mounted and where. For example, if the answer is
/dev/sda1 on / type ext3 (rw), it means that the root directory of the first partition on your first hard disk is actually the root of your "big tree".
Now let's say that you also get the line
/dev/sdb1 on /home type ext3 (rw). It means that the contents of your
/home directory will be the root directory of the first partition on your second hard disk. There can be other, virtual file systems that doesn't exist in any partition, but dynamically managed by the system. For example, you boot from a live CD, and mount the partition that is used as a system partition. Let's say, you type.
mount /dev/sda1 /media/system
Now, you will see the root directory of your system (that you use regularly, not what you are booted into with the CD) in
/media/system. You will also see that directories like
dev are empty, because they usually contain a virtual filesystem.
If you are asking how does the system decide where this partition is, it is all written in
man 5 fstab if you want some more information about this. Yes, I know there is a chicken-egg problem here, because the system has to find
/sbin/mount, for that matter) before mounting the directories. The answer is, I don't know, but it works pretty well.
EDIT: I may add that all partitions have a unique identifier called UUID. You can mount using this identifier by typing
mount -U uuid_of_partition /media/mount_point. It is usually done this way in
/etc/fstab, to avoid the problem that, for example, you change the disk order in your machine.