That depends on what kind of IP address you have (assuming static IPv4 here) and with whom your domain is hosted.
You will need to set the A Resource Record of your domain to point to your IP address. The provider I use for hosting domains lets me change the DNS entries online, but you might have to open a support ticket with your provider if he needs to change it for you.
UPDATE: Your provider seems to offer a tool for editing DNS online as well, take a look at their documentation. At the end of the page you will find links to detailed documentation on the different DNS records, you need the one called "Adding or Editing A Records".
As for the mail server, I suggest you start with postfix and dovecot, which can be installed together on ubuntu with the package dovecot-postfix. If you manage to set it up to your liking, you will then have to change the MX Record of your domain to point to your IP address as well, so you can receive e-mail from the outside world.