I would like to know which one of these formats requires less memory for the same quality of the picture, and what are the main differences between these formats.
BMP - Bitmap. This was probably the first type of digital image format that I can remember. Every picture on a computer seemed those days to be a BMP. In Windows XP the Paint program saves its images automatically in BMP. However, in Windows Vista and later images are now saved to JPEG. BMP is the basis platform for many other file types.
JPG / JPEG - (Joint Photographic Experts Group) Jpeg format is used for color photographs, or any pictures with many blends or gradients. It is not good with sharp edges and tends to blur them a bit unless stored at high quality. This format became popular with the invention of the digital camera. Most, if not all, digital cameras download photos to your computer as a Jpeg file. Obviously the digital camera manufacturers see the value in high quality images that ultimately take up less space.
GIF - (Graphics Interchange Format) Gif format is best used for text, line drawings, screenshots, cartoons, and animations. Gif is limited to a total number of 256 colors or less, so Gif images are relatively small. It is commonly used for fast loading web pages. It also makes a great banner or logo for your web-page. Animated pictures can also be saved in GIF format as a sequence of static images. For example, a flashing banner would be saved as a Gif file.
PNG - (Portable Networks Graphic) Apparently this is one of the best image formats, however it was not always compatible with all web browsers or image software. Nowadays it is the best image format to use for website. I use .png for logos and screenshots.
TIFF - (Tagged Image File Format) This file format has not been updated since 1992 and is now owned by Adobe. It can store an image and data (tag) in the one file. TIFF can be compressed, but it is rather its ability to store image data in a lossless format that makes a TIFF file a useful image archive, because unlike standard JPEG files, a TIFF file using lossless compression (or none) may be edited and re-saved without losing image quality. This file is commonly used for scanning, faxing, word processing, and so on. It is no longer a common file format to use with your digital photos, as jpeg is great quality and takes up less space.
The existing answers include very little technical data, so I'll include that here.
Most computer monitors run at 24-bit color depth. The human eye can distinguish about that many colors. Additional color depth is mostly to be able to retain information from a sensor so that manipulation of a photograph has more data to work with. Trying to represent a photograph in 8-bit color is going to result in graininess.
This basically refers to how large the final file will be. More compression equals a smaller file. However, JPEG attains small file sizes by throwing data away. This is referred to as "lossy" compression, because you can never get the original uncompressed data back. Its compression is also optimized for photographs where high-contrast edges are uncommon. As stated in other answers, it's a bad choice for anything other than photographs.
Alpha refers to transparency, but it implies that there's more than one level of transparency. GIF has the ability to define transparent pixels, but it's either opaque or 100% transparent, and "transparent" gets used as one of the 256 colors. PNG and BMP have the ability to mark each pixel as opaque, transparent, or partially transparent, like a piece of colored glass. Most commonly, there are 256 levels of transparency, though PNG can actually have up to 65,536 levels. JPEG has no support for transparency.
Effectively, of these formats, only GIF has any support for animation. There are specifications for animation with PNG (MNG, APNG) and JPEG (MJPEG), but they are not widely supported. (APNG works in recent versions of Firefox and Opera.) In practice, most animations you see on web pages are implemented in Flash.
BMP uses either raw bits with a small header, or Run-length Encoding. JPEG uses Discrete Cosine Transform. See the block at the bottoms of the Wikipedia articles for other compression/encoding algorithms.
You should be aware of a few key factors...
Lossless means that the image is made smaller, but at no detriment to the quality. Lossy means the image is made (even) smaller, but at a detriment to the quality. If you saved an image in a Lossy format over and over, the image quality would get progressively worse and worse.
With Indexed it means that the image can only store a limited number of colours (usually 256) that are chosen by the image author, with Direct it means that you can store many thousands of colours that have not been chosen by the author.
BMP - Lossless / Indexed and Direct
This is an old format. It is Lossless (no image data is lost on save) but there's also little to no compression at all, meaning saving as BMP results in VERY large file sizes. It can have palettes of both Indexed and Direct, but that's a small consolation. The file sizes are so unnecessarily large that nobody ever really uses this format.
Good for: Nothing really. There isn't anything BMP excels at, or isn't done better by other formats.
GIF - Lossless / Indexed only
GIF uses lossless compression, meaning that you can save the image over and over and never lose any data. The file sizes are much smaller than BMP, because good compression is actually used, but it can only store an Indexed palette. This means that for most use cases, there can only be a maximum of 256 different colours in the file. That sounds like quite a small amount, and it is.
GIF images can also be animated and have transparency.
Good for: Logos, line drawings, and other simple images that need to be small. Only really used for websites.
JPEG - Lossy / Direct
JPEGs images were designed to make detailed photographic images as small as possible by removing information that the human eye won't notice. As a result it's a Lossy format, and saving the same file over and over will result in more data being lost over time. It has a palette of thousands of colours and so is great for photographs, but the lossy compression means it's bad for logos and line drawings: Not only will they look fuzzy, but such images will also have a larger file-size compared to GIFs!
Good for: Photographs. Also, gradients.
PNG-8 - Lossless / Indexed
PNG is a newer format, and PNG-8 (the indexed version of PNG) is really a good replacement for GIFs. Sadly, however, it has a few drawbacks: Firstly it cannot support animation like GIF can (well it can, but only Firefox seems to support it, unlike GIF animation which is supported by every browser). Secondly it has some support issues with older browsers like IE6. Thirdly, important software like Photoshop have very poor implementation of the format. (Damn you, Adobe!) PNG-8 can only store 256 colours, like GIFs.
Good for: The main thing that PNG-8 does better than GIFs is having support for Alpha Transparency.
Important Note: Photoshop does not support Alpha Transparency for PNG-8 files. (Damn you, Photoshop!) There are ways to convert Photoshop PNG-24 to PNG-8 files while retaining their transparency, though. One method is PNGQuant, another is to save your files with Fireworks.
PNG-24 - Lossless / Direct
PNG-24 is a great format that combines Lossless encoding with Direct color (thousands of colours, just like JPEG). It's very much like BMP in that regard, except that PNG actually compresses images, so it results in much smaller files. Unfortunately PNG-24 files will still be much bigger than JPEGs, GIFs and PNG-8s, so you still need to consider if you really want to use one.
Even though PNG-24s allow thousands of colours while having compression, they are not intended to replace JPEG images. A photograph saved as a PNG-24 will likely be at least 5 times larger than a equivalent JPEG image, with very little improvement in visible quality. (Of course, this may be a desirable outcome if you're not concerned about filesize, and want to get the best quality image you can.)
Just like PNG-8, PNG-24 supports alpha-transparency, too.
I hope that helps!