By default, when you add any object in Blender it adds it at the location of the cursor with the object's axis aligned with the global axis. So a new camera will be pointing down like this. I don't know if there is a way around this in the Blender 2.4x series. In any version of Blender of course, once you create a camera, you can move it around and make it track an object, etc.
In Blender 2.55 (the latest beta that was released yesterday) at least, they seemed to have changed this so that when you add a camera it is aligned with the view to align the new camera's Z-axis perpendicular with your current view. So it looks at what you are looking at.
I would actually recommend downloading the latest 2.55 beta from the blender website, its pretty functional, stable and complete. Plus the interface is much nicer, especially if you're used to other 3d software, you'll probably get up to speed faster with it. I use it regularly now without any noticeable issues.