Craig Watson has already answered about the combination of hardware you list, so I will focus on the power supply and computer case.
With a 65W maximum TDP CPU and no off-board graphics, just about any power supply that fits physically should work well, but some may work better than others. You'll probably be drawing on the order of 100-150 W in that case when everything is running at top speed (the hard drive needs about 7W working and 2A ~ 25W startup (reference); fans will generally need a couple of watts each; the motherboard itself will draw some power; ...), and if you want to conserve power, you should definitely not over-rate the PSU as even though the efficiency curve varies from one PSU to another they tend to be most efficient around 50-75% of their maximum rated load (Wikipedia isn't a source but it does provide that number). You want a margin, but not an excessively large margin. The largest energy hogs will likely be the CPU, motherboard and hard disk, followed by the fan(s). Anything else should pretty much be negligible unless you add high-powered expansion cards.
For a load like what you would be seeing, I would look at a power supply capable of delivering approximately 200 W, and certainly not considerably more than that. You might even be able to get away with a PSU rated for around 150 W, but then you'd have to look closely at the power rating of the different rails (+3.3V, +5V, +12V, -12V, etc.) to make sure you don't overload any one rail. Going much higher will likely result in a fairly serious drop in efficiency at the amount of power you will be drawing, as well as the PSU itself costing more.
More generally, you'd have to look at the power draw of the various components in the system on the various voltages needed (there's commonly +3.3V, +5V and +12V to be taken into serious consideration, and if you want to play it extra safe you should be looking at +5VSB and -12V as well), add up the various numbers (make sure to look up both continuous usage as well as startup power requirements; anything with a motor in it, such as a spinning-platter hard disk or a fan, will have some specific startup power requirement that is usually larger than what is needed for normal operations), add a bit of a margin to the power sum to compensate for anything you might be forgetting about, and start looking at power supplies which are capable of delivering around 30-50% above the calculated necessary peak power. If you are planning on installing more hardware which relies on the computer's PSU in the near future you should also add a margin for that. Look at power rail ratings (note that some rails may be combined!) and determine whether the power supply in question will be able to deliver the amount of power necessary at the needed voltages. Don't dismiss all power supplies of a specific power rating if one of them fails this test; this is one area where manufacturers can actually differentiate themselves from others, so even if one 200W PSU doesn't meet your specific needs, a 150W PSU might!
Which case to get is entirely up to you. The motherboard is Micro-ATX (reference; look under Form factor) so any case that can hold a Micro-ATX motherboard will work. Beyond that, it's a matter of style and expansion capability, which besides the fact that any specific answer would quickly become out of date as manufacturers' offerings change is something only you can decide on.