Yes and no, it depends on purpose. Assuming "CPU Cabinet" means, well, the case, then yes that is all you will need to make a functioning system. Depending on your use case however, you may need some other optional extensions, such as a graphics card.
It may also be worth purchasing an anti-static wrist strap, and some thermal paste for your first PC build. (The strap to make sure you don't accidentally damage components, and the paste for when you insert your new CPU).
That said, you have to realise you cannot just buy any old CPU, and any old motherboard, and hope they fit together - you must buy compatible components, which can be a pain to figure out what works with what.
If i were to build a system, i would generally work out the compatibility like this:
Choose a CPU
Find a motherboard that matches the Socket of the CPU (Will be listen in the spec of the CPU as well as the spec of the motherboard. Also make sure the motherboard supports what i need, for example, has on-board graphics and sound, has an Ethernet port, etc. Also make sure it has the right connector for the HDD technology you want, and enough sockets for any other items like a CD drive.
Choose compatible RAM. This can be tricky, essentially however you can work it out then ask somebody to confirm they will work together. Remember that when you buy RAM, its speed will be limited to the lowest speed of either the Motherboard FSB or the speed of the RAM itself, so if you buy fast ram, be sure the motherboard supports the speed!
4.Choose a hard drive (or more than one). Make sure that the motherboard supports the connector on the hard drive (e.g. SATA).
Choose a case to mount it in that fits the form factor (generally, any case will work with any motherboard, but do double check)
Choose a power supply. Make sure it's from a reputable maker, as a bad power supply can fry everything in short order. 500w should be enough unless you have extension cards.
Choose any extension cards, making sure your motherboard has the correct slot for them (AGP, PCIe, etc). These are things like Network cards, Graphics cards, etc.
As you can see, there are a vast array of things to consider with regards to compatibility. Definitely find somebody who is knowledgeable about all of this to verify the parts you have chosen before spending cash on them.