Knowing more about what you want to accomplish would help us give you a more appropriate answer. Are you working on some sort of voice/video chat client? If so, what's it going to run on? PCs only, or are you looking at portable (smartphone) implementations too? Or are you looking at some sort of audio streaming? What's your content? What are your clients?
If you're looking to minimize encoding delay, Speex may be your most mature implementation right now over a wide range of natural audio. Opus and AAC-LD are also available, but I'm not as confident of their relative maturity or penetration AAC-LD has the advantage of being backed by a large standards body and may have more hardware implementations available than the others.
That said, in a recent 64 kb/s listening test over at Hydrogen Audio, CELT was ranked above all of Vorbis, Nero HE-AAC and Apple's HE-AAC implementations with regards to sound quality. Results link.