If you haven't got libx264 installed and enabled yet:
Follow this guide in order to install all necessary tools.
Remove the default Ubuntu packages first, then install the missing ones for building the tools we need:
sudo apt-get remove ffmpeg x264 libx264-dev
sudo apt-get update
sudo apt-get install build-essential checkinstall git libfaac-dev libjack-jackd2-dev \
libmp3lame-dev libopencore-amrnb-dev libopencore-amrwb-dev libsdl1.2-dev libtheora-dev \
libva-dev libvdpau-dev libvorbis-dev libx11-dev libxfixes-dev libxvidcore-dev texi2html \
yasm zlib1g-dev
Download and install x264 from source.
git clone git://git.videolan.org/x264
cd x264
./configure --enable-static
make
sudo checkinstall --pkgname=x264 --pkgversion="3:$(./version.sh | \
awk -F'[" ]' '/POINT/{print $4"+git"$5}')" --backup=no --deldoc=yes \
--fstrans=no --default
Download and install ffmpeg from source. Here, the --enable-libx264 flag is important.
git clone git://git.videolan.org/ffmpeg
cd ffmpeg
./configure --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb \
--enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 \
--enable-libxvid --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab
make
sudo checkinstall --pkgname=ffmpeg --pkgversion="5:$(date +%Y%m%d%H%M)-git" --backup=no \
--deldoc=yes --fstrans=no --default
hash x264 ffmpeg ffplay ffprobe
When we're done, we should be able to use x264 over ffmpeg.
If you have libx264 installed and enabled:
First of all, those commands you use look syntactically incorrect. In order to have ffmpeg use x264, you need to supply the -vcodec libx264 argument.
ffmpeg -i input.yuv -vcodec libx264 output.mp4
This should be enough to create an h.264 output file embedded in an MP4 container.
However, this might result in an error due to ffmpeg overriding the x264 settings. I got the error:
[libx264 @ 0x101811200]broken ffmpeg default settings detected
[libx264 @ 0x101811200]use an encoding preset (vpre)
So, let's do this and supply the vpre option:
ffmpeg -i input.yuv -vcodec libx264 -vpre hq output.mp4
Here, "hq" is the name of an ffmpeg preset that includes several options to be passed to x264. There are other presets as well, and they contain most settings that we need for a typical encoding process.
Where you find these presets depends on your ffmpeg installation. Search for files named "ffpreset" in your ffmpeg source folder (if you've compiled from source). Here's a guide on ffmpeg presets too.