ffplay - how to play together separate video and audio files

Question

So I am able to transcode separate video and audio files into one movie file, like so:

ffmpeg -ss 0 -lowres 0 -r 24 -i dcp/picture.mxf -vf "xyz2rgb" -i mov/sound.wav -ac 2 -ab 256k -vb 3000k -t 10 -y mov/movie.mp4

And see and hear the resulting movie like so:

ffplay -ss 0 -lowres 1 -vf "xyz2rgb" mov/movie.mp4

However, to avoid having to transcode the movie and sound together into one file before being able to see it, I would like to simply play the two separate files together. As far as I know, ffplay does not let you combine two different files (right?).

Reading these forums it seems that you can pipe an ffmpeg output to ffplay. The following is as close as I got. It's not that close. :( Video information is clearly being carried over, but it is green "snow" in the right frame size, with hints of the actual imagery dancing inside the horrible green snow. This way I know that something of the actual picture data is carrying over, just not rendering correctly. At all.

And no audio plays. The sound is not carrying over (I don't think.) Now... I'm stuck. The following suggests what I'm trying to do, but it's not working:

ffmpeg -ss 0 -lowres 0 -r 24 -i dcp/picture.mxf -vf "xyz2rgb" -i mov/sound.wav -f rawvideo - | ffplay -f rawvideo -vf "xyz2rgb" -s 1998x1080 -

Answer 1

Video information is clearly being carried over, but it is green "snow" in the right frame size, with hints of the actual imagery dancing inside the horrible green snow.

This must be the pixel format. You're applying the xyz2rgb filter which converts the DCP colorspace into RGB, and then you're piping raw video to ffplay. Here, you're applying the xyz2rgb filter again, which probably causes the distortiions, since FFmpeg has no idea how raw video is encoded. It'll do whatever you want with the video, even converting the colorspace twice.

---

I've had good results piping the Matroska format to ffplay instead of rawvideo, which also gives you the ability to decode audio (see below). The Matroska format can contain any kind of codec.

Note that the order of options matters in FFmpeg, and you should probably use a more recent version as it's more strict about where you put the options:

• Global options like -y come first.
• Then, specify options for reading input files – the -r 24 forces the frame rate for the picture.mxf file.
• Then specify all input files.
• Then come the codec options such as video filters, the video bitrate, etc.

In your situation, it makes sense to just copy the audio stream, as we don't really need to transcode it. The video has to be transcoded in order to be able to apply the filter.

Something like this should do. Note how we explicitly set the output format as matroska and let ffplay figure out the rest itself:

ffmpeg -r 24 -i dcp/picture.mxf -i mov/sound.wav \
-filter:v "xyz2rgb" -b:v 3000k -c:a copy -t 10 -f matroska - | \
ffplay -

I've used the (IMO less ambiguous) syntax for options such as -filter:v instead of -vf, -b:v instead of -vb, and -c:a instead of -acodec.

If that format doesn't work for you, try AVI:

ffmpeg -i picture.mxf -i sound.wav -f avi - | ffplay -

This worked for me without problems, and it didn't even require me to use the filter you needed. Audio was working as well.

Answer 2

ffmpeg -lowres 2 -ss 0 -r24 -i MOVIE.MOV -ss 0 -i AUDIO.WAV -vf xyz2rgb -ac: 2 -c:v mpeg2video -f avi - | ffplay -

For my purposes, this allows quick review of the contents of a DCP (Digital Cinema Package) that you can build with open source tools . The DCP uses the xyz color space, so you need to have an xyz video filter installed, otherwise you cannot use it in the line above. (That's okay, it'll just look heavily orange tinted, you won't get to see the true colors.)

I added mpeg2video codec b/c, though reducing quality, it's low rate and buffers a little, so it performs more smoothly on my machine. (Also, for some reason, without transcoding first I get only a freeze frame from the jpeg2000 mjpeg sequence feeding at a mighty 125mb/s from the MXF, unless I start "- ss 0", that is, at the very beginning of the file.) With a VERY FAST machine, specifying no codec (remove "-c:v mpeg2video"), full size ("-lores 0"), and starting at 0 (-ss 0), you can theoretically review the 2K cinema quality DCP exactly as is. Well, except for any special instructions included in the XML's -- subtitles or other programming. But generally. (BTW "-ac: 2" downmixes 5.1 to 2.0 -- though you may have hardware that supports auditioning 5.1, in which case use "-c:a copy".)

Thanks to Matthias at Belle-Nuit, and slhck above for FOR PATIENTLY walking me through this. Thank you!!!! This is quite useful. And in fact, saved me from a potentially embarrassing film screening at Aero theater Sunday evening. My image sequence numbering was out of sequence -- found it with this and fixed!

Wow, all this for just one command line, who knew ;)

Cheereo - b worldpoop.com

Answer 3

This wouldn't transcode the streams:

ffmpeg -i video.mkv -i audio.mkv -c copy -map 0:0 -map 1:1 output.mkv

-map 1:1 = second file, stream 1 (usually audio). -c copy = -c:v copy -c:a copy. By default the muxed file is as long as the longer file, but -shortest would make it as long as the shorter file.