2

I am attempting to create the appearance of a picture in a picture video using multiple video files.

I've created a .mp4 file using FFMpeg. The video displays correctly in Windows "Movies & TV" app. It does not display correctly in Chrome, Firefox, VLC Media Player or any other app I have. I plan to display the video in a Browser, so cross browser support is important to me.

The video was created by taking two parts of a video call and doing the following with FFMpeg:

  1. Put remote video (remote.mp4) in background.
  2. The remote video should show the first image of the video and appear paused until about 2.501 seconds. I've done this using the itoffset.
  3. The remote audio should also start playing after 2.501 seconds. Setting the itoffset did not seem to accomplish this so I added a "adelay" option to the filter_complex.
  4. Placed local video (local.mp4) as an overlay in the bottom right corner.
  5. The local video and audio should start playing at 0 seconds (immediately upon playing the video.)
  6. Rotated the background picture 90 degrees (it was recorded sideways).
  7. Add a audio delay to the remote video so it starts playing the audio stream 2.501 seconds after the start.
  8. Merge the audio inputs.
  9. Used aac codec (because it seems popular, I'm totally willing to switch if that would be useful.)
  10. Use h264 video codec, again because it seems popular.

My FFMpeg command is as follows:

ffmpeg -itsoffset 2.501 -i remote.mp4 -i local.mp4 \
     -filter_complex \
     " [1:v]scale=iw/4:-1:flags=lanczos[loc0]; \
     [0:v]transpose=1[rotate1]; \
     [rotate1][loc0]overlay=main_w-overlay_w-10:main_h-overlay_h-10:eof_action=pass[rem0]; \
     [0:a]adelay=2501|2501[0adelay]; \
     [0adelay]apad[0a]; \
     [0a][1:a]amerge=inputs=2[a]" \
     -map "[rem0]" -map "[a]" \
     -ac 2 -vcodec libx264 \
     -ar 44100 -acodec aac \
     completed.mp4

Video Properties:

  1. completed.mp4 (16 seconds, 480x640)
  2. local.mp4 (16 seconds, 640x480)
  3. remote.mp4 (14 seconds, 640x480, turned sideways)

When I look at the completed video in a browser, instead of being 16 seconds it says 18 seconds. When I use ffprobe on the completed video it says: Duration: 00:00:16.68. The local audio and video also waits 2.501 seconds to play in a browser.

Any help would be wonderful. I have no idea why the browser is rendering it differently than I intend. Thanks!

2

With timestamp offsets, ffmpeg will effect that through edit lists in the output MP4. Looks like browsers don't parse them. So, we need a workaround.

ffmpeg -i remote.mp4 -i local.mp4 \
     -filter_complex \
     " [1:v]scale=iw/4:-1:flags=lanczos[loc0]; \
     [0:v]transpose=1,setpts='if(eq(N,0),PTS,PTS+2.501/TB)',fps=30[rotate1]; \
     [rotate1][loc0]overlay=main_w-overlay_w-10:main_h-overlay_h-10:eof_action=pass[rem0]; \
     [0:a]adelay=2501|2501,apad[0a]; \
     [0a][1:a]amerge=inputs=2[a]" \
     -map "[rem0]" -map "[a]" \
     -ac 2 -vcodec libx264 \
     -ar 44100 -acodec aac \
     completed.mp4

The setpts shifts timestamps of all frames except the first. The fps filter then fills in that gap with duplicates of the first frame. I've assumed an input stream rate of 30.

  • Sorry, n should be N – Gyan Jun 8 '18 at 15:32
  • That works perfectly! Thanks a bunch. It totally explains and resolves the issue! – Daryl Jun 8 '18 at 15:35

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.