I've successfully managed to extract OCR data from frames using ffprobe with the following command:

ffprobe -loglevel quiet -hide_banner -select_streams v:0 -show_entries frame_tags=lavfi.ocr.text -f lavfi "movie=movie.mov,ocr" -of flat

It neatly outputs the info on one line like this:

frames.frame.0.tags.lavfi_ocr_text="EE 6:\n_ 00:59:35.00\n"
frames.frame.1.tags.lavfi_ocr_text="ET =:\na Clo Koy\n"
frames.frame.2.tags.lavfi_ocr_text="EE =:\na loony\n"
frames.frame.3.tags.lavfi_ocr_text="=:\na 00:59:35.03\n"
frames.frame.4.tags.lavfi_ocr_text="U6:\na 00:59:35.04\n"
frames.frame.5.tags.lavfi_ocr_text="6:\nal 00:59:35.05\n"
frames.frame.6.tags.lavfi_ocr_text="EE Zz:\nim 00:59:35.06\n"
frames.frame.7.tags.lavfi_ocr_text="=:\nim ll¢kosc hc Lon OYA\n"
frames.frame.8.tags.lavfi_ocr_text="=:\nim 00:59:35.08\n"
frames.frame.9.tags.lavfi_ocr_text="Zz:\nim 00:59:35.09\n"
frames.frame.10.tags.lavfi_ocr_text="=:\nim 00:59:35.10\n"

However, since scanning each frame is a resource intensive operation and depending on how much text is on screen it can sometimes take several seconds per frame I created another probe to detect the first (almost) completely black frame:

ffprobe -loglevel quiet -hide_banner -select_stream v:0 -show_entries frame_tags=lavfi.blackframe.pblack -f lavfi "movie=movie.mov,blackframe" -of flat | head -n1

This command outputs the following:


I managed to strip off all the stuff except for the "240" number I need. What I want now is to use this number to offset the starting frame for the first command so that its output will start on that frame like this:

frames.frame.240.tags.lavfi_ocr_text="EE 6:\n_ 00:59:35.00\n"
frames.frame.241.tags.lavfi_ocr_text="ET =:\na Clo Koy\n"
frames.frame.242.tags.lavfi_ocr_text="EE =:\na loony\n"
frames.frame.243.tags.lavfi_ocr_text="=:\na 00:59:35.03\n"
frames.frame.244.tags.lavfi_ocr_text="U6:\na 00:59:35.04\n"
frames.frame.245.tags.lavfi_ocr_text="6:\nal 00:59:35.05\n"
frames.frame.246.tags.lavfi_ocr_text="EE Zz:\nim 00:59:35.06\n"
frames.frame.247.tags.lavfi_ocr_text="=:\nim ll¢kosc hc Lon OYA\n"
frames.frame.248.tags.lavfi_ocr_text="=:\nim 00:59:35.08\n"
frames.frame.249.tags.lavfi_ocr_text="Zz:\nim 00:59:35.09\n"
frames.frame.250.tags.lavfi_ocr_text="=:\nim 00:59:35.10\n"

I've already tried creating JPG's with ffmpeg that I can then use with ffprobe but this means creating temp files and stuff. I want to prevent this if possible. I also looked at the -read_intervals option but this doesn't seem to work at all.

I'm running macOS Mojave 10.14.3 with ffprobe:

ffprobe version 4.1.1 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.0 (clang-1000.11.45.5)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.1.1 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags='-I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include -I/Library/Java/JavaVirtualMachines/openjdk-11.0.2.jdk/Contents/Home/include/darwin' --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-videotoolbox --disable-libjack --disable-indev=jack --enable-libaom --enable-libsoxr
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Simple multimedia streams analyzer
usage: ffprobe [OPTIONS] [INPUT_FILE]

Can what I want be done using ffprobe alone?


Turns out the solution was quite simple.

I added the trim parameter to the lavfi filter with the start_pts option and also changed some of the output options.

Full command:

ffprobe -loglevel error -hide_banner -select_streams v:0 -show_frames -show_entries frame=lavfi.ocr.text -f lavfi -i "movie=movie.mov,trim=start_pts=20,ocr" -of flat

It now outputs the following:

frames.frame.0.tags.lavfi_ocr_text="=:\nim 00:59:35.20\n"
frames.frame.1.tags.lavfi_ocr_text="=:\nim 00:59:35.21\n"
frames.frame.2.tags.lavfi_ocr_text="=:\nim lelehkosc clon)\n"
frames.frame.3.tags.lavfi_ocr_text="=:\nim 00:59:35.23\n"
frames.frame.4.tags.lavfi_ocr_text="Zz:\nim 00:59:35.24\n"
frames.frame.5.tags.lavfi_ocr_text="i\nim 00:59:36.00\n"
frames.frame.6.tags.lavfi_ocr_text="=:\nim 00:59:36.01\n"
frames.frame.7.tags.lavfi_ocr_text="=:\nim 00:59:36.02\n"
frames.frame.8.tags.lavfi_ocr_text="=:\nim 00:59:36.03\n"
frames.frame.9.tags.lavfi_ocr_text="=:\nim 00:59:36.04\n"
frames.frame.10.tags.lavfi_ocr_text="=:\nim 00:59:36.05\n"

It still starts at "frame.0" but these are not the frame positions of the original video as you can see from the detected timecode which now starts at 00:59:35.20 instead of 00:59:35.00.

Hope this helps others.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.