Is there a program that will take the audio from a DVD and turn it into a written transcript. Meaning will it take what is spoken and write down each word.
6 Answers
What you need is software for "speech recognition". There are many available, but be aware that the results usually need a lot of correcting. DVDs will be particularly difficult, because they have several different speakers, possibly a lot of noise and bad/non-standard pronounciation. So you can only expect a rough draft of a transcript, you'll still have to do a lot by hand.
On the technical front, you'll probably have to extract the audio from the DVD (using some ripper software, e.g. mplayer
), then run it through a speech recognition program.
Have a look at the answers under tags speech-recognition
and speech-to-text
on superuser for software you could use.
You could probably find something that reads the captions. I do not know of such a product.
If the content has Closed Captions, those are directly and reliably machine-translatable in the DVD data. However, they are often not what is said exactly, but rather paraphrased so that they can be read, when the dialog is fast (and rarely, just plain wrong).
Subtitles can suffer from the same paraphrasing. Also, on DVDs, they are implemented as graphic overlays, so you actually need some kind of OCR to convert them back to text.
OpenAI's Whisper (MIT license, Python 3.9, CLI) yields some highly accurate transcription.
To use (tested on Ubuntu 20.04 x64 LTS, but also works on Windows or macOS):
conda create -y --name whisperpy39 python==3.9
conda activate whisperpy39
pip install git+https://github.com/openai/whisper.git
sudo apt update && sudo apt install ffmpeg
whisper recording.wav
whisper recording.wav --model large
If using an Nvidia 3090 GPU, add the following after conda activate whisperpy39
pip install -f https://download.pytorch.org/whl/torch_stable.html
conda install pytorch==1.10.1 torchvision torchaudio cudatoolkit=11.0 -c pytorch
Performance info below.
Model inference time:
Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed |
---|---|---|---|---|---|
tiny | 39 M | tiny.en |
tiny |
~1 GB | ~32x |
base | 74 M | base.en |
base |
~1 GB | ~16x |
small | 244 M | small.en |
small |
~2 GB | ~6x |
medium | 769 M | medium.en |
medium |
~5 GB | ~2x |
large | 1550 M | N/A | large |
~10 GB | 1x |
WER on several corpus from https://cdn.openai.com/papers/whisper.pdf:
WER on several languages from https://github.com/openai/whisper/blob/main/language-breakdown.svg:
This is not a task at which computers excel. Have you considered using mechanical turk?
accurate speech recognition without training the software is impossible...for known prompts(like in an IVR) you can do a fuzzy logic thing where it "sounds like" x,y,z or say again...this is not possible to generate transcripts/subtitles for DVDs
-
Well, YouTube has been working on this for quite some time, and nowadays offers automatic caption for many user uploaded videos (yes, experimental)...– ArjanNov 18, 2010 at 22:21
-
1
-