How to separate voice and background music from a video file

Question

Is it possible to separate voice and background music from a video file?

I only need the background music.

Type of video: MP4

Working on Windows XP 32 bit SP3

Answer 1

Unless they're separate audio tracks in your video, not easily. What you'll probably have to do is extract the audio track from the video into a separate file, edit the audio file with a dedicated tool, then remux the result back into the video.

The demux/remux part is easy. What's going to be difficult is attempting to isolate the background music. You'll probably have to experiment with different effects, all of which will most likely result in either a significant loss of fidelity in the audio or not entirely removing the dialogue, if not both. What's more is that you're going to be re-encoding that output into a new mp3/aac file, and between the re-encoding and audio processing, your output is going to sound much worse than the original.

You may have better results by trying to re-master the background music and replacing the audio track in the movie file entirely.

Answer 2

Spleeter

Spleeter is a Python library that can extract music and vocals from a joint audio source. It is machine-learning based and can provide different output types (the number of stems extracted).

It provides the following output:

• Vocals (singing voice) / accompaniment separation (2 stems)
• Vocals / drums / bass / other separation (4 stems)
• Vocals / drums / bass / piano / other separation (5 stems)

Audacity

Audacity – a free and open-source cross-platform audio editor – can do this, using the Vocal Reduction and Isolation effect. You should first extract the audio from the video file, e.g. using ffmpeg:

ffmpeg -i video.mp4 -c:a pcm_s16le audio.wav

And then load the audio.wav file into Audacity:

If you only want to get background music, select the Remove Vocals option; if you want the opposite, choose Isolate Vocals.

Note that this is never going to sound perfect. Vocal isolation is a hard task, as everything you hear is basically mixed into two tracks. An algorithm will never be as good as your brain in isolating different sound sources. Your audio source should be a stereo file with the vocals being panned dead-center. It might also produce false-positives, removing other instruments in the process.

Answer 3

I managed to remove background conversation/noise from a video by running a low-pass filter that was built in to the editor that I was using. The whirrs of my robot, which was the target of my video, remained.

Note that I am not an audio expert, and I cannot guarantee this will work for your particular case, but it's worth a try.