You will need to adjust the audio speed to match the video and re-encode it, but you can do this without re-encoding the video.
First, compute how much you need to adjust the audio. Assuming this was originally a 25fps video you are slowing to 23.976, your speed factor is 23.976/25 = 0.959. We'll plug that into the
atempo filter to adjust the speed of the audio, pulling it from the original
ffmpeg -i source.mkv -r 23.976 -i video.h264 -filter_complex "[0:a]atempo=0.959[out]" -map 1:v -map "[out]" -c:a aac -c:v copy out.mp4
-filter_complex here is taking the audio stream from the zeroth input file (
source.mkv) and running it through the
atempo filter, then naming the output
[out]. We then use
-map arguments to say that we want to take that audio stream as well as the video stream from the first input file (starting from zero, so actually the second one,
video.h264). We have to reencode the audio because we want to change it, but we can just copy the video into the container.