Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

is it possible to do partial edits to each frame of video (e.g. adding a logo) without transcoding? And save video with the same format?

I mean, pixel positions of each frame are calculable; so is it possible to modify file bytes in such a way that they result in the desired change on a frame (e.g. adding a logo)?

I know in formats like XviD this is not possible (easily at least) but it is completely impossible or is possible for some kind of formats (e.g. MPG or raw AVI)?

share|improve this question

2 Answers 2

up vote 4 down vote accepted

This is only (practically) possible for lossless video encoding.

Lossless to lossless

It's very simple to do what you want for losslessly encoded video codecs. Many of those store the video data pixel-by-pixel, mostly in the YUV colorspace. Also, every frame stands on its own.

It would be pretty easy to edit such a video because you know where every frame is, and where every pixel position is in the byte stream. You can simply overwrite parts of the byte stream without consequences to the remaining stream. Even if the lossless video was arithmetically compressed, you could uncompress it first, edit, and then compress and save again—no problem with that.

So, for example, if you have a raw YUV video in an AVI container, you could edit it on a frame-by-frame basis and save it to YUV in AVI again.

Keeping the lossy encoding

But if you want to do lossy video encoding—or keep the original lossy encoding—after the editing step, this is impossible. There are two problems with that: the encoding process itself, and the fact that often, frames depend on each other.

Lossy compression tries to remove redundancy as much as possible, by removing details the human eye doesn't see. This is done in several steps, but the most important one involves transforming the pixel domain into the frequency domain, often with variants of the Discrete Cosine Transform.

What this step does is that it takes a block of, say, 8×8 pixels and transforms it into a block of frequency coefficients. From this block, certain coefficients are dropped, which reduces the amount of information (and thus compresses the size), but also throws away visual information, which reduces the quality of the video. Which coefficients are dropped depends on the quality setting of the encoder. The video is then not stored as pixels, but as frequency coefficients.

When you want to edit a lossy video, you first have to reconstruct the frequency coefficients into a pixel-by-pixel representation again (like mentioned above). At this point you could edit the video, and insert a logo, but once you'd want to store it again, you'd have to perform the transformation step again—and throw away information. This is, in essence, the main cause for generation loss.

Another problem is the fact that for most lossy video, some frames depend on the information contained in other frames. More specifically, B- and P-frames only contain offsets of earlier (or later) B/P or I-frames. If you changed the contents of an I-frame, all the other dependent frames would change when they are decoded, which is typically not what you want. Peter Cordes' answer below hightlights this point. And indeed he's right that in principle, you could edit a lossy I-frame only video in place, but it would be practically very hard to accomplish.

So, very simply speaking, unless you can store a video losslessly, you can't edit a video in-place without sacrificing quality.

share|improve this answer
    
Actually, you don't technically need your source to be a lossless representation of anything. Editting without doing any lossy compression to the output of your edit will result in a file MUCH bigger than the original, but will have exactly the same quality as the original. –  Peter Cordes Jan 17 at 16:57
    
Absolutely. I tried to clarify that part, but your answer is good. I guess I was more thinking about what's realistic in the sense of… is there a tool that does this job. And I have yet to see a nice implementation of something that modifies bitstreams to change some macroblock frequencies in such a way that the inverse transformation gives you what you originally wanted (as specified in the pixel domain). –  slhck Jan 17 at 17:37
    
Yeah, I agree most people wouldn't want to inflate their videos by a factor of 10 to 100. Since if there are already artifacts from something, it's easier to accept more. –  Peter Cordes Jan 17 at 21:59
    
re: bitstream hacking: There are still-image jpeg tools like jpegclub.org/jpegtran (packaged for ages in most Linux distros as part of libjpeg-progs / libjpeg-turbo-progs) that do lossless transformation like crop, rotate, or even rescale (DCT is cool). Replacing some blocks with other blocks should be easy, too, but IDK if there's existing code for it. –  Peter Cordes Jan 17 at 22:04
    
actually, looks like there is existing code: I see jpegclub.org/jpegjoin/jpegjoin.txt. "The main purpose of Jpegjoin is to compose (join) multiple images into a single image, similar to the arrangement of row and column cells in a table. Also provided is a feature to drop logo and legend images on the bottom of the cells." –  Peter Cordes Jan 17 at 22:06

slhck is correct if we're talking about complex video codecs with motion vectors (edit: actually, any kind of intra or inter prediction). You could modify a h.264 stream by re-encoding only the macroblocks in the area you want to replace with your logo. Logo in I frames, skip blocks in P and B frames. But any time the camera panned, there would be nearby blocks that get their picture data by copying from the part of the picture that's now logoed. So the edges of your logo would get copied around, corrupting other parts of the picture until the next I frame.

So he's right, but it's because of references, not lossy-ness. (edit: His edits clear up most of the confusion.)

For simpler codecs where every block is independent (not even any intra prediction), e.g. mjpeg, you could replace the coded blocks only in the area you wanted to logo, without decoding / re-encoding the rest of the picture. I think there's support for doing this in a single jpeg image file, but IDK if anything supports it for mjpeg, without breaking out the frames to separate jpeg files.

http://jpegclub.org/jpegjoin/jpegjoin.txt has info on how to use jpegtran to replace some blocks of jpeg files, without decoding/encoding the rest of the image.

Here's an example of mucking around with a video stream without decoding/re-encoding. Hiding data in a h.264 stream, in the low coefficients of iPCM macroblocks. (literal, not DCTed, coding.) You have to decode / re-encode the CABAC entropy-coding layer. (It's the final zip-style compression for the bitstream.)

If the source is in a lossless format, you're not incurring a huge filesize cost by decompressing, editting, and recompressing the whole thing, just CPU. Many lossless formats essentially zip each whole frame separately, rather than operating on blocks, so you'd have to do a full decompress-edit-recompress. Lossless h.264 has all the inter and intra prediction of normal lossy h.264, so even though it does have macroblocks, a full decode-edit-encode is needed there, too.

None of this changes anything for the actual practical purpose of logoing a video, UNLESS your lossy source was mjpeg. I'm just writing this for those curious about why.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.