CSE 228F Lecture 9

Compressed Domain Editing

Why?

There are two primary reasons for working in the compressed domain:

1. Decompressing and Recompressing is very time-consuming.
2. Decompressing and Recompressing introduces quality losses when dealing with lossy compression techniques.

Properties of DCT

The Discrete Cosine Transform has some nice properties that help enable editing in the compressed domain. In particular:

• DCT[A+B] = DCT[A] + DCT[B]
• DCT[A*B] = DCT[A] * DCT[B]

A Simple Scenario

Take simple editing operations such as cut, copy, and paste. If we want to do a cut in the compressed domain, how is this performed?

 If we have an MPEG stream such as the following (in presentation order):

I₁ B₂ B₃ B₄ P₅ B₆ B₇ B₈ I₉ B₁₀ B₁₁ B₁₂ P₁₃ B₁₄ B₁₅ B₁₆ I₁₇

 And say we want to cut frames 7-11 out of the sequence, leaving us with:

I₁ B₂ B₃ B₄ P₅ B₆ B₁₂ P₁₃ B₁₄ B₁₅ B₁₆ I₁₇

We have a problem that B₆ is related to I₉, as we see in the following diagram.

Given that Macroblock MB in frame B₆ is related to the highlighted macroblocks in  I₉ and P₅, how can we perform this operation?

Remember that a motion vector for MB is stored as error values (e) from the values of the predicted blocks (f) and (b). Therefore, the true DCT values for MB are equal to . If we want to convert B₆ into a P-frame so that I₉ will no longer be necessary, we can re-encode e as e' where . This transform is possible because of the additive property of DCT shown above. Now, the macroblock is forward predicted and can be reconstructed simply by adding f to e', and it will give the same result as the original prediction.

Is it really this easy?

The answer is, almost. In this example, we assume that we know the DCT values for f and b. If f or b happen to lie on macroblock boundaries, this is in fact true. However, as we have said before, f and b need not lie on 16x16 macroblock boundaries, and we must take this into consideration. If we have a situation as we see below, where b does not lie within a macroblock boundary, but rather lies between macroblocks B1, B2, B3, and B4, we can use the second property of DCT to give us a DCT value for b.

 If we know the DCT values for macroblock B1, we can find the DCT values for the portion of b that lies in B1 by doing the following. We first need to mask B1 and only keep the portion that is part of b. This is done my multiplying the macroblock by a matrix that has 1's for the pixels you want to keep and 0's for the pixels you want to discard. The result of this looks like the following:

Next, we want to shift this region into the upper-left corner of the matrix, and this is again done with a matrix multiply.

These steps are repeated, masking and shifting the region from B2 to the upper-right, B3 to the lower-left, and B4 to the lower-right until we get the result that looks like the following:

What is the DCT for this region? Using the second property of DCT, we can see that the following equation holds:

You will also have to do the same for macroblock f in order to compute e', and then the whole process will be repeated for frame B₁₂, but we have achieved our goal of compressed-domain cut with no quality loss!