Up until now, we have focused on the compression layer of MPEG. The system layer wraps around the compression layer.

It is responsible for:

• Interleaving of Audio & Video/Multiplexing
• Synchronization and Continuity
• Transmission on Networks

MPEG-2 adds a transport layer. It has special methods for Error Control and Transmission over networks.

An MPEG encoder takes video frames and audio samples and first encodes them. Then, the encoded video frames and encoded PCM audio bytes are sent to the packetization stage. Here, boundaries between frames are lost and we only deal with packets. For MPEG-1, packets can be any size. MPEG-2 sets the packet size to 188 bytes.

MPEG has two timestamps that are inserted into the packet header, the DTS or decoding timestamp, and the PTS, or presentation timestamp. Remember the difference between a stream in decoding order and a stream in presentation order? The decoder will read the packets and use the DTS to know when to decode the frame. The PTS will be used to know when to display the frame. The DTS and PTS for a packet refer to the first frame that begins in a packet. They are obtained using the encoder's clock as shown in the above diagram.

It is not possible to present a frame before decoding it. Generally, the PTS is given at a constant rate (such as 30 frames per second), and the difference between PTS and DTS must be large enough to permit the frame to be decoded before presenting it.

The final step in encoding is producing packs. A pack is a common system layer that multiplexes (also called interleaving) packets from various media and produces one stream A stream of packs in MPEG-1 is called an ISO 11172 stream. An example pack is shown here that interleaves six video packets with one audio packet.

A third timestamp is inserted at the pack level. This timestamp is called the SCR, or System Clock Reference. We will go into more detail about this in a moment.

A diagram of an MPEG STD or System Target Decoder is shown here:

The STD has a clock that it uses to decide when to decode and present the packet information. A problem arises if you consider the case that the decoder's clock may not match the encoder's clock. If, for example, you encoded a movie at 5:00 and your encoder's clock was set to 5h 0m 0s and you want to play this movie back when your decoder reads 10h 0m 0s, the decoder will think that all packets arrived late and it will not display anything!

The SCR is designed to alleviate this problem. The SCR in the very first pack of a sequence (encoded in the System Header found in the first pack) will set the decoder's clock to the value that the encoder's clock was set to when the sequence was encoded. In addition, since different clocks drift away from real time at different rates, SCR values must be sent at a rate of at least one every 0.7 seconds (700 ms) so that the decoder's clock can be re-synchronized with the encoder's clock before it drifts too far away.

• Continuity means that each frame is displayed at a constant rate within a media.
• Synchronization means that different media play at the same rate (e.g. Audio and Video)
• Delay refers to the time for one packet to get from source to destination.
• Jitter refers to the variation in time it takes for packets to get from source to destination.