The Implementation of an Interactive Video Codec

Edward Chang
(echang@cs.stanford.edu)
Last Updated: December 9, 1998

I. Overview

My plan is to implement a video codec that supports VCR-like interactive operations such as slow motion, pause, replay, and fast forward. A few application ideas brought up in class such as personal sports replay and self-paced lectures can be realized via this codec.

To implement a codec that supports VCR-like interactive features, the codec needs a cache manager that is able to buffer large amount of data and supplies data to the decoder in real-time. A naive approach is buying a lot of memory. However, since the memory requirement can be huge (described shortly), this main-memory-only buffering approach may be cost prohibitive. For instance, asking consumer to pay $1,000 to buy a set-up box can be a tough sell.

The alternative is using a memory-disk integrated cache (MEDIC). Since the per-byte disk cost is about one hundredth of the per-byte memory cost, MEDIC is economically attractive. To examine the need for MEDIC, we address two questions:

  • Why is a cache needed at the client side?
    Why do we need an integrated memory-disk cache?
  • The answer to the first question is fairly obvious: a client cache is needed as a cushion against variability in the data delivery rate, and against differences in delivery and consumption rates. If one assumes that the media server always pumps out the data at precisely the rate it will be consumed by the client, and one assumes a perfect network, which can deliver the data at this same rate, then one does not need a buffer at the client. However, the server and network may deliver data at a rate different from that at which data is consumed, and the rate may not be constant, due to network disturbances, traffic congestion, router failures, server glitches, and so on.

    A large cache at the client site may enable an array of client applications. A cache can be made larger at low cost by adding disk storage, and hence the usefulness of a disk cache. A large cache gives clients the flexibility to receive much more data than it consumes. For instance, video panorama may require eight-channel data delivery at high rates but decodes, stitches, and displays only the frames that the user views. These applications can easily require gegabytes of buffering space. A large client cache also gives the server more delivery flexibility. For instance, the server can speed up data delivery when the channel is free, anticipating the peak hours are arriving. Also, when a VCR-like function. e.g., a pause or replay command, is initiated by the viewer, the client can continue receiving bits into its local disk without disrupting the server. Without client assistance, these operations either cannot be implemented (in the broadcast case) or prolong playback time and hence significantly decrease server throughput (in the point-to-point transmission case).

    II. Deliverables

    APIs:

    A set of APIs, which allow the upper level applications to pause the playback, perform instant replay, or change the playback speed (e.g., slow motion).

  • DTVPause(int channel, boolean pause);
  • channel: The channel number.
    pause: The playback is paused if the pause variable is set to TRUE, or the playback continues.
  • DTVReplay(int channel, int offset);
  • offset: The number of frames to go back to restart the playback. The actual first frame to be played must be an I frame.
  • DTVChangeSpeed(int channel, int bitrate);
  • Codec:

    A video (only) codec that supports pause, fast forward, and replay operations in addition to the regular playback. I have discussed with the "Lecture of the Future" group to perhaps merge two efforts into one.

    III. Related Infrastructure

    Special Help Needed from Milton: Need to be able to find out the starting/ending address of the decoder buffer and the next byte of data the decoder is going to read. Need to be able to write to the decoder buffer (I suppose this is already allowed.)

    Hardware: PC/Pentium II and ATI 3D Rage Pro (DirectX).

    Software: VC++, MFC, DirectDraw, etc.

    IV. Related Publications

    V. Development Plan