Media Technology
Research Centre
  Shrink to Fit: Software Results
  High resolution images produced in the post production phase of image compositing yields a huge quantity of pixel data. The subsequent demands for high memory capacity and processing speed means that digital special effects are only feasible on expensive hardware. IRCS, the Image Rendering and Compositing System, was designed to be pixel-independent so as not to be constrained by such demands.

Image data structures mimic the rostrum model used in traditional cartoon animation. Images are painted onto virtual "cels" (transparent celuloid sheets), which are layered over each other in a stack. There are controls for front and back lighting on each cel. The final image is then made of a composite of all the overlapping cels in the stack. The contents of each stack are saved to a stack file. This contains details of the various objects in the structure hierarchy, the textures assigned to them, the cels to which they belong as well as other details such as brightness and lighting.

The Rostrum Setup

The cels may be manipulated by matrices that allow simple transformations (such as scaling and rotation). More specific operations may also be applied (e.g. blur and changes to brightness). On-demand computation avoids the rendering of objects that are occluded in the background, thus improving processing speed.

Once the stack structure has been created, it is then rendered and composited. IRCS computes the image scan-line by scan-line (each one being compressed as soon as it is computed) by producing colour coherent spans. Each span consists of a row of pixels that are defined by a flat colour (where each pixel has the same value), by a gradient (where the pixels are interpolated between the leftmost and rightmost pixels) or by an array (where the value of each pixel is stored explicitly). Not only do the spans compress the pixel data, but they also allow the rendering to be independent of the width of the image. This is possible if no arrays are used and the rendering/compositing algorithms make use only of arbitrary-precision spans (not the individual pixels themselves). IRCS does this wherever possible. Avoiding the use of a frame buffer allows rendering with almost no limit on resolution or aspect ratio.

Variations on the basic IRCS allow specialist visual effects. The "NPR" (Non-Photorealistic Rendering) variant renders images in such a way as to mimic hand drawn artwork (such as a pen and ink illustration). Current work on a "Quasi3D" variant handles cel intersection. Those parts of the cels that are behind the plane of intersection are occluded by those parts that are closer to the camera. There is also a utility "Stkps" version that renders stack files, as produced by the image processing utilities, into PostScript files.

Vector-Based Image Manipulation

The first stage of representing the image is to decompose it into structural regions. The regions are identified from areas, roughly outlined by the user, that are characteristic in their texture content. A segmentation of the image is produced, by assigning probabilities to each pixel belonging to a particular region and then using relaxation. Homogeneous regions are thereby separated by sharp boundaries - meaning at arbitrary precision - that represent areas of characteristic texture. These boundaries are the dynamic curves called "snakes". Note that the separated areas do not represent separate objects in the image, but areas of characteristic texture that may be grouped together if so desired. Each snake is associated with an energy function that determines its rigidity.

Having separated the image with structural contours, it is then necessary to fill its interior. This may be done by using one flat colour whose value is determined by calculating the average of all the colours contained within the original structural contour. This approach would be appropriate for cartoon animation. For a photo-realistic application, however, continuous varying colours have to be computed. This is done by triangulating each region using a quality-conforming Delauney triangulator. A colour is assigned to each vertex based on the colour of the nearest pixel in the original image. Colour within the enclosing space of the triangle is then interpolated as a function of the vertex colours.

Having decomposed the image and assigned textures to each region, IRCS is then used to render the image. IRCS is particularly suited to this application as manipulating separated objects is made easier by assigning tham to different layers within the 2.5 dimensional structure. Furthermore, its ability to render without the use of a frame buffer and to operate on multiple pixels (instead of one at a time) makes it very efficient. It is thus able to render images of very high resolution (those that would be produced by film). Using the analyser in conjunction with IRCS, one can represent the image as a continuous vectorial imitation of the original, or (more interestingly) as an altered version that allows for image deformation, such as warping.

Back to Shrink to Fit