Text only

CM30076 / CM30082
Individual Project

Project Ideas

Dr Peter Hall

pmh@cs.bath.ac.uk

A front-end for a Cubist paintbox
You will write a front-end to a unique paintbox system that was developed here, in Computer Science at Bath: the Cubist paintbox.
The Cubist paintbox takes ordinary digital photographs as an input, and semi-automatically processes them to look like Cubist paintings; examples of output are on the corridor walls.
The "semi-automatic" means that some user input is required, for two reasons. The first reason is that the photographs need to be segmented, which means breaking them into meaningful pieces, such as an eye, or an apple. Segmentation is very difficult, if the input photograph can be a picture of anything at all --- we allow users to help the segmentation. The second reason for user input is to allow some artistic control over the artistic output.
Your task will be to make the Cubist paintbox more specialised. You will write a front end that allows a user to take mug-shots of someone or other, and automatically produce a portrait. Because you can assume a picture of a face as input, the segmentation is feasible, but still challenging. Automatic segmentation of faces will be a key component of your project. This will be bound in to a larger piece of software that includes an interactive user interface to set the artistic controls, choose photos, etc, and will of course include the Cubist paintbox (which will be supplied).
In the end you will be able to demonstrate a unique piece of software: one which allows users to easily create Cubist-like portraits.
Pre-requisite knowledge: You should be able to program well, and have sound mathematical skills. You should have an interest in both Computer Vision and Computer Graphics. The Cubist code is written in MATLAB --- which is easy to learn, and which interfaces to Java and C.
Indicative reading: Any book on Image Processing/Computer Vision, there are many in the library, will give broad background. Look on the WWW for face recognition and mug-shot processing. I can supply a technical paper that describes the Cubist paintbox.
Computer Vision projects: synopsis
Computer vision is a field of study that aims to make computers see, in some kind of way. It has uses in medicine, security, entertainments, manufacturing and many other industries.
Computer vision as a subject studies problems such as: how to divide a picture into distinct parts (called the segmentation problem); how to make three-dimensional model of the world given a set of images (called reconstruction); and how to identify objects in the world (recognition). Modern trends tend to emphasis the importance of machine learning --- do you think people learn to see?
The basic questions in computer vision remain unsolved (seeing is just too difficult for us to understand right now); there is no way for a computer to segment all scenes in a reliable way, for example. Instead, computer vision focuses on problems we have a hope of solving. For example, we can segment _some_ images into parts based on "texture". By "texture" I mean regions of a photograph which have the same colour or pattern. To give another example, we can work out the shape of objects based on the way they are shaded (called "shape from shading").
I am happy to consider projects in any area of computer vision. These include texture-based segmentation, or shape from shading, or any you may want to suggest yourself. I have suggested example projects in greater detail (image mosaicing, stero reconstruction, painting streak-lines on video), but do not imagine I am limited to these; I am happy to discuss any ideas you have. I am especially interested in using computer vision for computer graphics applications (streak-lines in one example).
A note on how I run projects. I like to divide the aims of any project into two parts: the "baseline" or "necessary" part; and an "advanced" or "optional" part. The baseline is what I consider necessary to show you have a basic grasp of the topic your project is dealing with, and should be a sufficient basis for an acceptable write up. The optional part requires more work, but gives you the chance to enjoy yourself, and show us how good you are.
Pre-requisite knowledge: You must be able to code in any language of our choice (including MATLAB). Taking the Computer Vision option may help, but only in the new year, so you must be willing to engage with basic computer vision literature. Computer vision is a mathematically oriented subject, so you should be comfortable with analytic mathematics, such as linear algebra and geometry. For some projects basic statistics will prove useful too.
Indicative reading: You can get a feel for what computer vision is about by leafing through any standard computer vision textbook. But be aware that for any given project you will need only part of the text. If you look at journals such as IEEE Transactions on Pattern Analysis and Machine Intelligence, or International Journal of Computer Vision, or Image and Vision Computing, then you will have a handle on contemporary issues. I have conference proceedings you may want to browse. You may also want to look at some of the graphics journals: ACM Computer Graphics (SIGGRAPH proceedings), The Visual Computer, Computer Graphics Forum, and IEEE Transactions on Visualization and Computer Graphics; all these provide potential vehicles for applications of vision.
Note that you are not expected to understand what you read at this stage, but you are expected to want to understand a specific part of it.
Image mosaicing
Many of us have glued together holiday snaps to make a larger picture. This project asks you to do exactly that, but on a computer. A moments thought will reveal the kind of processes involved. You must be able to move photographs about so that features line up properly. You may also want to correct for lighting differences, so that one photograph does not look darker than another. You may even want to stretch and distort photographs so that they line up better, or to eliminate effects that arise from taking snaps from different points of view. If you are gluing several photographs together you will want to make sure that all photographs line up in a sensible way.
Image mosaicing is currently one the red-hot topics, because it is used by so many industries in a broad range of applications. Perhaps the most examples are the web pages that let you look around some place or other (I think Bath tourist board uses image mosaics, Encarta 2000 certainly does). The games industry also use mosaicing, and TV studios are interested in ``moving mosaics'', to increase to numbers in a crowd at the Colleseum, say (as used in Gladiator). Mosaicing also forms the basis of yet further applications, such as "super resolution" (raising the number of pixels in an image), and removing moving elements in a scene, such as walking people, to give a "clean shot".
This project is divided into parts. The first part requires you to implement a basic system for using a pair of images. The second part involves you to advance in a direction of your choice. You may wish to correct for lighting differences, or perhaps to mosaic several images together. Throughout the project you will be expected to produce performance metrics for your code; metric include (but are not necessarily limited to) time taken, quality of match, and some indication regarding the range of photographs that your software can cope with (for example, a scene containing a highly repetitive pattern may be impossible for your code to mosaic effectively).
Pre-requisite knowledge: You must be able to code in any language of our choice (including MATLAB). Taking the Computer Vision option may help, but only in the new year, so you must be willing to engage with basic computer vision literature. Computer vision is a mathematically oriented subject, so you should be comfortable with analytic mathematics (such as linear algebra); basic statistics may help too.
Indicative reading: Mosaicing is a new area, of interest to computer vision and computer graphics. You can find out more about it by looking up recent SIGGRAPH proceedings (search for the journal ``Computer Graphics'' in the library). The computer vision elements you need include linear filtering and correlation, look up ``edge detection'' in any standard computer vision textbook to get an idea. The analytic maths you should already have from previous courses. If you can understand edge detection and linear filtering, you should be able to grasp any statistics you may care to use.
Title: Stereo reconstruction
One of the reasons humans have two eyes is to enable "stereo-vision". This means that we are able to make judgments about how far away things are from us, and even make a guess as to the shape of three dimensional objects.
Stereo reconstruction is a branch of computer vision that is concerned with building three-dimensional models from a pair of stereo-photographs - that is a pair of photographs of the same scene, taken just a little distance apart.
This project will require you to write software for stereo reconstruction. The key to doing this is to find points in one image that correspond to points in the second. By "correspond" I mean points that come from the same place in the real three-dimensional world (eg. the corner of a cube).
Initially you can write an interface that allows a user to pick corresponding points, on stereo images of simple scenes, such as few cubes. Your software will use the corresponding points to construct a 3D model (and, as a by-product, work out where the cameras are in 3D). To get this far would be a baseline for this project.
Later you may consider more advanced options of your choice (not necessary, but will make you project look better). For example you may find corresponding points automatically, perhaps after a little user initialization. This may allow you to reconstruct more complicated objects because you are able to identify many more corresponding points. Alternatively, having identified the planes (eg, faces of a dice, sides of a house) in 3D you may want to texture map them. This involves cutting out the corresponding region of photograph (eg the bricks in the house wall) and pasting them onto to plane you have constructed in 3D; a standard computer graphics technique. For a little extra work, you get to make your models look great.
Throughout the project you will be expected to produce performance metrics for your code; metric include (but are not necessarily limited to) time taken, quality of reconstruction, and some indication regarding the range of photographs that your software can cope with (for example, you may be able to reconstruct planes but not curved surfaces).
Pre-requisite knowledge: You must be able to code in any language of our choice (including MATLAB). Taking the Computer Vision option may help, but only in the new year, so you must be willing to engage with basic computer vision literature. Computer vision is a mathematically oriented subject, so you should be comfortable with analytic mathematics, geometry will play a particularly important role.
Indicative reading: Many standard computer vision text-books deal with stereo reconstruction, and it remains an important part of the research literature (look up journals such as ``International Journal of Computer Vision''). This project requires only the rudiments of reconstruction - the more advanced and esoteric areas are not required.
Painting streak-lines on video
You may have seen special effects whereby a moving image is made to look a bit like a painting. And you must have seen cartoons that use streak-lines to indicate motion. This project is about making streak-lines appear on real video.
To make streak-lines appear you first have to compute what is called the "flow field". To make a flow field you have to connect points in one frame of the video with corresponding points in another frame. By "corresponding" I mean points in the two frames that come from the same point in 3D space. If you draw lines between corresponding points of adjacent frame, you have drawn a "flow field". Notice that the lines have a direction (you can tell which way a point is moving). If you continue this process for a few frames, then you have the basic way to draw streak-lines.
Computing (and drawing) a flow-field is a baseline for this project; it is the necessary step. You can go further to make your pictures look good. To do this this you will have to decide which lines of the flow filed to draw, and which to drop. Because you will end up with so many lines in the flow field you will have to come up with some way of automatically paring out unwanted field lines. The final output of the more advanced stage is a single still image, but with streak-lines to indicate motion. (It may be we can provide software to colour the underlying still photograph to make your final picture look really impressive, but that colouring is not part of this project).
Throughout the project you will be expected to produce performance metrics for your code; metric include (but are not necessarily limited to) time taken, quality of flow field, and some indication regarding the range of photographs that your software can cope with (what is some points disappear from view, for example).
Pre-requisite knowledge: You must be able to code in any language of our choice (including MATLAB). Taking the Computer Vision option may help, but only in the new year, so you must be willing to engage with basic computer vision literature. Computer vision is a mathematically oriented subject, so you should be comfortable with analytic mathematics. Some basic statistics (eg correlation) may help too. For the latter part of this project you must be willing to use your imagination.
Indicative reading: Many standard computer vision text-books deal with flow field computation, and it remains an important part of the research literature (look up journals such as "International Journal of Computer Vision"). This project requires only the rudiments of the area --- the more advanced and esoteric directions are not required.
Shape Classification of extracted image features
A well known problem in Computer Vision concerns the classification of an object into one of many predetermined classes of shape. For example, an object may be described as "rectangular", or "triangular".
The baseline for this project is concerned with deciding whether a given object belongs to one of a set of pre-defined classes. There are a number of current shape classification techniques in the literature you may wish to consider. You will pick a number of shapes classes ("rectangle", "circle", "triangle") and decide to which class a new object belongs. Ideally, the new objects will be output from some computer vision software (which we will supply), or be hand-drawn.
Subsequent stages of the project will entail comparative studies of current techniques. For example, analyzing how the robustness of each classifier varies with noise, loss of resolution, distortion of input shape etc. There is scope for extensions toward development of voting schemes, whereby data from multiple classifiers are fused to produce an improved overall classification.
Pre-requisite knowledge: You must be able to code in any language of your choice (including MATLAB). Taking the Computer Vision option may help, but only in the new year, so you must be willing to engage with basic computer vision literature. A knowledge of (or willingness to learn) moderate statistics is required.
Indicative reading: Standard vision textbooks consider classification, but not necessarily shape classification. The book "Statistical Pattern Recognition" by Duda, Hart, and Stork is an excellent book into the kind of statistics that wil be useful. You are not expected to understand this book now, but are expected to want to understand parts of it.
Artificial Painting Toolbox
A number of "painterly rendering" algorithms have been developed, capable of processing digital images to give a "hand-painted" effect. You will implement of a number (three or four) of these techniques, to produce a 'tool box' of painterly rendering algorithms. Given the timescale of the project, strong guidance will be given to the candidate in locating current techniques in the literature.
The second stage of the project will examine the relative merits of each painterly rendering technique through a comparative study. The comparison should include both objective measures (such as detail retained within the painting, robustness in presence of edge/texture clutter), and more subjective measures such as visual aesthetics (judged via user surveys). The identification of good measures will form part of the study. Equal weight will be given to both the implementation and evaluation stages of the project.
Pre-requisite knowledge: Choice of programming language falls to the candidate; however implementation in Matlab, or C would be preferred. This is the least mathematical of all my projects, by some mathematical strength is still preferred. A willing to engage with computer vision texts ahead of the course would prove beneficial.
Indicative reading: Look up ACM Computer Graphics (SIGGRAPH proceedings) for a taster of the kinds of things modern paintboxes can do. Or you may want to use any one of the many packages available for processing home snaps on a PC. (eg. photoshop, pro-paint, photodeluxe, etc).
Locating target shapes in a scene
Object recognition is one of the most important areas of computer vision. You will research and implement a range of techniques for identification of arbitrary target shapes e.g. triangles, squares, in a cluttered image.
The techniques may include, but are not limited to, autocorrelation, the classical Hough/Radon transform, deformable templates, generalised Hough transform (GHT), etc. The candidate will draw conclusions regarding the performance of these techniques in varying debilitating conditions e.g. clutter, noise, versus their algorithmic complexity.
The candidate may wish to propose their own methods as an extension.
Pre-requisite knowledge: Choice of programming language is not enforced but a combination of Matlab and C is advised. you must be willing to engage with the image processing and computer vision literate prior to the course, which is in the new year. You should be have strong analytic mathematics.
Physically Modelling Artistic Media
Many digital 'paint brush' programs use little more than uniform daubs of colour to mimic the paint brush on canvas. This project concerns the physical modelling of brush and canvas e.g. the transfer of paint from bristle to canvas, with an eye to producing more convincing brush strokes. These physical simulations would ideally form part of an interactive paint program.
You will write simple interactive software that allows a user to paint using a brush / canvas model. As baseline you will implement an existing design from the literature. Optionally you will implement additional designs, in which case a critical evaluation of methods will form part of the project. Alternatively, you may design your own canvas/brush approach.
Pre-requisite knowledge: You should be able to code in any language of your choice, but implementation in MATLAB or C would be preferable. A moderately strong mathematical background is required.
Indicative reading: Browse through ACM Computer Graphics (proceedings SIGGRAPH) to get an idea of current thinking.