Image Processing and Computer Vision

Labs for bildat08

General info

There will be three compulsory labs (Lab1, Lab2, Lab3) that are done in Matlab. Students should work in pairs. It is possible to do the labs individually but those working in pairs will have precedence during the oral presentations of labs. Single labs may not be kept unreported for more than a year. Upon failure to complete the labs, they all have to be repeated next time the course is taught. The reason for this is that the course usually changes from one year to the next. Also observe that labs completed before the deadlines will result in 5 bonus points added to your exam. During the presentation of the labs results, the most important evaluation criteria are the experimental results and conclusions you draw based in these. If required from the TA, you may be asked to reproduce the results. You have to follow the questions stated in the lab instructions and answer them. These questions serve the purpose of summarizing the results you obtained during the laboratory work.

Lab instructions in english

Laboration 1

in english
in swedish

Laboration 2

in english
in swedish

Laboration 3

in english
in swedish

Schedule

Date	Place	Time		Who		What
23.1.	Brown	15-17		Oscar/Per		Help
25.1.	Brown	13-15		Javier/Per		Help
29.1.	Orange	15-17		Javier		Help
30.1.	Brown	15-17		Per/Oscar/Javier		Redovisning
1.2	Green	13-15		Per/Oscar/Javier		Redovisning/Deadline
6.2.	Brown	15-17		Oscar/Javier		Help
8.2.	Yellow	13-15		Per/Oscar		Help
13.2.	Green	15-18		Per/Oscar/Javier		Redovisning
15.2	Green	13-15		Per/Oscar/Javier		Redovisning/Deadline
20.2.	Brown	10-12		Oscar/Per		Help
22.2.	Orange	13-15		Javier/Oscar		Help
27.2.	Orange	15-17		Javier/Per		Help
5.3.	Green	15-18		Per/Oscar/Javier		Redovisning
7.3.	Brown	13-15		Per/Oscar/Javier		Redovisning/Deadline

Supervision

If you need help during the booked lab hours, you preferably contact the course leader or assistants on site. If you are in serious need of help, you may use the booking system and reserve an empty time slot. Students with reserved time slots have precedence over those without. You can also talk to the lecturer in connection to the regular seminars. Assistants may be contacted by telephone or email (the latter is preferable). Finally, you may talk to the course leader during visiting hours (after taking contact by email). From experiences gained during the courses of previous years, we know that the need for supervision varies a lot from one person to the next. Furthermore, the work load is particular high for the assistants during the weeks ending with a lab report deadline, while the capacity is considerably higher during the rest of the weeks. If you believe you will need individual supervision, we thus strongly suggest that you start working on the labs as soon as possible and take advantage of the assistants availability for questions and advice, while the work load is low. In the course program above you see when the necessary theoretical parts of each respective lab should have been completed. We hope that as many students as possible complete their labs well before the scheduled deadlines. For understandable reasons we are unable to accept more lab reports than the number of scheduled time slots. Reserving additional terminal rooms is practically impossible in the middle of the term.

Bonus points

We apply a bonus system to encourage students to follow the recommendations of the course. For labs reported on time the following bonus points are awarded (5 points in total):

No. labs in time	Bonus points
1	1
2	2
3	2

These points will be added to your score on the exam and will only affect your grade on the regular exam and the first additional exam. When you are logged in on a CSC machine you may see the labs you have so far completed by writing

res show bildat08

NOTE! It may take the assistants a day or two until the results have been reported into the system. As an extra precaution make sure that the assistants sign the back of the lab notes after each completed lab.

CSCs terminal halls

Labs are made for you to gain some practical experience about the basic methods in image processing, image analysis and computer vision and learn how to solve some of the basic problems. Vi have chosen to use Matlab for this purpose since most of the students are familiar with it.

The easiest for the work is to use the machines in CSCs terminal rooms. You are also allowed to use your own computer and it is therefore worth to note that some of the images you will be working with are saved as Matlab binary files. Some of these may not be possible to use on other machines than SUN that are available at CSC.

If you want to transfer files from CSCs system to your own computer you can use sftp command and transfer the files by using ftp.nada.kth.se with your user name at CSCs. Again, the course directory is under

/misc/info/NADA-kurser/datorseende-gk

Do not forget that there are a few subdirectories there!

Account

If you already have a CSC user account, you will use this one throughout the course. Otherwise, you have to contact Delphi (system group) that will give you one (take a valid ID with you).

Course information

All the info and resources you will need for the lab work is under

/info/bildat08

There is a module file

/info/bildat08/module/bildat08

that will be loaded directly you log in once you have joined the course using

course join bildat08

Do this ASAP!

Course directory

There is a course directory under

/info/NADA-kurser/datorseende-gk

that includes following subdirectories:

Functions
A small set of Matlab-functions for handling and displaying of images.

Images-mat
Test images stored in Matlab binary and compact format.

The image in such a file thisimage.mat is stored as a Matlab variable and can be loaded into Matlab as

load thisimage

After loading the image, you may find the name of the variable using e.g. who.

Images-m
Test images stored in a more memory intensive ASCII-format.

The image of type thatimage.m is represented as a Matlab function and can be loaded with e.g.

x = thatimage;

If you happen to forget the ending semi-colon you will easilly remember it for the next time.

Images
Test images in JPEG- and TIFF-format.

The images in a animage.jpg or animage.tiff format, can be loaded with

hund1 = double(imread('animage.jpg'));
hund2 = double(imread('animage.tiff'));

If the search paths $MATLABPATH at CSCs UNIX-system are correctly set to your Matlab system (set by /info/bildat08/module/bildat08 at CSCs computers), you should have access to these functions and images, once you have started up Matlab. Before doing so, you better check whether these paths are included:

/info/NADA-kurser/datorseende-gk/Functions
/info/NADA-kurser/datorseende-gk/Images-m
/info/NADA-kurser/datorseende-gk/Images-mat

NOTE! The course directory and image catalogues may be updated during the course. However, all files related to a particular lab should be up-to-date once the lab has started. Thus, you preferably download the files just before you begin working on each lab.

Presentation requirements

Each lab is graded either as passed or failed. Upon failure you may be requested to complement your experiments and once again present all your results on a later occasion.

During your presentations you should be able to answer questions, general and specific, regarding the methodologies applied, as well as questions necessary for you to reproduce (parts of) the exercises. Preferably, you store all Matlab commands necessary to produce the results in Matlab-scripts (see below).

Prior to a presentation you should write down answers to all the questions posed in the lab notes. This should also include a summary of results and conclusions drawn in a manner that is easy for the lab assistants to understand. These short summaries of each lab serves as complementary tools for later exams.
Be prepared before you try to present your results.

Reservation of time-slots

We use program bok to administrate reservations of time-slots for lab presentations. You make a reservation on CSCs Unix machines using the command

bok new bildat08

and follow the instructions. When you start the program you should see a list of available time-slots. Note that only the first available time-slot can be reserved for each scheduled lab occasion. Once you have made a reservation you can see the reserved time-slot using

bok show bildat08

If you need to cancel a reservation, you may use

bok remove bildat08

Registration

Observe that you have to be registered to the course in order for bok to work. This is done using the command res. If you haven't done so earlier, you register yourself with

res checkin bildat08

and wait until the database in res has been updated. (Note that this may take a day or so. Thus you better register as soon as possible once the course has started!). When the registration is complete, you then have access to bok.

Bonus points

As mentioned during the lectures, you will be awarded bonus points for the exam if you pass a lab before the corresponding deadline. More information on this can be found in the course program and on the webpage.

Images in Matlab

If you are not already familiar with Matlab, you might be interested in ``Användarhandledning till Matlab", that can be bought at CSCs studentexpedition. You may also read through the first sections of Chapter 2 in the ``Matlab's User's Guide'', while trying out some examples on your computer. Note that Matlabhas a built-in help facility based on the command help. Thus it might be possible to use the system based solely on this functionality, given the short introduction found below.

Starting up Matlab

Before you start Matlab for the first time, you should check the search paths. On CSCs Unix machines you do this with

module load /info/bildat08/module/bildat08

Then you can start Matlab by

matlab

Within a few seconds a window with the Matlab logo will be shown, and moments later a window with a welcoming message and a prompt. If not, make sure that all above mentioned instructions have been properly followed. If your environment is too slow, you may prefer to use terminal based interaction. You should then start up Matlab with

matlab -nojvm

Loading and displaying images

Images in Matlab's binary format.

In order to load one of the binary images from the image database of this course, you may e.g. write

load canoe256

With command who you can now see the following variables defined in Matlab

Canoe zmax zmin

Here the variable Canoe includes the image. You may then display the image using the function showgrey, which has specifically been written for this lab course

showgrey(Canoe)

Images in the ASCII-based format.

To load one of the images stored in the ASCII-based text format, you may e.g. write

phone = phonecalc256;

This operation assigns the value of the image to the variable phone, and the semi-colon suppresses an output of the result of this assignment. (As noted this format require considerable more time for an image to be read.) If you wish to see the result display, use

showgrey(phone)

You may test another similar image, using a shorter notation, with

showgrey(nallo128)

As you see the contrasts are somewhat low in the lower part of the image. In the first lab we will try to use different methods to improve the images in order to highlight information that can otherwise be hard to see.

Your own images.

If you have your own images that you like to process, you may use {\sc Matlab}s embedded function imread, that is able to handle a number of different image formats, including tiff and jpeg -- see help imread. You read such an image with the sequence

rawpic = imread('image.tif');
floatpic = double(rawpic);
showgrey(floatpic);

Observe that you need to convert the image with the command double, before you can use the output of imread for lab related functions such as showgrey.

Conversions of image formats can be done in CSCs computer environments using a program called convert, that is able to convert image between a large range of file formats. In order to assign with formats you like to convert between you just apply the suffixes corresponding to the formats at the end of the file names, such as

module add gnome
convert image.gif image.tiff
convert onepage.ps onepage.tiff

You may also use the interactive program xv for image conversions and in order to grab images directly from the screen.

Clean up old images.

If you use a large number of images, there might be a risk that you hit the memory limit, below which Matlab is able to effectively function. In order to clean up old images and data structures, simply write

clear

and then control with who whether all variables have indeed been erased. You may close a Matlab window with

close

or by terminating Matlab with

exit

In the forthcoming text, a short introduction to Matlab will be given for students who are not yet familiar with the system. For an experienced user most of this information might not be of interest. However, also these users might benefit from reading the sections called ``Syntax and computational efficiency'', ``Coordinate systems'' and ``Memory usage''. In order to facilitate presentations of labs you should particularly study the guidelines in section ``Scripts''.

Short introduction to MATLAB

MATLAB is a system that was originally developed for processing of matrices using a compact notation. Since images can be represented as matrices Matlab can in this respect be used for image processing.

However, MATLAB is suitable for operations that can easily be expressed in terms of matrices. As you will notice, this has affected the design of the labs for this course.

Dokumentation

MATLABs embedded help functionality works as follows. If you write

help

you will get a list of a large number of topics, under which all embedded commands (and a short description of some key concepts) have been collected. This list is hierarchically organised. If you then write

help topic

where topic is among the headlines of this comprehensive list, you get a list of all the commands associated to this topic. Finally, if you write

help command

you get specific information regarding a particular command. Some of the commands/topics that will be most relevant to this course are arith, axis, clear, close, colon, color, colormap, diary, figure, for, function, help, if, image, input, keyboard, linspace, load, ones, paren, pause, plot, print, punct, relop, save, script, showgrey, size, slash, title, while, and zeros.
Test it yourself!

Storage in files

Variables in MATLAB can be stored as files using

save filename x y z

which results with a binary representation of the variables x, y och z being written to the file filename.mat. These can be used in a new MATLAB-session by

load filename

Note that the variables stored previously with the same name will be overridden. To check which variables are defined do

who

If you want to know the size of the variables, write

whos

Displaying images

Images may be displayed with image or showgrey (for grey level images). The images that are to be displayed, as well as histograms and other graphical contents, will be drawn in so called figure windows. The first such window is created as soon as it is needed. Additional windows, to display multiple images simultaneously, can be opened with

figure

All graphical contents will be drawn in the currentfigure window. This is usually the last opened figure window, or the window that was activated last, e.g. by clicking onto it, making it the window in the foreground. If you like a particular window, say window number id, to become the current window, just write

figure(id)

Command

close

terminates the current window while

close(id)

terimates window with number id. With the command subplot, a few different illustrations can be displayed in a single figure

help subplot

Command

title(sprintf('Titel: (arg1, arg2) = (%d, %f)', intarg, floatval))

can be used to give an informative title for your figure, including the parameter setting used to generate it.

Printing out

What is shown in a figure window can be printed out onto a laser printer named skrivarnamn using

print -Pprintername

You can also store store files in the POSTSCRIPTformat to a file filename using

print -deps filename

Before you print out the image you might want to complement it with a title or change the coordinate axes. The command

title textsträng

assigns the title string to the current figure. Coordinate axes can be drawn using

axis on

and removed with

axis off

Keeping a diary

In some cases it might be appreciated to keep all input commands and results in a diary file, such that the Matlab-session can later be recreated. This is done by writing

diary filnamn

which makes Matlab copy all inputs and outputs (visible in the command window) to the file filnamn. Once a diary is opened it may be closed or reopened with diary off and diary on.

Scripts

A {\sc Matlab}-script is a file containing a series of Matlab commands. A script-file must have a name similar to scriptname.m. When you type

scriptname

in the Matlab command window (or in another script or function) the series of commands in the script-file will be executed in the same way as if you wrote the same commands directly in the command window (at the prompt). Thus the script may affect the Matlab variables stored in memory.

To speed of the presentation of your labs you ought to create a script such that the most important steps of the labs can easily be reproduced . For the purpose of presentation you might want to interrupt the script at strategical points of execution, such as the partial results can be more closely inspected. This can be done using the commands keyboard, pause and input. In many cases the first command is preferable, since it allows Matlab's command window to be accessed.

In many cases, a practical way of creating a script-file is by first investigating the methods using the command window with a diary activated, manually edit the diary-file once it has been deactivated, and then successively update the script until it can be executed in its entirety. (To prevent interference with some global variables you better reset these before every execution using clear variabelnamn.

Functions

A function in Matlab very much works like a script. The difference is that a function

• uses local variables,
• accepts input arguments,
• returns (one or many) output variables

A function that accepts two different input arguments and returns a single variable has a declaration similar to

function utdata = functionname(indata1, indata2)

while a declaration of a function that returns three variables, without any input arguments, will be written as

function [utdata1, utdata2, utdata3] = functionname()

In the function-file the arguments are treated like ordinary Matlab variables. Results are returned by assigning a value to the output variables (see below).

Search paths

For Matlab to find a particular requested file, such as a file containing Matlab variables, a Matlab script or function, the file has to be placed either in Matlab's current working directory or among the directories included in the search paths.

You may output Matlab's current working directory with the command pwd and change it with cd. You may also, within Matlab, see the current search paths of the system, as well as change these, using the command path . (If you need any help, just use the help-function).

If you create your own library of Matlab functions and scripts, you might want to include these to the search paths. On CSCs Unix computers this can be done by updating the environment variable $MATLABPATH in your .login-file or by creating your own module-file (see /info/bildat08/module/bildat08 for an example of how to create module-files).

Command line editor

MATLAB has a simple command line editor, that makes it possible for you to edit earlier input commands, instead of re-writing everything in the command window. These are some of the (emacs-like) commands of this editor:

ctrl-P	uppåt till föregående kommandorad
ctrl-N	nedåt till nästa kommandorad
ctrl-B	en position bakåt
ctrl-F	en position framåt
ctrl-A	till radens början
ctrl-E	till radens slut
ctrl-D	radera aktuell position
ctrl-K	radera till radens slut

Here ctrl-P means that you at the same time have to press the ctrl and ``p'' buttons. Ongoing calculations and printouts can be terminated by ctrl-C.

Ongoing calculations and outputs can in general be interrupted with $<$ctrl$>$-C. If you are interested in finding a particular command line, such as the line on which the variable thisvar is assigned to value given by an expression, the easiest way is by first typing thisvarand then repeatably pressing ctrl-P. Then the command editor will go through only those lines starting with this combination of characters.

Suppression of outputs

If a command (usually a line) ends with a semi-colon ;, the results are suppressed from being output. This is practical if you are working with large images, that would otherwise fill up your screen with endless rows of numbers. This is also the case for most scripts and functions.

Comments

If a command line (e.g. in a script or a function) is preceded by a percent character, the rest of the information on this line will be ignored. In fact, Matlab's embedded help -function is based on such comment lines. The command

help matlabfile

where matlabfile.m is a script- or function-file, prints out the first unended sequence of comment lines found in this file. You may thus use the help-function also for your own functions and scripts. Just remember that the first line in the function-file needs to include the declaration of the function (see above).

Syntax and computational efficiency

The notation in Matlab was primarily developed for treatment of matrices and vectors using a compact notation. There are a number of common constructions that allows you to avoid explicit loops and thus increase the computational speed. Since Matlab is an interpreting system, you should try to use these constructions whenever it is possible. Some examples:

• Arithmetic sequences are generated using a colon : or linspace, as for example

  x = (1 : 2 : 97)
  x = linspace(1, 97, 48)

• Colons can also be used to extract subsets of vectors and matrices

  x(1 : 5 : 16)

• Constant matrices are easily generated with zeros and ones

  17 * ones(3, 4)

• Multiplications of matrices are written with * and element-wise multiplication with .*

  ones(2 : 3) * ones(3 : 4)

• Matrices and vectors are concatenated using parentheses [ ]. Spaces or commas separate elements within a line, while semi-colons work as separators between lines. An {apostrophe 'denotes a transpose. An example with a matrix created using these operators is

  y = [[ones(1, 2), zeros(1, 2); linspace(1, 2, 4); 1 2 3, 4]' ...
       17*linspace(3, 81, 4)' * [0.1 0.2]]
  

As you see, like in most other languages, you may choose a notation that is more or less readable for the less experienced user. Thus, choose a programming style that is easy to read!

If you still have to write an explicit loop, you should try to avoid making your function too slow, by allocating all necessary memory outside the loop, instead of inside it. For example, compare the following two constructions

clear x;
for n = 1:10000
  x(n) = n*n;
end

for n = 1:10000
  x(n) = n*n;
end

In the first loop considerably more CPU time will be needed than in the last one, since the vector x is reallocated in every iteration in the first case, while it already is of sufficient length when the second loop is executed.

None of the above solutions are satisfactory, of course. Assignment like to those done above should in Matlab be done with the embedded functionality:

tmp = (1 : 1 : 10000);
x = tmp .* tmp;

Coordinate systems

In image processing and computer vision there are a number of different conventions for coordinate systems. During the seminars, as well as in the book of Sonka, we use a Cartesian coordinate system with a horizontal right-sided $x$-axis and a $y$-axis vertically pointing straight up. However, the coordinate system in the book of Gonzalez \& Woods and in Matlab is rotated 90 degrees in relation to the system we use, i.e. the first coordinate axis points downwards and the second one horizontally to the right.

How matrices are defined and accessed in Matlab will be highlighted by a simple example. This example shows how the ASCII-based image format is defined as a function that returns an image matrix as output.

function pixels = matlabcoordimage()

% Test image with imax = 3, jmax = 4 and pixel values
% pixels(i, j) = 10*i + j
% where i and j are Matlab indeces as they
% are used for accessing matrix elements

pixels = [11 12 13 14; ...
          21 22 23 24; ...
          31 32 33 34];

To create a corresponding image in \sc Matlab where the values instead reflect the image points in Cartesian coordinates, you write:

function pixels = cartesiancoordimage()

% Test image with xmax = 4, ymax = 3 and pixel values
% f(x, y) = 10*x + y
% where x and y are Cartesian coordinates
% as they appear on the screen

pixels = [13 23 33 43; ...
          12 22 32 42; ...
          11 21 31 41];

Note that this increases the risk of confusion. Furthermore, in some programming environments vectors are accessed starting with the index zero, while the first value of a vector in Matlab always has the index one. This is another degree of freedom that might complicate the situation. If you happen to see a problem, take a few minutes and think again!

Memory usage

Image information requires large amounts of memory, both for external storage on disk as well as in the computer's internal memory. Thus when you need to debug your algorithms, the work is considerably simplified, if you initially use relatively small test images.

Try to avoid storing large numbers of images (Matlab-variables, POSTSCRIPT-images or images in similar formats) directly onto your account. Most graphs and images can be regenerated with relative ease and can thus be efficiently ``stored'' in terms of the {\sc Matlab}-scripts that originally generated them.

If you use a large number of images in Matlab, the Matlab-process might sooner or later be very large. To release the memory of an allocated variable x, you may type clear x. Doing this on a regular basis will considerably lower the amount of memory needed by the Matlab-process, which on some machines may significantly improve also the computational speed.