Escolar Documentos
Profissional Documentos
Cultura Documentos
. Candidate Department of Electrical and Computer Engineering Distributed Multimedia Computing Research Lab EPH 408 rphan@ee.ryerson.ca
Some slides were taken from Prof. R. A. Peters PPT slides: http://www.archive.org/details/Lectures_on_Image_Processing
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
http://www.ee.ryerson.ca/~rphan/ele532/MATLABTutorial.ppt
http://www.rnet.ryerson.ca/~rphan/IEEEDIPTalk
Introduction of Myself
Started in 2002 in the B.Eng. Computer Engineering
program
Started my M.A.Sc. in 2006 ELCE Finished in 2008 Winner Ryerson Gold Medal (2008 SGS) Started my Ph.D. in 2008 ELCE Will finish before my back gives out Currently a 4th year Ph.D. Candidate 2010 NSERC Vanier Canada Graduate Scholar
System
Featured in the Toronto Star (January 4th, 2008) Searching for images using actual images, rather than
keywords
3D conversion
What is MATLAB?
MATLAB Stands for MATrix LABoratory Created by Cleve Moler @ Stanford U. in 1970 Why? Makes linear algebra, numerical analysis and optimization a lot easier MATLAB is a dynamically typed language Means that you do not have to declare any variables All you need to do is initialize them and they are created MATLAB treats all variables as matrices Scalar 1 x 1 matrix. Vector 1 x N or N x 1 matrix Why? Makes calculations a lot faster (will see later)
2) Use any Ryerson computer on to the ACS network Log in with your Matrix ID and Password, then go to Start MATLAB R2010b 3) Install it on your own laptop Go to http://www.ee.ryerson.ca/matlab for more details You must be on the Ryerson network to sign up for an account After, you can download MATLAB from anywhere
MATLAB uses the Image Processing Toolbox (IPT) Should already be installed with MATLAB!
Color images have 3 values per pixel; monochrome / grayscale images = 1 value/pixel.
Pixels
Monochromatic Case: We call the values at each pixel intensities Smaller intensities denote a darker pixel Bigger intensities denote a lighter pixel Colour Case: Think of a colour image as a 3D matrix First layer is red, second layer is green, third layer is blue Why RGB? Trichromacy theory All colours found in nature can naturally be decomposed into Red, Green and Blue This is basically how CCD cameras work! The three element vector tells you how much red, green and blue the pixel is compromised of (i.e. [R G B]T = [0 255 0] No red, no blue, all green
a rectangular grid, and grab the colour in each location of the grid
row index
real image
sampled
quantized
of possible colours
We will now need to quantize the colours Quantizing will reduce the total number of colours to a
smaller amount Key Quantize accurately so that we cant tell much difference between the original image and the quantized one
row index
real image
sampled
quantized
discrete and positive values Usually, digital images need 8 bits for B & W and 24bits for colour (8 bits for each primary colour)
B & W 0 for Black and 255 for White All integers
Colour 0 to 255 for Red, Green and Blue All integers Note: We can consider a colour image as three 2D images
Without compression, files would be very large! Compression algorithms (PNG, JPEG, etc.) eliminate
im = imread(name_of_image.ext) Use single quotes, and type in the full name of the image
with its extension (.bmp, .jpg, etc.) im will contain a 2D matrix (rows x cols) of B&W values or a 3D matrix (rows x cols x 3) of colour values Matrix corresponds to each pixel in the digital image for B & W, or a colour component of a pixel in colour
row & col: Row & column of the pixel to access pix contains the intensity value Access elements in an array by round braces, not square!
How do I access a pixel in MATLAB Colour case? pix = im(row,col,1); Red colour value pix = im(row,col,2); Green colour value pix = im(row,col,3); Blue colour value 3rd argument 3rd dimension of matrix Only grabs one colour value at a time!
command
pix = im(row,col,:); : means to grab all values of one dimension However, this will give you a 1 x 1 x 3 matrix we just
want an array! Call the squeeze() command pix = squeeze(im(row,col,:)); Now a 3 x 1 vector. To access R, G and B values, do: red = pix(1) Red, gr = pix(2) Green, blue = pix(3) Blue
the image?
im(row,col) = pix; For a colour image, do either: im(row,col,1) = red; im(row,col,2) = green; im(row,col,3) = blue; or im(row,col,:) = [red; green; blue] or im(row,col,:) = rgb; %rgb - 3 x 1 vector
im2 = im(row1:row2,col1:col2); Do the following for colour images: im2 = im(row1:row2,col1:col2,:); This will grab a rectangular region between rows 1 and 2, and columns 1 and 2 e.g., if I wanted to get rows 17 31, and columns 32 45 for colour, do: im2 = im(17:31,32:45,:);
Resizing Images
One common thing that many people do is resize
images
imresize command How do we use it? out = imresize(im, scale, method); or out = imresize(im, [r c], method); For both methods im is the image we want to resize, and out is the resized image
out = imresize(im, scale, method); scale takes each of the dimensions of the image (# of rows and columns), and multiplies by this much to determine the output
e.g. If we have an image that is 20 rows x 40 columns: If scale = 2, the output 40 rows x 80 columns If scale = 0.5, the output 10 rows x 20 columns
create an image with a lack of information present There are three main types of interpolation
Nearest Neighbour method = nearest
Uses the best pixels that are near the original pixels and fills in missing information Uses linear interpolation in 2D to fill in missing information Uses cubic interpolation in 2D to fill in missing information
out = imresize(im, [r c], method); This routine will resize the image to any desired dimensions you want
You can customize how many rows and columns the
65 rows x 90 columns, with bilinear interpolation, do: out = imresize(im, [65 90], bilinear);
We can also do!
Rotating Images
out = imrotate(im, angle, method); im: The image we want to rotate angle: How much we want to rotate the image
Angle is in degrees! Positive angle denotes counter-
information method is like before with resizing out: The rotated image Example: Lets rotate CCW by 45 degrees by bilinear: out = imrotate(im, 45, bilinear);
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
Takes c and add / subtract to every pixel in the image Adding / subtracting makes the image brighter / darker Round off occurs if out of range (i.e. set to 255 if > 255/set to 0 if < 0)
What about another way? We can also scale the image by a constant Do: im2 = c*im;
background When performing brightness enhancing, youll notice that it looks white washed out Poor contrast We can do a contrast enhancement to make objects look better, leaving background relatively unaffected How? Use the power law s = r
r is the input pixel intensity / colour and s is the output
intensity / colour
intensities / colours
For colour images,
Exponent determines
out = imadjust(im, [], [], gamma); im: Input image to contrast adjust Ignore 2nd and 3rd parameters Beyond the scope of our talk gamma: The exponent that weve seen earlier out: The contrast adjusted image Example use: If = 1.4, we do: out = imadjust(im, [], [], 1.4);
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
using histograms Before we cover this we should probably cover what histograms are! So, whats a histogram?
It measures the frequency, or how often, something
occurs Lets look at a grayscale image for now Expressed as H(x) = q, x is an intensity- [0,255] for 8 bits This tells us that we see the intensity value of x for a total of q times
There are ~1500 pixels with a gray level of around 10 There are ~1200 pixels with a gray of around 170, etc.
grayscale image, which can be used for imhist We can also call the histogram function by: h = imhist(im); Will create a 256 element array, where the (i+1)th element contains how often we see the grayscale i. Histograms give us a good insight on image contrast
If the histogram has too many values towards the left
Image is too dark If the histogram has too many values towards the right Image is too bright If the histogram has too many values in the middle Image looks very washed out
for good contrast Stretches the histogram out For you probability nerds If you divide each histogram entry by the total number of entries, this forms a Probability Density Function (PDF)
out = histeq(im); %im given by imread What about for colour images?
You will have to perform histogram equalization on each
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
Intro to Convolution
Before we proceed, we need to understand what
Probably learned in ELE/BME 532, ELE 792, etc. Bleck! But! Actually very simple when dealing with 2D images
Preliminaries: First, we need to create an N x N matrix called a mask The numbers inside the mask will help us control the kind of operation were doing Different #s allow us to blur, sharpen, find edges, etc. We need to master convolution first, and the rest is easy!
(r,c) = (6,4)
Pixels are a, b,
g, and centre is e
3) Create an output image and: a) Add up all of these values b) Store the output at (r,c) (i.e. the same row and column location as the input image) in the output image Example:
out = a*z + b*y + c*x + d*w + e*v + f*u + g*t + h*s + i*r, then store out into (r,c) of the output image
Each pixel in the output image is this weighted sum Do this for each location of the input image and assign to same location in the output image What about for colour!? Do convolution on each channel separately Note: If the output value is floating point (decimal),
we must round in order to make this an 8-bit number Heres one more example to be sure you understand
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
RGB values after the blurring of each colour layer The bigger the neighbourhood, the more the blurring How can we efficiently implement this?
3 x 3 Averaging Mask
5 x 5 Averaging Mask
9 x 9 Averaging Mask
out = (1)(1/9) + (2)(1/9) + (3)(1/9) + (4)(1/9) + (5)(1/9) + (6)(1/9) + (7)(1/9) + (8)(1/9) + (9)(1/9) out = (1/9)(1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9) out = (1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + 9) / 9 isnt this exactly the same thing as averaging!? You are essentially adding up all of the pixels in the neighbourhood, and dividing them by the total number of pixels (9) Note: If we get a decimal number, we round to be sure that we have an integer number
out = imfilter(im, mask); out: output image (this case averaged output) im contains the image we want to blur mask: Convolution mask to use (this case averaging)
Why are masks important? You change the size of the mask, and values in the mask and you get different results! Essentially how most filters in Adobe Photoshop and GIMP work
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
Edge Detection
Next? Edge detection what is an edge? Edges are sudden discontinuities appearing in images A.K.A. sudden jumps in intensity or colour in images How can we detect edges? Very simple: Think Calc. Find the derivative of each point in the image When slope is very high, that means you found an edge But, derivative is only for 1D signals what about 2D!? You must take the gradient Derivative in both the x
and y directions You combine both of these directions to create your final derivative function
with a different mask Corresponds to discrete approximation of the derivative Two possible masks we can use:
1 1 1 0 0 0 1 1 1 2 1 0 0 1 2 1 0 1
Prewitt 3 x3 Mask Sobel 3 x 3 Mask These masks detect changes in the horizontal direction If there are no changes, then weighted sum of the pixels in 3 x 3 neighbourhood should have same value Gradient = 0
gradient information?
each term, and do the same for the vertical gradient image Now, add both images, and square root them This is our output image
Values greater than a threshold is an edge (white) Values less than the threshold are not edges (black)
I = edge(im, sobel); or I = edge(im, prewitt); So, we can pretty much ignore the previous slide, but I put that in there so you can see where it comes from
How do we use the thresh variable? Choose an intensity (e.g. 128) Any gradient value > 128 will be labeled white
images, convert the colour image to a B & W image Use gray = rgb2gray(im); rgb2gray converts a colour image into B & W by doing: I = (R + G + B) / 3 Each colour pixel is the average of the red, green and blue components
Unsharp Masking Probably heard this terminology in CSI? So whats unsharp masking? Lets go back to Signals and Systems
Get a blurred version If we take the original image, and subtract its blurred version, what are we doing? Removing all low frequency components, so whats left? High frequency!
Simple contrast and brightness enhancing Advanced enhancing using image histograms Intro to convolution in images Blurring / Smoothing images Edge detection Sharpening images break for 10 minutes!
Image Sharpening
Last image enhancing topic Image Sharpening What is image sharpening? Make the image look
clearer, sharper, make the details pop out more How do we do this?
Edges are the detail behind the image Edges are high frequency
How do we make images sharper? Find overall magnitude of the gradient for the image Add these values on top of the original image Result? Increase the visibility of the edges Sharper
where when you convolve, it will automatically subtract the image with a blurred version of itself and add the original image on top Good for both x and y direction changes How? mask = fspecial(unsharp); sharp = imfilter(im, mask); This will perform the detection of abrupt changes, and adding them on top of the image all in one go.
things we want) from the background Segmentation To make this easier, we convert to a B & W image first if the image is in colour. Else just leave it alone After conversion, most of the time the intensities of the objects are distinctly different from background We can write code to save the pixels that match these intensities, and disregard the rest Pixels that match Set to white, else set to black Output image will be binary White belonging to object, and black belonging to background
We can use this to mask out the background pixels
Noise Filtering
A common task in image processing is to eliminate or
reduce image noise from an image Image Noise: Pixels in an image that are corrupted undesirably
in transmitting, etc.
How do we reduce image noise? Think in the frequency domain Noise is essentially high-frequency information If we blur the image, we would eliminate the noise, but the quality would reduce blurring details
How do we generate artificial noise? Use imnoise We will be concerned with two ways of generating noise
out = imnoise(im, gaussian, mean, var); out = imnoise(im, salt & pepper, prob); First method generates Gaussian-based noise
Usually encountered in transmitting process
subset of pixels centered at (r,c) Sort these pixels in ascending order, and grab the median value
The output image at (r,c) is this value
Template Matching
Template matching is using a small test image, which
this patch is in the entire image Useful in a variety of applications: Image Retrieval, Eye Detection, etc.
stitching
been taken at slightly different perspectives Only horizontal shifting is concentrated on here
C = normxcorr2(template, im);
encapsulate both images together 5) Place one image on the left, then place the other image by displacing it over by the horizontal and vertical shift
Conclusion
MATLAB is a great tool for digital image processing Very easy to use This is not an exhaustive tutorial! There are many
more things you can do with MATLAB For more image processing demos, check out:
http://www.mathworks.com/products/image/demos.html Lots of cool image processing stuff you can find here
http://www.ee.ryerson.ca/~rphan/ele532/MATLABTutorial.ppt
For a more comprehensive MATLAB tutorial, check: You can access the slides, images and code at:
http://www.rnet.ryerson.ca/~rphan/IEEEDIPTalk