Author Archives: admin

Research – Haar Cascades

Posted on by

A Haar Cascade is an object detection algorithm.

image courtesy of: https://docs.opencv.org

It uses a classifier which is trained to detect an object and does this by superimposing a positive image over a set of negative images over a number of stages. Quite a large number of stages.

This machine learning approach is computationally heavy and reliant upon possession of a training set. However there seem to be a number of them available online. Each with their own focus of object detection.

 

Haar-cascade Detection in OpenCV

OpenCV has it’s own trainer and detector which can be used to train your own classifier, which in our case would need to track the left eye of a person over a series of three images.

By applying some demo code kindly provided by OpenCV to one of our project images, we can see that this could well be a viable approach to take with our project and worthy of further research.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
import numpy as np
import cv2
 
face_cascade = cv2.CascadeClassifier('haarcascade_frontalface_default.xml')
eye_cascade = cv2.CascadeClassifier('haarcascade_eye.xml')
 
img = cv2.imread('Ilovecats1.bmp')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
 
faces = face_cascade.detectMultiScale(gray, 1.3, 5)
for (x,y,w,h) in faces:
    cv2.rectangle(img,(x,y),(x+w,y+h),(255,0,0),2)
    roi_gray = gray[y:y+h, x:x+w]
    roi_color = img[y:y+h, x:x+w]
    eyes = eye_cascade.detectMultiScale(roi_gray)
    for (ex,ey,ew,eh) in eyes:
        cv2.rectangle(roi_color,(ex,ey),(ex+ew,ey+eh),(0,255,0),2)
 
cv2.imshow('img',img)
cv2.waitKey(0)
# cv2.destroyAllWindows()

Similar to HAAR Cascades are LBP (Local Binary Pattern) Cascades which seem to operate similarly in the respect that they both use training sets. However, LBP Cascades can provide quicker results with slightly less accuracy.

Local Binary Patterns could be thought of as a computationally cheaper version of the Haar Cascade. LBP works by thresholding with the use of a 3×3 grid. The value of the centre pixel in the grid is used as a threshold value and the surrounding pixels in the grid are thresholded and summed together to form the descriptor used. LBP yields results that would be similar to Haar Cascades while offering a much simpler process.

References:

Face Detection using Haar Cascades (accessed 25.10 2018)
https://docs.opencv.org/3.0-beta/doc/py_tutorials/py_objdetect/py_face_detection/py_face_detection.html#face-detection

Viola P. Jones MJ. Rapid Object Detection using a Boosted Cascade of Simple Features. IEEE Computer Society Conference on Computer Vision and Pattern Recognition 1:I-511- I-518 vol.1
https://www.researchgate.net/publication/3940582_Rapid_Object_Detection_using_a_Boosted_Cascade_of_Simple_Features

Deep Learning Haar Cascaded Explained (accessed 25.10 2018)
http://www.willberger.org/cascade-haar-explained/

Pietikäinen M. Local Binary Patterns. Machine Vision Group, Department of Electrical and Information Engineering, University of Oulu, Finland.
http://www.scholarpedia.org/article/Local_Binary_Patterns

Huang D. Shan C. Ardebilian M. Wang Y. Chen L. Local Binary Patterns and Its Application to Facial Image Analysis: A Survey
https://liris.cnrs.fr/Documents/Liris-5004.pdf

Week 6 – Lab Work

Posted on by

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 1. Open image Googly.jpg;
 
# 2. Use thresholding to separate Googly and his friend from the background;
 
# 3. Use statistical analysis to choose a threshold for better results;
 
# 4. Try multiple thresholds on different channels.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("googly.jpg", cv2.IMREAD_GRAYSCALE)
 
# I = cv2.imread("image.jpg")
 
# RGB = cv2.cvtColor(I, cv2.COLOR_BGR2RGB)
 
T = 100
T, B = cv2.threshold(I, thresh = T, maxval = 255,
type = cv2.THRESH_BINARY)
 
cv2.imshow("Googly", B)
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 1. Open image sudoku.jpg
 
# 2. Use adaptive thresholding to create a black & white image;
 
# 3. Modify the region size and constant for better results.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("sudoku.jpg", cv2.IMREAD_GRAYSCALE)
 
J = cv2.adaptiveThreshold(I, maxValue = 255,
  adaptiveMethod = cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
  thresholdType = cv2.THRESH_BINARY,
  blockSize = 5, C = 15)
  
cv2.imshow("Sudoku", J)
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
# 1. Open image Orange.png;
 
# 2. Using thresholding, create a mask with the orange as ROI;
 
# 3. Use this mask to extract the orange from the image;
 
# 4. Open Water.jpg;
 
# 5. Use the inverse of the orange mask to cut an orange-shaped hole in
# the water picture;
 
# 6. Combine the orange and water masked images to create a
# composite image.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("orange.png")
Z = cv2.imread("water.jpg")
G = cv2.cvtColor(I, cv2.COLOR_BGR2GRAY)
 
J = cv2.bitwise_not(G)
 
T = 20
T, B = cv2.threshold(J, thresh = T, maxval = 255,
type = cv2.THRESH_BINARY)
 
K = cv2.bitwise_not(B)
 
ROI = cv2.bitwise_and(I,I,mask=B)
 
ROI2 = cv2.bitwise_and(Z,Z,mask=K)
 
ROI3 = cv2.bitwise_or(ROI,ROI2)
 
cv2.imshow("Composite", ROI3)
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# 1. Open Trump.jpg;
 
# 2. Use an appropriate colour space and range to segment out areas of
# flesh (you will need to research this);
 
# 3. Use morphological processes to clean up the ROI.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("trumpcopy.jpg")
 
HSV = cv2.cvtColor(I, cv2.COLOR_BGR2HSV)
 
# A Numpy array is created of upper and lower fleshtone values.
lowerFleshValue = np.array([3, 48, 80])
upperFleshValue = np.array([20, 255, 255])
# Mask is created of all fleshtone values.
fleshMask = cv2.inRange(HSV, lowerFleshValue, upperFleshValue)
 
# Structuring element
shape = cv2.getStructuringElement(cv2.MORPH_RECT,(6,6))
 
# Closing
# newMask = cv2.dilate(fleshMask,shape)
# newerMask = cv2.erode(newMask,shape)
 
# Opening
# newestMask = cv2.erode(newerMask,shape)
# mostNewestMask = cv2.dilate(newestMask,shape)
 
newMask = cv2.morphologyEx(fleshMask,cv2.MORPH_CLOSE,shape)
newerMask = cv2.morphologyEx(newMask,cv2.MORPH_OPEN,shape)
boundary = cv2.morphologyEx(newerMask,cv2.MORPH_GRADIENT,shape)
 
C = cv2.bitwise_and(I,I,mask=newerMask)
 
cv2.imshow("Original", I)
cv2.imshow("HSV", HSV)
cv2.imshow("Mask", fleshMask)
cv2.imshow("Closing", newMask)
cv2.imshow("Opening", newerMask)
cv2.imshow("Boundary", boundary)
cv2.imshow("Combined", C)
key = cv2.waitKey(0)

 

Assignement 1 – Lighting

Posted on by

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
# To accomplish this task I did the following:
 
# Stage 1. I converted the image to grayscale.
 
# Stage 2. I then applied Contrast Limited Adaptive Histogram  
# Equalization to try and correct the uneven lighting on the page.
 
# Stage 3. I then applied an Adaptive Threshold which turned
# the information on the page black or white and therefore
# getting rid of a lot of detail making the text unreadable.
# So I ended up using the threshold just to assist canny find
# the edges of the ROI.
 
# Stage 4. I used the opencv'canny' function to find the edges of my
# ROI (Region of Interest)
 
# Stage 5. I used the numpy 'argwhere' function to crop the image
# using the edges located by the 'canny' function.
 
# Final stage. I output a copy of the corrected image and display
# the corrected image on screen.
 
# Summary: I was hoping that my result would be a crisp white
# background with clear black text in the foreground, but as the
# priority was to make the text easier to read I had to make a
# call with that in mind, so in terms of contrast, I think my
# program has provided results somewhere in the middle, but
# the text is definitely more readable.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
stage1 = cv2.imread("text.jpg", cv2.IMREAD_GRAYSCALE)
 
stage2a = cv2.createCLAHE(clipLimit = 2.0, tileGridSize = (8, 8))
stage2b = stage2a.apply(stage1)
 
stage3 = cv2.adaptiveThreshold(stage2b, maxValue = 255,
  adaptiveMethod = cv2.ADAPTIVE_THRESH_GAUSSIAN_C,
  thresholdType = cv2.THRESH_BINARY,
  blockSize = 5, C = 15)
 
height = np.size(stage3, 0)
width = np.size(stage3, 1)
 
stage4 = cv2.Canny(stage3, height, width)
 
stage5a = stage4 > 0
 
stage5b = np.argwhere(stage5a)
 
horiz0, vertic0 = stage5b.min(axis = 0)
horiz1, vertic1 = stage5b.max(axis = 0) + 1
 
stage5d = stage2b[horiz0: horiz1, vertic0: vertic1]
 
cv2.imwrite('Corrected_image.jpg', stage5d)
 
cv2.imshow("Corrected Image", stage5d)
key = cv2.waitKey(0)

 

Week 5 – Lab Work

Posted on by

Today we explored kernels in more depth.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# 1. Open any image;
 
# 2. Build a kernel for sharpening an image
# (you will need to research this);
 
# 3. Apply this filter to the image.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("Lenna.png")
 
G = cv2.cvtColor(I, cv2.COLOR_BGR2GRAY)
 
k = np.array([[0,-1,0], [-1,5,-1], [0,-1,0]],
dtype=float)
 
F = cv2.filter2D(G,ddepth=-1,kernel=k)
 
cv2.imshow("image", F)
key = cv2.waitKey(0)

The second part of this lab was used to finish off our first modules .assignment.

Project Research – Preliminary Stage

Posted on by

To succeed in the aims of our project to track an item within an image across a series of similar images we are going to need to explore the technology within four potential ways of tackling this challenge before deciding on any particular approach.

Tracking motion – Looking for things that are different between images, if it doesn’t move and the pixel value hasn’t changed we are not interested, we are only interested in what has changed.

Tracking texture – The gradient orientation that occurs in the localized portions of an image is counted in texture based technique. It is then computed using overlapping local contrast normalization on a dense grid of uniformly spaced cells for improving accuracy.

Shape based – Simple pattern matching can be applied with appropriate templates.

Colour based – Creates a gaussian mixture model to describe the colour disrtibution within the sequence of images and to segment the images into background and objects.

References:

Klimek RB. Wright TW. Sielken RS. Color Image Processing and Object
Tracking System. Lewis Research Center, Cleveland, Ohio. Trinity University, San Antonio, Texas .
https://ntrs.nasa.gov/archive/nasa/casi.ntrs.nasa.gov/19960016954.pdf

Yazdi M. Bouwmans T. New Trends on Moving Object Detection in Video
Images Captured by a moving Camera: A Survey. HAL Archives
https://hal.archives-ouvertes.fr/hal-01724322/document

Object tracking & Motion detection in video sequences (accessed 16.10.2018)
http://sccg.sk/~cernekova/

Balaji S RDr. S. Karthikeyan. A survey on moving object tracking using image processing. Conference: 2017 11 th International Conference on Intelligent Systems and Control (ISCO)
https://www.researchgate.net/publication/315711962_A_survey_on_moving_object_tracking_using_image_processing

Pramod Patil SP. Techniques and Methods for Detection and Tracking of Moving Object in a Video . International Journal of Innovative Research in Computer and Communication Engineering. Vol. 4, Issue 5, May 2016
http://www.rroij.com/open-access/techniques-and-methods-for-detection-and-tracking-f-moving-object-in-a-video-.pdf

The First Computer I used

Posted on by

Today in class I was asked, what was the first computer I used and I had to think waaaaay back. It also took a while.

Was it the Aston 3 I though! A primitive beast that looked like it had been made in somebody’s shed and generated TV captions. It stored everything on a fearsome looking 7″ floppy disc that could be erased by a change of wind direction.

 

Or was it the Abekas A72, another Caption Generator with a sleek futuristic keyboard and half a rack of hardware to support its amazing ant-aliasing powers.

 

Or could it have been the CMX Edit control system, driven by Unix command prompt with GPI’s for external FX units such as Vision Mixers and DVE’s.

 

Or maybe it the Quantel Paintbox, which was the forerunner to Adobe Photoshop but was definitely not as portable.

 

No it was in fact the Amiga 500.

This was a popular gaming machine in its day but we used it for generating info trivia boxes for the 1980’s Channel 4/ITV music Show ‘The Chart Show’. An illegible example of this can be seen at around 2 minutes on this clip from the show.

Week 4 – Lab Material

Posted on by

Transformation along with more image adjustments such as cropping, rotating, scaling. We also covered image Math such as adding, dividing, subtraction and multiplication. Finally we had in introduction to Kernels, yikes…

Code for the lab tasks:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# For finding the size of an image
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
# I = cv2.imread("orange.png")
J = cv2.imread("water.jpg")
 
# Getting the size of the image:
size = np.shape(J)
# size = np.shape(J)
 
print size
 
# orange.png 360L, 630L, 3L
# water.jpg 360L, 630L, 3L

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
1. Open image "Wartime.jpg";
 
2. View its histogram alongside the image;
 
3. Use histogram equalisation to improve its contrast;
 
4. View the new histogram alongside the new image.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("wartime.jpg")
G = cv2.cvtColor(I, cv2.COLOR_BGR2GRAY)
 
plt.figure()
plt.subplot(2,2,1)
 
plt.imshow(I)
plt.subplot (2,2,3)
H =cv2.equalizeHist(G)
plt.imshow(H)
 
 
Values = G.ravel()
plt.subplot(2,2,2)
plt.hist(Values,bins =256,range=[0,256]);
Values = H.ravel()
plt.subplot(2,2,4)
plt.hist(Values,bins =256,range=[0,256]);
plt.show()

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
# 1. Open any image;
 
# 2. Crop out the fourth quadrant of the image;
 
# 3. Rotate this cropped section by 45 degrees
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("colors.jpg")
 
C = I[90:180,135:270]
 
M = cv2.getRotationMatrix2D(center=(150,5),
angle=45, scale=1)
R = cv2.warpAffine(C, M = M, dsize=(270,180))
 
cv2.imshow("image", R)
 
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
# 1. Open images Orange and Water;
# 2. Scale each image to 50 % by multiplying each by 0.5;
# 3. Add the two scaled images to get a composite image;
# 4. Adjust the scaling to give a nicer output.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("orange.png")
J = cv2.imread("water.jpg")
 
# doesn't accept float so couldn't mulitple by 0.5
h, w, d = I.shape
S = cv2.resize(I, dsize=(2*w, 2*h))
 
# doesn't accept float so couldn't mulitple by 0.5
h, w, d = J.shape
T = cv2.resize(J, dsize=(2*w, 2*h))
 
M = cv2.add(S,T)
 
cv2.imshow("M", M)
cv2.imshow("S", S)
cv2.imshow("T", T)
key = cv2.waitKey(0)

 

Week 3 – Drawing on images & Thresholding

Posted on by

This week we performed further image manipulation in the form of adding, circles and lines and borders on images loaded into OpenCV. We also looked into Threholding and were introduced to the Histogram in relation to image Processing.

Code for the lab tasks.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# 1. Open a user-selected image;
 
# 2. Show this image on the screen;
 
# 3. Capture the user’s click on the image;
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("colors.jpg")
 
Original = I.copy()
 
def draw(event,x,y,flags,param):
if event == cv2.EVENT_LBUTTONDOWN:
cv2.circle(img = I, center = (x,y),radius = 5, color = (255,255,255), thickness = -1)
cv2.imshow("image", I)
cv2.namedWindow("image")
cv2.setMouseCallback("image", draw)
cv2.imshow("image", I)
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
# 4. Draw a 201 x 201, 5-pixel thick red square around this location;
 
# 5. Convert the pixels within the square to YUV.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("colors.jpg")
 
YUV = cv2.cvtColor(I, cv2.COLOR_BGR2YUV)
 
cv2.rectangle(img = I, pt1 = (4,4), pt2 = (270,177), color = (0,0,255), thickness = 10)
 
cv2.imshow("image", I)
key = cv2.waitKey(0)

 

The Project – Initial thoughts

Posted on by

Our project is describes as follows:

‘Find the left eye in the first image and track it through the image sequence’

The image sequence described contains a person talking about cats (I think) but the solution required needs to be generally applicable.

My initial thoughts are two create a program that can isolate a unique colour in conjunction with recognising patterns within the detail. of the image but I am sure there are many potential approaches that can be taken. So…

 

 

 

 

 

 

 

After research more specifics will follow…