Monthly Archives: November 2018

Implementation – Final Code

Posted on by

In the final implementation of our project we decided to create a menu within our program offering the user a choice of all of the approaches we took towards realising our image tracking objective.

Options 1 & 2 Template Matching and Camshift are my contribution to the project and options 3 & 4 Gabor Filter and Colour Isolation are the work of my project partner Modestas Jakuska.

This is the final code for out project and indeed the final post in this blog. Thank you for reading this and I hope it has been interesting.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
# Gavin Morrison & Modestas Jakuska - Image processing Project
 
# Project objective: Find the left eye in the first image and track it through the image sequence.
 
# Methods used:
 
# 1. Template Matching.
# 2. Camshift
# 3. Gabor Filter
# 4. Colour Isolation
 
# Template Matching:
# Author: Gavin Morrison
# How it works: The object required within the image is found
# by comparing a smilar image.
 
# 1. Import image to be treated
# 2. Imported image converted to Grayscale
# 3. Import template image
# 4. Store width and height coordinates of template
# 5. OpenCV has a number of methods for template matching, I have chosen two which I have put into a list
# 6. Loop to implement the method
# 7. Perform match operation
# 8. Draw a rectangle around the matched area
# 9. Display the results
 
# Camshift:
# Author: Gavin Morrison
# How it works: Detects the density of a set of points provided by a back projection within the image being tracked, while checkig ize and rotation.
 
# 1. Import image to be treated
# 2. Copy of image to isolate region of interest
# 3. Coordinates of region of interest [row:row, column:column]
# 4. The region of interest is converted to HSV colour space
# 5. Create histogram of region of interest, using hue. The hue  range is from 0 to 179.
# 6. Original image is converted to HSV colour space
# 7. Back projection used to create mask from the hue of the region of interest histogram.
# 8. Filter applied in an attempt to reduce noise in the mask (later addition)
# 9. The coordinates of our region of interest are assigned to variables.
# 10. Define criteria
# 11. Rectangle of tracking area is created using camshift function
# 12. These are the points for the rectangle
# 13. cv2.polylines used instead of 'cv2.rectangle' to accommodate rotation of bound space.
# 14. Display the processed image
 
# Gabor Filter:
# Author: Modestas Jakuska
 
# 1. Import image to be treated
# 2. Convert image to RGB
# 3. Convert image to Grayscale
# 4. Get 90 and 0 degree Gabor Images
# 5. Create a mask
# 6. Simplify image
# 7. Display the processed image
 
# Colour Isolation:
# Author: Modestas Jakuska
 
# Title:  Skin Segmentation using YCrCb color range
# How it works:
# Convert image to YCbCr.
# Go through the image array and turn non-skin pixels black.
# Skin pixels are determined by their Cr and Cb values.
 
# 1. Select image
# 2. Convert BGR to RGB and then to YCrCb
# 3. Skin values taken from this paper:
# "Comparative Study of Skin Color Detection and Segmentation in HSV and YCbCr Color Space" by Khamar Basha Shaika, Ganesan P, V.Kalist, B.S.Sathish , J.Merlin Mary Jenitha
# They suggest using this range:
# 150 < Cr < 200 and 100 < Cb < 15
# 4. Create skinRegion, a binary image containing the skin region
# 5. Then superimpose the skin region onto the original image
# so that we can see the skin region with colour
# 7. Display the processed image
 
import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt
import easygui
 
# Menu code written by Gavin Morrison D12124782
 
def main_menu():
print("\nPlease select your Tracking method from the menu:\n")
print("1. Template Matching")
print("2. Camshift")
print("3. Gabor Filter")
print("4. Colour Isolation")
print("5. Exit")
while True:
try:
selection=int(input("\nPlease enter your choice...  "))
if selection==1:
template_matching()
break
elif selection==2:
camshift()
break
elif selection==3:
gabor_filter()
break
elif selection==4:
colour_isolation()
break
elif selection==5:
break
else:
print("\nInvalid choice. Please enter 1-5")
main_menu()
except:
print("\nInvalid choice. Please enter 1-5")
exit
 
def template_matching():
 
# Code written by Gavin Morrison D12124782
 
# Opening an image using a File Open dialog:
# f = easygui.fileopenbox()
# I = cv2.imread(f)
 
# Import image to be treated
original_image = cv2.imread("Ilovecats3.bmp")
 
# Imported image converted to Grayscale
original_image_grayscale = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
 
# Import template image
left_eye_template = cv2.imread("leftEye.bmp", 0)
 
# Store width and height coordinates of template
width, height = left_eye_template.shape[::-1]
 
# OpenCV has a number of methods for template matching, I have chosen two which I have put into a list
templateMatchingMethods = ['cv2.TM_CCORR_NORMED', 'cv2.TM_CCOEFF_NORMED']
 
# Loop to implement the method
for method in templateMatchingMethods:
methods = eval(method)
 
# Perform match operation
result_1 = cv2.matchTemplate(original_image_grayscale, left_eye_template, methods)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result_1)
top_left_1 = max_loc
bottom_right_1 = (top_left_1[0] + width, top_left_1[1] + height)
 
# Draw a rectangle around the matched area
cv2.rectangle(original_image, top_left_1, bottom_right_1, (0,0,255), 2)
 
 
# Display the results
finished_image = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)
plt.subplot(111),plt.imshow(finished_image,cmap = 'gray'), plt.title('Template Matching Result'), plt.xticks([]), plt.yticks([])
plt.show()
raw_input("Please press enter to return to Main Menu")
main_menu()
 
def camshift():
 
# Code written by Gavin Morrison D12124782
 
# Opening an image using a File Open dialog:
# f = easygui.fileopenbox()
# I = cv2.imread(f)
 
# Import image to be treated
original_image = cv2.imread("Ilovecats1.bmp")
# Copy of image to isolate region of interest
tracking_image = cv2.imread("Ilovecats1Copy.bmp")
 
# Coordinates of region of interest [row:row, column:column]
region_of_interest = tracking_image[125: 160, 210: 250]
# Coordinates for right eye
# region_of_interest = tracking_image[125: 175, 145: 200]
# The region of interest is converted to HSV colour space
HSV_region_of_interest = cv2.cvtColor(region_of_interest, cv2.COLOR_BGR2HSV)
#Create histogram of region of interest, using hue. The hue  range is from 0 to 179.
region_of_interest_histogram = cv2.calcHist([HSV_region_of_interest], [0], None, [180], [0, 180])
 
#Original image is converted to HSV colour space
original_image_HSV = cv2.cvtColor(original_image, cv2.COLOR_BGR2HSV)
 
# Back projection used to create mask from the hue of the region of interest histogram
mask = cv2.calcBackProject([original_image_HSV], [0], region_of_interest_histogram, [0, 180], 1)
 
# Filter applied in an attempt to reduce noise in the mask (later addition)
filter_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
mask = cv2.filter2D(mask, -1, filter_kernel)
_, mask = cv2.threshold(mask, 10, 255, cv2.THRESH_BINARY)
# The coordinates of our region of interest are assigned to variables
row = 210
column = 125
width = 250 - row
height = 160 - column
# Coordinates for right eye
# row = 145
# column = 125
# width = 200 - row
# height = 175 - column
 
# define criteria
criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10,1)
 
#rectangle of tracking area is created using camshift function
rectangle, tracking_area = cv2.CamShift(mask, (row, column, width, height), criteria)
 
#These are the points for the rectangle
points = cv2.boxPoints(rectangle)
points = np.int0(points)
# cv2.polylines used instead of 'cv2.rectangle' to accommodate rotation of bound space.
cv2.polylines(original_image, [points], True, (0, 255, 0), 2)
 
# Display our processed image
finished_image = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)
plt.subplot(111),plt.imshow(finished_image,cmap = 'gray'), plt.title('Camshift'), plt.xticks([]), plt.yticks([])
plt.show()
raw_input("Please press enter to return to Main Menu")
main_menu()
 
def gabor_filter():
 
# Sorting def. taken (and modified) from Geeks for Geeks
# URL: https://www.geeksforgeeks.org/python-sort-list-according-second-element-sublist/
def Sort(sub_li):
circlesList.sort(key = lambda x: x[0])
return sub_li
 
input_image  = cv2.imread("test.bmp")
rgb  = cv2.cvtColor(input_image, cv2.COLOR_BGR2RGB)
gray = cv2.cvtColor(input_image, cv2.COLOR_BGR2GRAY)
height, width, colourChannes = input_image.shape            
 
# Get 90 and 0 degree Gabor Images
g_kernel_90  = cv2.getGaborKernel((30, 30), 4.0, np.pi/2, 10.0, 0.5, 0, ktype=cv2.CV_32F)
g_image_90   = cv2.filter2D(gray, cv2.CV_8UC3, g_kernel_90)
 
# Create a mask
kernel = np.ones((10,10),np.uint8)
mask = cv2.bitwise_not(g_image_90)
mask = cv2.dilate(mask, kernel, iterations = 1)  # Dilate to include eyes
cropped = cv2.bitwise_and(gray,gray,mask = mask)
 
# Simplify image
cropped[cropped < 100] = 0
cropped[cropped > 100] = 255
 
circles = cv2.HoughCircles(cropped, cv2.HOUGH_GRADIENT,1,20,
param1=10, param2=10, minRadius=0, maxRadius=10)
 
circles = np.uint16(np.around(circles))
 
circlesList = list(circles[0])
bestGuess = Sort(circlesList)[0] # Get leftmost circle
cv2.circle(rgb,(bestGuess[0], bestGuess[1]),bestGuess[2]+20,(500),2)
 
plt.subplot(111)
plt.imshow(rgb)
plt.title("Output Image")
 
plt.show()
 
raw_input("Please press enter to return to Main Menu")
main_menu()
 
def colour_isolation():
 
# Author: Modestas Jakuska
# Title:  Skin Segmentation using YCrCb color range
# How it works:
# Convert image to YCbCr.
# Go through the image array and turn non-skin pixels black.
# Skin pixels are determined by their Cr and Cb values.
 
# Select image
f = easygui.fileopenbox()
I = cv2.imread(f)
 
# Convert BGR to RGB and then to YCrCb
I     = cv2.cvtColor(I, cv2.COLOR_BGR2RGB)
YCC   = cv2.cvtColor(I, cv2.COLOR_RGB2YCR_CB)
 
# Skin values taken from this paper:
# "Comparative Study of Skin Color Detection and Segmentation in HSV and YCbCr Color Space" by Khamar Basha Shaika, Ganesan P, V.Kalist, B.S.Sathish , J.Merlin Mary Jenitha
# They suggest using this range:
# 150 < Cr < 200 and 100 < Cb < 15
min_val = np.array([0,150,100],np.uint8)
max_val = np.array([255,200,150],np.uint8)
 
# Create skinRegion, a binary image containing the skin region
# Then superimpose the skin region onto the original image
# so that we can see the skin region with colour
skinRegion = cv2.inRange(YCC,min_val,max_val)
skinRegion = cv2.bitwise_and(I, I, mask = skinRegion)
 
plt.subplot(121)
plt.imshow(I)
plt.title("Input Image")
 
plt.subplot(122)
plt.imshow(skinRegion)
plt.title("Skin Region")
 
plt.show()
 
raw_input("Please press enter to return to Main Menu")
main_menu()
 
main_menu()

 

Implementation – Template Matching

Posted on by

The Eyes Have It!

Template matching is a method of finding something within an image by using another image of the item being sought. In our case we are looking for the left eye in a sequence of images so we are going to use a template which is an image of another left eye in an image.

Template matching can be quite limiting in the respect that simply making a comparison between the template and the image under inspection might not find provide accurate results if the item being sought is at a different angle or is of a different size.

The image can be scaled and rotated during the matching operation however, at the end of the day this method is relying on the image under inspection bearing a resemblance to the image in the template.

I created a program to perform a template matching operation on our images which worked out very well when we used as a template the left eye in the first image in our sequence.

When I used a template consisting of another eye image the results have a very different level of accuracy.

In image 1 and 2 the template has recognised an eye within the images but not the eye we are looking for, and in image 3 the matching operation has not recognised either eye within the image.

So to summarize, template matching has worked well using a template based on one of the images within our sequence but with a template image of a similar object unrelated to our images, template matching has not worked well at all. With this in mind, it is possible that if there had been radical changes in lighting or other detail within our sequence of images, template matching could quite possibly fall over.

Code for my Template Matching program:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt
import easygui
 
# Code written by Gavin Morrison D12124782
 
# Opening an image using a File Open dialog:
# f = easygui.fileopenbox()
# I = cv2.imread(f)
 
# Import image to be treated
original_image = cv2.imread("Ilovecats3.bmp")
 
# Imported image converted to Grayscale
original_image_grayscale = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY)
 
# Import template image
left_eye_template = cv2.imread("leftEye.bmp", 0)
 
# Store width and height coordinates of template
width, height = left_eye_template.shape[::-1]
 
# OpenCV has a number of methods for template matching, I have chosen two which I have put into a list
templateMatchingMethods = ['cv2.TM_CCORR_NORMED', 'cv2.TM_CCOEFF_NORMED']
 
# Loop to implement the method
for method in templateMatchingMethods:
methods = eval(method)
 
# Perform match operation
result_1 = cv2.matchTemplate(original_image_grayscale, left_eye_template, methods)
min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result_1)
top_left_1 = max_loc
bottom_right_1 = (top_left_1[0] + width, top_left_1[1] + height)
 
# Draw a rectangle around the matched area
cv2.rectangle(original_image, top_left_1, bottom_right_1, (0,0,255), 2)
 
 
# Display the results
finished_image = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)
plt.subplot(111),plt.imshow(finished_image,cmap = 'gray'), plt.title('Template Matching Result'), plt.xticks([]), plt.yticks([])
plt.show()
# Code below is for combined Matplotlib display, not required for function.
# treated_image_1 = cv2.imread("Ilovecats1_TM_with_rectangle.jpg")
# treated_image_2 = cv2.imread("Ilovecats2_TM_with_rectangle.jpg")
# treated_image_3 = cv2.imread("Ilovecats3_TM_with_rectangle.jpg")
 
# Change colour space for MatPlotLib display
# finished_image_1 = cv2.cvtColor(treated_image_1, cv2.COLOR_BGR2RGB)
# finished_image_2 = cv2.cvtColor(treated_image_2, cv2.COLOR_BGR2RGB)
# finished_image_3 = cv2.cvtColor(treated_image_2, cv2.COLOR_BGR2RGB)
 
# Display the results
# plt.subplot(131),plt.imshow(finished_image_1,cmap = 'gray'), plt.title('Image 1'), plt.xticks([]), plt.yticks([])
# plt.subplot(132),plt.imshow(finished_image_2,cmap = 'gray'), plt.title('Image 2'), plt.xticks([]), plt.yticks([])
# plt.subplot(133),plt.imshow(finished_image_3,cmap = 'gray'), plt.title('Image 3'), plt.xticks([]), plt.yticks([])
# plt.suptitle('Template Matching Result')
# plt.show()

 

References:

Multi-scale Template Matching using Python and OpenCV (accessed 17.11.2018)
https://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/

Template Matching (accessed 17.11.2018)
http://www.swarthmore.edu/NatSci/mzucker1/opencv-2.4.10-docs/doc/tutorials/imgproc/histograms/template_matching/template_matching.html#template-matching

Object detection with templates (accessed 17.11.2018)
https://pythonspot.com/object-detection-with-templates/

Template matching (accessed 17.11.2018)
https://en.wikipedia.org/wiki/Template_matching

Cox GS. Template Matching and Measures of Match in Image Processing. Department of Electrical Engineering, University of Cape Town.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.9646&rep=rep1&type=pdf

 

Assignment 2 – Where’s Wally

Posted on by

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
# Gavin Morrison - D12124782 - DT211C-4
 
# 1. I inputted the image and converted it to Greyscale and HSV for the
# operations within the program.
 
# 2. I created a Numpy array of the upper and lower red values of our
# HSV image
 
# 3. I created a mask of the red values
 
# 4. Red content of the image is isolated via a Bitwise 'And' operation
 
# 5. The Greyscale image is inverted
 
# 6. A Binary threshold is applied to our inverted Greyscale image.
 
# 7. The thresholded image is then submitted to another bitwise
# inversion which produces the white content of the image.
 
# 8. The isolated red and white content are combined.
 
# 9. The red and white components of the original image are dilated to
# produce a slight overlap allowing the detection of connected regions.
 
# 10. The dilated images are 'Anded' to produce an image of only the
# overlapping regions.
 
# 11. Canny is applied to detect edges within the image of connected regions.
 
# 12. A Hough Line Transform is applied to the results of Canny to try and
# find the horizontal lines within Wally's sweater.
 
# 13. Wally is located. Please see comments throughout the code for further
# step by step details.
 
# I initially tried using Contours to locate Wally but I had very limited
# results owing to the low resolution of Wally within the overall image.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
 
# Original image is imported
originalImage = cv2.imread("Where.jpg")
 
# Original image is converted to greyscale for white segmentation.
originalImageGrey = cv2.cvtColor(originalImage, cv2.COLOR_BGR2GRAY)
 
# Original image is converted to a HSV Image for red segmentation.
HSVImage = cv2.cvtColor(originalImage, cv2.COLOR_BGR2HSV)
 
# A Numpy array is created of upper and lower red values.
lowerRedValue = np.array([139,100,100])
upperRedValue = np.array([189,255,255])
 
# Mask is created of all red values.
redMask = cv2.inRange(HSVImage, lowerRedValue, upperRedValue)
 
# The red content of the image is isolated.
redContent = cv2.bitwise_and(originalImage,originalImage, mask= redMask)
 
# The Greyscale image is inverted via bitwise.
greyInverted = cv2.bitwise_not(originalImageGrey)
 
# A binary threshold is applied to the inverted greyscale image.
T = 20
T, binaryInvertedGreyscale = cv2.threshold(greyInverted, thresh = T, maxval = 255,
type = cv2.THRESH_BINARY)
 
# This bitwise not operation creates the white content image.
whiteContent = cv2.bitwise_not(binaryInvertedGreyscale)
 
# white and red segmented images are written to file and the images inputed.
cv2.imwrite("white.png", whiteContent)
cv2.imwrite("red.png", redContent)
redImage = cv2.imread("red.png")
whiteImage = cv2.imread("white.png")
 
#The red and white images are combined together and written to file.
redWhiteCombined = redImage + whiteImage
cv2.imwrite("combined.png", redWhiteCombined)
 
# Dilation is perfomed on red and white images to create a slight overlap to allow the location of where red and white are connected.
whiteKernel = np.ones((2,1), np.uint8)
redKernel = np.ones((2,1), np.uint8)
whiteDilation = cv2.dilate(whiteContent, whiteKernel, iterations=1)
redDilation = cv2.dilate(redMask, redKernel, iterations=1)
 
# Dilated images are combined in 'And' operation which isolates connecting areas between red and white
connectedRegions = whiteDilation & redDilation
 
# Connected regions written to file and then inputed
cv2.imwrite("connectedregions.png", connectedRegions)
connectedRegionsImage = cv2.imread("connectedregions.png")
 
# The connected regions are converted to greyscale.
connectedGreyscale = cv2.cvtColor(connectedRegionsImage, cv2.COLOR_BGR2GRAY)
 
# The edges are located on the connected regions.
connectedEdges = cv2.Canny(connectedGreyscale, 500, 300)
 
# A Hough Line Transform is applied that scans the image for lines in the hope that
# it will locate the stripes on Wally's sweater.
houghLines = cv2.HoughLinesP(connectedEdges, 1, np.pi/90, 30, maxLineGap=2)
 
#For loop plotting location of lines found in image.
for line in houghLines:
    x1, y1, x2, y2 = line[0]
    cv2.line(connectedRegionsImage, (x1, y1), (x2, y2), (255, 255, 255), 90)
 
# Location written to file and inputed
cv2.imwrite('wallylocation.png',connectedRegionsImage)
wallyLocation = cv2.imread("wallylocation.png")
 
# Location found is combined with original image in 'And' operation and written to file.
wallyFound = wallyLocation & originalImage
cv2.imwrite('wallyfound.png',wallyFound)
 
# Colour space conversions for matPlotLib display.
OImatPlotLib = cv2.cvtColor(originalImage, cv2.COLOR_BGR2RGB)
RCmatPlotLib = cv2.cvtColor(redContent, cv2.COLOR_BGR2RGB)
RWmatPlotLib = cv2.cvtColor(redWhiteCombined, cv2.COLOR_BGR2RGB)
WFmatPlotLib = cv2.cvtColor(wallyFound, cv2.COLOR_BGR2RGB)
 
# A display of the result of our operations
plt.subplot(231), plt.imshow(OImatPlotLib, cmap ='gray'), plt.title( 'Original Image' )
plt.subplot(232), plt.imshow(whiteContent, cmap ='gray'), plt.title( 'White Content' )
plt.subplot(233), plt.imshow(RCmatPlotLib, cmap ='gray'), plt.title( 'Red Content' )
plt.subplot(234), plt.imshow(RWmatPlotLib, cmap ='gray'), plt.title( 'Red & White combined' )
plt.subplot(235), plt.imshow(connectedRegions, cmap ='gray'), plt.title( 'Connected Regions' )
plt.subplot(236), plt.imshow(WFmatPlotLib, cmap ='gray'), plt.title( 'Wally Found' )
plt.show()

 

Week 9 – Lab Work

Posted on by

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# 1. Open 'Sudoku.jpg';
 
# 2. Create a Harris corner image for this image;
 
# 3. Use Shi-Tomasi to find the location of the strongest corners.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("sudoku.jpg")
 
gray = cv2.cvtColor(I,cv2.COLOR_BGR2GRAY)
# Harris Corner code
gray = np.float32(gray)
dst = cv2.cornerHarris(gray,2,3,0.04)
 
#result is dilated for marking the corners, not important
dst = cv2.dilate(dst,None)
 
# Threshold for an optimal value, it may vary depending on the image.
I[dst>0.01*dst.max()]=[0,0,255]
 
# Shi-Tomasi code
corners = cv2.goodFeaturesToTrack(gray,25,0.01,10)
corners = np.int0(corners)
 
for i in corners:
    x,y = i.ravel()
    cv2.circle(I,(x,y),3,255,-1)
 
# plt.imshow(I),plt.show()
 
cv2.imshow('dst',I)
# cv2.imshow("Shi-Tomasi", K)
key = cv2.waitKey(0)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# 1. Open "Micro.jpg";
 
# 2. Use either thresholding or edge detection to create a binary mask;
 
# 3. Extract and draw the boundary of the cell by finding the largest
# contour in the image.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
im = cv2.imread('Micro.jpg')
imgray = cv2.cvtColor(im,cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(imgray,127,255,0)
im2, contours, hierarchy = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
 
cv2.drawContours(im, contours, -1, (0,255,0), 3)
 
# create hull array for convex hull points
hull = []
# calculate points for each contour
for i in range(len(contours)):
    # creating convex hull object for each contour
    hull.append(cv2.convexHull(contours[i], False))
 
# draw contours and hull points
for i in range(len(contours)):
    color_contours = (0, 0, 255) # green - color for contours
    color = (255, 0, 0) # blue - color for convex hull
    # draw with contour
    cv2.drawContours(im, contours, i, color_contours, 1, 8, hierarchy)
    # draw with convex hull object
    cv2.drawContours(im, hull, i, color, 1, 8)
 
cv2.imshow('Combined',im)
key = cv2.waitKey(0)

 

Brother Parsons

Posted on by

I discovered today that the history of Digital Image Processing started with developments made in JPL (Jet Propulsion Laboratory) in the 1960’s and a few other research facilities. More can be found on this topic at the Wikipedia link below.

Of course there is one founder member of JPL that I am very familiar with and that is Jack (Whiteside) Parsons. Who also happens to be the subject of the TV drama series ‘Strange Angel’.

Only in the irrational and unknown direction can we come to wisdom again.

-Jack Whiteside Parsons-

 

Digital image processing (Accessed 09.11.2018)
https://en.wikipedia.org/wiki/Digital_image_processing

 

 

Week 8 – Lab Work

Posted on by

This week Features and Corners.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# 1. Open any image;
 
# 2. Use Sobel to find the horizontal and vertical gradients;
 
# 3. Create an edge mask using Canny (255 for edges 0 for not);
 
# 4. On the edges, show the magnitude of the gradients.
 
# Advanced Task: Devise a way to visualise the direction of the gradients on the edges.
 
import numpy as np
import cv2
from matplotlib import pyplot as plt
from matplotlib import image as image
import easygui
 
I = cv2.imread("ball.jpg")
 
G = cv2.cvtColor(I, cv2.COLOR_BGR2GRAY)
 
Ix = cv2.Sobel(G,ddepth=cv2.CV_64F,dx=1,dy=0)
Iy = cv2.Sobel(G,ddepth=cv2.CV_64F,dx=0,dy=1)
 
M = np.sqrt(Ix*Ix + Iy*Iy)
 
# D = np.arctan(Iy/Ix)
 
E = cv2.Canny(G,threshold1=100,threshold2=200)
 
F = cv2.bitwise_and(Ix,Iy,mask=E)
 
# plt.figure()
# plt.subplot(2,3,2)
# plt.imshow(Ix)
# plt.subplot(2,3,1)
# plt.imshow(Iy)
# plt.subplot(2,3,3)
# plt.imshow(E)
# plt.show()
# cv2.imshow("image", F)
# key = cv2.waitKey(0)
 
plt.subplot(231), plt.imshow(G, cmap ='gray')
plt.subplot(232), plt.imshow(Ix, cmap ='gray')
plt.subplot(233), plt.imshow(Iy, cmap ='gray')
plt.subplot(234), plt.imshow(E, cmap ='gray')
plt.subplot(235), plt.imshow(F, cmap ='gray')
plt.subplot(236), plt.imshow(M, cmap ='gray')
plt.show()

 

Research & Implementation – Tracking Texture & Tracking Motion

Posted on by

Part 2 – Tracking Motion

While I was thinking about which way to research this approach, it set me thinking about what motion was, or more specifically, what is a computer’s idea of motion.

Definition of Motion taken from OED

The answer would be that it would not understand motion, it would detect that the information contained within an image may have changed in comparison to the next one in a sequence but all we are talking about are specific 2D or 3D values associated with individual pixels within a specific colour space.

So to achieve a result that we would recognise as motion the computer has to take the pixel values and analyse the change in values in a way that we can use for tracking movement.

Centroid (find the centre of a blob)

So what is a blob exactly? in image processing terminology a blob is apparently a group of connected pixels sharing common values, and the centroid in this instance is the mean or average of this blob or the weighted average if you will.

OpenCV finds this weighted average with what is known as ‘moments’ by converting the image to grayscale, then perform binarisation before conducting the centroid calculations.

Multiple blobs can can be detected using this method within OpenCV with the aid of ‘contours’.

Meanshift & Camshift

Meanshift and Camshift can be used in conjunction with Centroid for the purposes of locating the part of the image we need to track.

Meanshift is a non-parametric density gradient estimation. It is useful for detecting the modes of this density. Camshift combines Meanshift with an adaptive region sizing step.

Both Meanshift and Camshift use a method called histogram back projection which uses colour as hue from a HSV model.

Results of my Camshift experiment on our allocated images.

Code for my Camshift program:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
import sys
import numpy as np
import cv2
from matplotlib import pyplot as plt
import easygui
 
# Code written by Gavin Morrison D12124782
 
# Opening an image using a File Open dialog:
# f = easygui.fileopenbox()
# I = cv2.imread(f)
 
# Import image to be treated
original_image = cv2.imread("Ilovecat1.bmp")
# Copy of image to isolate region of interest
tracking_image = cv2.imread("Ilovecats1Copy.bmp")
 
# Coordinates of region of interest [row:row, column:column]
region_of_interest = tracking_image[125: 160, 210: 250]
# Coordinates for right eye
# region_of_interest = tracking_image[125: 175, 145: 200]
# The region of interest is converted to HSV colour space
HSV_region_of_interest = cv2.cvtColor(region_of_interest, cv2.COLOR_BGR2HSV)
#Create histogram of region of interest, using hue. The hue  range is from 0 to 179.
region_of_interest_histogram = cv2.calcHist([HSV_region_of_interest], [0], None, [180], [0, 180])
 
#Original image is converted to HSV colour space
original_image_HSV = cv2.cvtColor(original_image, cv2.COLOR_BGR2HSV)
 
# Back projection used to create mask from the hue of the region of interest histogram
mask = cv2.calcBackProject([original_image_HSV], [0], region_of_interest_histogram, [0, 180], 1)
 
# Filter applied in an attempt to reduce noise in the mask (later addition)
filter_kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (3, 3))
mask = cv2.filter2D(mask, -1, filter_kernel)
_, mask = cv2.threshold(mask, 10, 255, cv2.THRESH_BINARY)
# The coordinates of our region of interest are assigned to variables
row = 210
column = 125
width = 250 - row
height = 160 - column
# Coordinates for right eye
# row = 145
# column = 125
# width = 200 - row
# height = 175 - column
 
# define criteria
criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT, 10,1)
 
# rectangle of tracking area is created using camshift function
rectangle, tracking_area = cv2.CamShift(mask, (row, column, width, height), criteria)
 
#These are the points for the rectangle
points = cv2.boxPoints(rectangle)
points = np.int0(points)
# cv2.polylines used instead of 'cv2.rectangle' to accommodate rotation of bound space.
cv2.polylines(original_image, [points], True, (0, 255, 0), 2)
 
# Display our processed image
finished_image = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB)
plt.subplot(111),plt.imshow(finished_image,cmap = 'gray'), plt.title('Camshift Result'), plt.xticks([]), plt.yticks([])
plt.show()

 

REFERENCES:

Find the Center of a Blob (Centroid) using OpenCV (C++/Python) (accessed 2nd November 2018)
https://www.learnopencv.com

Meanshift and Camshift (accessed 02.11.2018)
https://docs.opencv.org/3.4.3/db/df8/tutorial_py_meanshift.html

Mean Shift Tracking (accessed 02.11.2018)
https://www.bogotobogo.com/python/OpenCV_Python/python_opencv3_mean_shift_tracking_segmentation.php

Back Projection (accessed 02.11.2018)
https://docs.opencv.org/2.4/doc/tutorials/imgproc/histograms/back_projection/back_projection.html

Bradski GR. Computer Vision Face Tracking For Use in a Perceptual User Interface. Microcomputer Research Lab, Santa Clara, CA, Intel Corporation.
http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=B9A5277FF173D0455494A756940F7E6B?doi=10.1.1.14.7673&rep=rep1&type=pdf