The Eyes Have It!
Template matching is a method of finding something within an image by using another image of the item being sought. In our case we are looking for the left eye in a sequence of images so we are going to use a template which is an image of another left eye in an image.
Template matching can be quite limiting in the respect that simply making a comparison between the template and the image under inspection might not find provide accurate results if the item being sought is at a different angle or is of a different size.
The image can be scaled and rotated during the matching operation however, at the end of the day this method is relying on the image under inspection bearing a resemblance to the image in the template.
I created a program to perform a template matching operation on our images which worked out very well when we used as a template the left eye in the first image in our sequence.
When I used a template consisting of another eye image the results have a very different level of accuracy.
In image 1 and 2 the template has recognised an eye within the images but not the eye we are looking for, and in image 3 the matching operation has not recognised either eye within the image.
So to summarize, template matching has worked well using a template based on one of the images within our sequence but with a template image of a similar object unrelated to our images, template matching has not worked well at all. With this in mind, it is possible that if there had been radical changes in lighting or other detail within our sequence of images, template matching could quite possibly fall over.
Code for my Template Matching program:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 |
import sys import numpy as np import cv2 from matplotlib import pyplot as plt import easygui # Code written by Gavin Morrison D12124782 # Opening an image using a File Open dialog: # f = easygui.fileopenbox() # I = cv2.imread(f) # Import image to be treated original_image = cv2.imread("Ilovecats3.bmp") # Imported image converted to Grayscale original_image_grayscale = cv2.cvtColor(original_image, cv2.COLOR_BGR2GRAY) # Import template image left_eye_template = cv2.imread("leftEye.bmp", 0) # Store width and height coordinates of template width, height = left_eye_template.shape[::-1] # OpenCV has a number of methods for template matching, I have chosen two which I have put into a list templateMatchingMethods = ['cv2.TM_CCORR_NORMED', 'cv2.TM_CCOEFF_NORMED'] # Loop to implement the method for method in templateMatchingMethods: methods = eval(method) # Perform match operation result_1 = cv2.matchTemplate(original_image_grayscale, left_eye_template, methods) min_val, max_val, min_loc, max_loc = cv2.minMaxLoc(result_1) top_left_1 = max_loc bottom_right_1 = (top_left_1[0] + width, top_left_1[1] + height) # Draw a rectangle around the matched area cv2.rectangle(original_image, top_left_1, bottom_right_1, (0,0,255), 2) # Display the results finished_image = cv2.cvtColor(original_image, cv2.COLOR_BGR2RGB) plt.subplot(111),plt.imshow(finished_image,cmap = 'gray'), plt.title('Template Matching Result'), plt.xticks([]), plt.yticks([]) plt.show() # Code below is for combined Matplotlib display, not required for function. # treated_image_1 = cv2.imread("Ilovecats1_TM_with_rectangle.jpg") # treated_image_2 = cv2.imread("Ilovecats2_TM_with_rectangle.jpg") # treated_image_3 = cv2.imread("Ilovecats3_TM_with_rectangle.jpg") # Change colour space for MatPlotLib display # finished_image_1 = cv2.cvtColor(treated_image_1, cv2.COLOR_BGR2RGB) # finished_image_2 = cv2.cvtColor(treated_image_2, cv2.COLOR_BGR2RGB) # finished_image_3 = cv2.cvtColor(treated_image_2, cv2.COLOR_BGR2RGB) # Display the results # plt.subplot(131),plt.imshow(finished_image_1,cmap = 'gray'), plt.title('Image 1'), plt.xticks([]), plt.yticks([]) # plt.subplot(132),plt.imshow(finished_image_2,cmap = 'gray'), plt.title('Image 2'), plt.xticks([]), plt.yticks([]) # plt.subplot(133),plt.imshow(finished_image_3,cmap = 'gray'), plt.title('Image 3'), plt.xticks([]), plt.yticks([]) # plt.suptitle('Template Matching Result') # plt.show() |
References:
Multi-scale Template Matching using Python and OpenCV (accessed 17.11.2018)
https://www.pyimagesearch.com/2015/01/26/multi-scale-template-matching-using-python-opencv/
Template Matching (accessed 17.11.2018)
http://www.swarthmore.edu/NatSci/mzucker1/opencv-2.4.10-docs/doc/tutorials/imgproc/histograms/template_matching/template_matching.html#template-matching
Object detection with templates (accessed 17.11.2018)
https://pythonspot.com/object-detection-with-templates/
Template matching (accessed 17.11.2018)
https://en.wikipedia.org/wiki/Template_matching
Cox GS. Template Matching and Measures of Match in Image Processing. Department of Electrical Engineering, University of Cape Town.
http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.51.9646&rep=rep1&type=pdf