How does Basic Convolution Work for Image Processing?

Image manipulation is a technique that many mobile and web applications based on image or video editing use, but how do they achieve the manipulation they want? And how could we implement image manipulation using machine learning? Let’s discuss it in this blog.

 

As we know that the images we see are made of pixels, these pixels can be represented in numerical forms, therefore, by making changes to the numeric values and keeping the dimension the same must lead to the image manipulation for sure. 

 

So let’s learn what convolution and kernels are, this will help us in the implementation of basic convolution using python.

 

What is Convolution?

 

Convolution is the most important topic in the field of image processing, a convolution is an operation with which we can merge two arrays by multiplying them, these arrays could be of different sizes, the only condition, however, is that the dimensions should be the same for both arrays. 

 

Below is the representation of a convolution, where the numerical value denotes the pixel values of an image.

---

Convolution represented with pixel values

---

Gray Level images are generally used as an input array as far as image processing is considered. To understand how convolution is performed, we must know about kernels as they are the most important part to perform convolution. So basically, two arrays merge to produce the third result, and that is how image manipulation is done.

 

(Also read: How to Implement Convolutional Autoencoder Using Keras)

 

 

What are Kernels?

 

Kernels are smaller portions taken from the convolution and are used to slide over the convolution, the main objective of these kernels are to retrieve valuable information from the convolution with fewer dimensions.

 

(Recommended blog: Applications and Functions of Opencv)

 

A kernel may be called a ‘mask’, or a ‘convolutional matrix’ as it is achieved by masking over a convolution. Many effects could be achieved with the help of image kernels, these effects include blurring the image, sharpening of image, increasing or decreasing the contrast, and many more. 

 

Below is the image showing the kernels being masked over a convolution.

---

Image kernels 

---

These convolutional kernels are used in one deep learning algorithm as well, i.e, convolutional neural networks. 

 

Now we shall discuss the working of convolutional kernels in detail.

 

Working of Convolutional Kernel

 

There are majorly three steps to keep in mind in order to understand the working of an convolutional kernel, therefore, below is the image for the architecture of the whole working-:

---

Working of convolutional kernel, Source

---

So in the process of convolution, the image is manipulated by rolling kernels over convolutional, in the image we can see that the convolution is mapped over an source pixel, the kernel values are then multiplied with the corresponding value of pixel it is covering, at the end the sum of all the multiplied values are taken, which becomes the first value (centre pixel value).
 

 

The new pixel values are filled by taking another patch of source pixel, and at the end, all we are left with is a new transformed pixel values that have features of the original image but also with less dimensions and transformation. Now we shall try to implement image manipulation using machine learning algorithms.

 

 

Implementing Convolution Using Python

 

In our first step, we are going to import some of the important libraries in order to implement convolution. These libraries include numpy for mathematical operation, matplotlib for data visualization, and cv2 for computer vision problems.

---

import numpy as np

import cv2

import matplotlib.pyplot as plt

---

In our next step, we are going to read the image first, cvtColor is used in the below code to change the color space of an input image, with the help of ‘cv2.COLOR_BGR2GRAY’ we are changing the image scale to Grayscale. Later, we are resizing the image to 100*100.

---

im = cv2.imread('Downloads/red-rose.jpg')

im = cv2.cvtColor(im, cv2.COLOR_BGR2GRAY)

im = cv2.resize(im, (100,100))

Im.shape

(100, 100)




plt.figure()

plt.imshow(im, cmap='gray')

---

---

Later on, we are implementing the kernel using numpy library, making use of numpy to create matrices for gaussian blur, vertical edge, and outline. We are storing all the array information under a variable named ‘kernel’

---

kernel = np.array([

    [1/16, 1/8, 1/16],

    [1/8 , 1/4, 1/8],

    [1/16, 1/8, 1/16],

]) # Gaussian Blur



kernel = np.array([

    [1, 0, -1],

    [1, 0, -1],

    [1, 0, -1],

]) # Vertical Edge



kernel = np.array([

    [-1, -1, -1],

    [-1,  8, -1],

    [-1, -1, -1],

]) # Outline

---

In our next step, we have to perform the working of transformation. All the steps we discussed above in the working of convolution is what we need to implement here, the multiplication of kernel values with pixel values and the sum is placed as a centre of a new pixel values.

---

# N - f + 1

convolved_image = np.zeros((im.shape[0] - kernel.shape[0] + 1, im.shape[1] - kernel.shape[1] + 1))

convolved_image.shape

(98, 98)



for i in range(convolved_image.shape[0]):

    for j in range(convolved_image.shape[1]):

        patch = im[i:i+kernel.shape[0], j:j+kernel.shape[1]]

        hprod = (patch * kernel).sum()

        convolved_image[i,j] = hprod



plt.figure()

plt.imshow(convolved_image, cmap='gray')

---

---

Let’s visualize using matplotlib library, we are visualizing RGB image of an original image, let’s see the transformation-:

---

im = cv2.imread('Downloads/red-rose.jpg')

im = cv2.cvtColor(im, cv2.COLOR_BGR2RGB)



for i in range(3):

    plt.figure(i)

    plt.title("RGB"[i])

    plt.imshow(im[:,:,i], cmap='gray')

---

---

---

---

---

---

We have successfully made the transformation on the original image, with the good knowledge of convolution, one can make any sort of changes and transformation to an image or a video, these techniques are mathematically understandable with easy implementation.

 

 

Conclusion

 

In the field of image processing, convolution and kernels play a very important role, thus, having a good knowledge about them helps in several operations which could be performed over an image or a video.

 

(Must catch: Transfer learning with CNN)

 

There are several other techniques and tools under image processing and machine learning which help in the transformation of image, these transformations may also help in restoration of some torned images as well, with so many use cases, this image processing should be considered as a great step in the advancement of the field.