Streaming Android Camera using OpenCV

In the previous blogs, you have seen OpenCV used with Python in various applications like image processing, face detection, object detection,  gesture recognition, object tracking, motion detection, etc.

 

OpenCV is an Open Source Computer Vision library, as it provides various functions, to perform all of the applications, like read, write, display the images, edge detection, contour detection, and many more. All these functions work for both images and videos either saved media or live streaming. (To know more about OpenCV and its application and functions, learn from here).

 

For live streaming, you can either use a web camera or some external camera like Pi camera, or android camera. This article is about how to access the android camera with the help of OpenCV in python and later will perform some of the common OpenCV applications.

 

 

Comparing Cases of Web Camera and External Camera

 

• When using the web camera for any application then it’s limited to one place as you cant carry it everywhere all the time for testing. But this is not the case with the external camera, as you can carry it anywhere with placing system in one place, so it makes it flexible for testing.

• Another used case is when you want to test your code in the dining hall, but you can’t as your system is in your room but if you use an external camera then it’s possible for you.

• Apart from this if you want to keep process everything going on in your house/offices by sitting in your room then also this can be possible by using an external camera, obviously, replacement if CCTV, like face detection, face recognition, object detection, motion detection, and many more.

 

Pre-requisite

 

• Python 3

• OpenCV

• Numpy

• Install IP webcam on your android device

• Jupyter notebook or any other code editor like vscode, sublime.

 

 

Getting Started

 

• Connect your PC/laptop and android device to the same wifi,

• Open the app on your phone,

• Navigate through the app and select the option Start server at the bottom,

• Note down the IP address that appears on the screen,

• Paste the same in the browser and then a page will open where you can control the camera of your phone with the available options there.

---

IP webcam controller in browser

---

• As from the above image, it’s clear that with the IP address there is a number of options available to control the android camera from the laptop like the way CCTV cameras.

• Capture image, focused image, can save it to the storage, video recording, live streaming in the laptop as seen in the below figure,

  ---

  

Live streaming from android in laptop

---

• Also can toggle flash and front camera from the browser.

 

Access Android Camera with OpenCV

 

• At the top of the page in the browser move to Video render option and then select Javascript and then copy the image address by right click on that, as it will be helpful to access the camera in OpenCV. 

  ---

   

Get the image address in JavaScript mode

---

Open Jupyter notebook and get started with coding;

 

• Import the required packages

  • Urllib- This package will allow us interacting with internet websites by their URL address. It provides different modules to open, parse the URLs as seen in the below image.

---

 

  

urllib packages in Python Sources

---

• CV2 - OpenCV package to deal with the images as to display the images captured from the camera or you can term this as live streaming.

---

Import the libraries

---

• Store the image address from the above in a variable URL.

---

Image address for live streaming

---

• Start the while loop till any key will be pressed to terminate the same, Q in this case. Next access the image from the URL using urlopen() function, bypassing the above URL as an argument and it will return and it will return http.client.HTTPResponse object.

• Next, convert the object into NumPy and later on, we will use cv2.imdecode() to convert the array into an image format from the streaming data.

• Now its time for live streaming using cv2.imshow() from android’s camera in python.

---

Live streaming from the android camera using OpenCV

---

Once you will be done with the above procedure, now you can any computer vision application with your android camera like object detection, motion tracking, face recognition, etc. Without investing in CCTV cameras you can spy your house by sitting in one place with your android phone.

 

OpenCV applications with the android camera

 

Face Detection

 

• Import the required libraries

• Load the pre-trained haar cascade classifier for face detection from the storage.

---

 

Load the haar cascade classifier for face detection

---

• Follow the same code to access the streaming image.

---

Get the streaming image 

---

• Convert the image into the grayscale format and then use “usign classifier” object to detect the faces present in the streaming image and store all of the coordinates which will be used to plot bounding boxes on the detected faces.

---

Live streaming of face detection from android phone

---

 

Motion Detection

 

The last article covered live motion detection using OpenCV with the help of a web camera but it has several limitations as discussed above because it will bound to use at only one place. Instead, use your Android camera to keep track of changes at any place concerning a particular frame at a particular time. 

Let’s suppose you are doubting that someone behind you makes changes in your workplace but you can’t be able to figure it out as CCTVs are too costlier to afford but this approach is more economical and easier to keep track of of victim.

• Import all the required libraries

• Assign static frame with the frame concerning the one you want to compare all other frames, in this case initially it will be none, and once a camera start it will initialize to the first frame. 

---

Import the required modules

---

• Load the streaming image into image format from the URL using urllib modules.

---

 

Store the streaming image into an image format

---

• Convert the image into a grayscale format followed by the image blurring using cv2.GaussianBlur() function, to detect motion changes.

• The next step is to the calculation of the difference between the static frame and image from streaming by an android camera which further if crosses the set threshold, demonstrates the motion detection. Later on, using contour detection bounding boxes will be plotted on the region of motion in the image.

---

Steps for motion detection

---

• Here is the completed code along with the output.

---

Completed code for live motion detection using Android Camera

---

 

Conclusion

 

OpenCV, a computer vision library, is open-source with several functions available to perform the different applications, make everything easy and effective. Also, it’s fast in computation as compared to other available languages like MATLAB. 

 

OpenCV can be used for images as well as videos either from the storage or from the live streaming using a web- camera or external camera. This blog has been specifically designed to use OpenCV with an external camera i.e. android camera by using third party application IP Webcam on the same network i.e. both the android device and laptop/PC.

 

With this, you can try different computer vision approaches like keep spying of your house in your absence, or when you come back to your place. Object detection, facial recognition, motion tracking, and many others.