Building Surveillance System using USB Camera and Wireless-Connected Raspberry Pi

This tutorial discusses how to build a surveillance system using a USB camera plugged into Raspberry Pi (RPi) which is connected a PC using its wireless interface. PyGame is used for accessing the camera and capturing images which are stored inside the SD card of RPi. A background model is created for the scene to discover the changes from the default state. A simple circuit, connected to the GPIO pins, lights a led as an indication of such states. 

The tutorial has the following steps:

1. Connecting RPi to a PC using the wireless interface.
2. Connecting a USB camera to RPi.
3. Capturing images using PyGame.
4. Building the background model.
5. Detecting changes to the background model.
6. Building a simple circuit that lights a led when a change occurs.

The tutorial assumes that the reader has a PC connected to a wireless network and RPi connected to a router using the Ethernet interface. For doing such work, please read the tutorial titled ”Building an Image Classifier Running on Raspberry Pi” which is available at this link:

https://www.dhirubhai.net/pulse/building-image-classifier-running-raspberry-pi-ahmed-gad

You should be able to know the IP address of the devices using a software such as “Advanced IP Scanner” and familiar with establishing the SSH connection using the MobaXterm software. After that, then we can start discussing each of the above steps. 

1. Connecting RPi to a PC using the Wireless Interface

In the previous tutorial “Building an Image Classifier Running on Raspberry Pi”, we built a network with three devices connected which are RPi, a router, and a PC. RPI was connected to the router using the Ethernet interface but the PC is connected to the router using the wireless interface. In this tutorial, we will modify such network by using the wireless interface of the RPi to connect to the router. This makes the network completely connected wirelessly and avoids the restrictions of the wires.

The wireless interface of RPi needs configuration before being able to use it. For such reason, we will still use the connection established using the Ethernet interface for configuring the wireless interface. Using DHCP in the router, the RPi Ethernet interface will be given an IPv4 address. Such an address can then be used for establishing a secure shell (SSH) connection. You can refer to the previous tutorial for more details about establishing the connection.

Inside the SSH session created using MobaXterm, we can start configuring the wireless interface using either terminal commands or using the GUI of the Raspbian OS. Both ways are simple.

Using terminal commands, we will start by scanning the available wireless networks to find their service set identifiers (SSIDs). This is done using the following command:

pi@raspberrypi:~ $ sudo iwlist wlan0 scan

The first few lines of the output of this command are given in the following figure. The SSID of the target wireless network is “TEData_864A”. Note that you do not have to use a router for connecting the PC to the RPi. Using a smartphone, we can create an access point for connecting them.

After knowing the SSID of the target network, we can use it for configuring the wireless interface. Such a configuration exists inside a file which can be accessed using the following command:

pi@raspberrypi:~ $ sudo nano /etc/wpa_supplicant/wpa_supplicant.conf

The file will be opened inside the terminal. Using the down arrow, go back to the end of the file to write the network configuration. This is by writing the SSID and the password of the network as follows:

network={

                ssid=”TEData_864A”

                psk=”YOUR_NETWOK_PASSWORD”

}

After that, press CTRL+X to exit the file followed by Y to save the changes. For my case, I configured 2 networks and thus the file content has two configured networks according to the next figure. 

You might need to change the status of the wireless interface from up to down them up again to apply such configurations. This is done using these 2 commands:

pi@raspberry:~ $ sudo ifdown wlan0

pi@raspberry:~ $ sudo ifup wlan0

To check whether the wireless interface is configured correctly, enter the “ifconfig” command to retrieve the configuration of the interface.

pi@raspberry:~ $ ifconfig

According to the output of this command, the wireless interface is given an IPv4 address of 192.168.1.8. This is how to configure the wireless connection using terminal commands.

We can do the same using the GUI of the OS which can be accessed using the startlxde terminal command. After the GUI opens, there will be an icon for the network connections located inside the right part of the OS bar. Clicking on it will open a menu of the available wireless networks according to the next figure. Clicking any of the items will open a window that allows you to enter the password. By entering the password for the network with SSID “TEData_864A” and entering the “ifconfig” command, the result will be identical to that shown in the previous figure.

At this point, we have both the PC and the RPi connected wirelessly to the router. The current network is illustrated in the following figure. We can now remove the Ethernet connection and just use the wireless one between RPi and the router. 

2. Connecting a USB Camera to RPi

RPi has its own camera module that can be used. Rather than buying this module, which might be costly to many, we can just use a USB camera which is might be available everywhere at less cost. The configuration of such a camera is very simple. Just connect the USB camera to one of the USB ports of the RPi. After doing that, we can check whether the camera works well or not by capturing images using the “fswebcam” package. At first, we need to install that package using the following command:

pi@raspberry:~ $ sudo apt-get install fswebcam

After that, we can use it to capture an image. A very basic way to do that is as follows:

pi@raspberry:~ $ sudo fswebcam test_image.jpg

This accesses the camera, captures an image, and saves it to the SD card in the current directory with the name “test_image.jpg”. The output after executing the command is given in the following figure.

Here is the captured image:

3. Capturing Images using PyGame

The “fswebcam” package is useful for quickly testing whether the camera is working well or not. After making sure it is functioning well, we can start building a Python script that accesses the camera to capture images continuously using the PyGame library. The following code uses PyGame for capturing a single image, opens a window for displaying that image, and finally save such image.

import pygame
import pygame.camera

# Captured image dimensions. It should be less than or equal to the maximum dimensions acceptable by the camera.
width = 320
height = 240

# Initializing PyGame and the camera.
pygame.init()
pygame.camera.init()

# Specifying the camera to be used for capturing images. If there is a single camera, then it have the index 0.
cam = pygame.camera.Camera("/dev/video0",(width,height))

# Preparing a resizable window of the specified size for displaying the captured images.
window = pygame.display.set_mode((width,height),pygame.RESIZABLE)

# Starting the camera for capturing images.
cam.start()

# Capturing an image.
image = cam.get_image()

# Stopping the camera.
cam.stop()

# Displaying the image on the window starting from the top-left corner.
window.blit(image,(0,0))

# Refreshing the window.
pygame.display.update()

# Saving the captured image.
pygame.image.save(window,'PyGame_image.jpg')

Assume that the above code is saved in a Python file named “im_cap.py”. To execute such code, we can issue the following command from the terminal:

pi@raspberry:~ $ python3 im_cam.py

Here is the window displayed after executing such file.

We can modify the previous code to capture more than one image. For example, we can use a for loop to capture a number of previously specified images. We can also use a while loop that is not limited to a number of images. Here is the modified code that captures 2,000 images using a for loop. 

import pygame

import pygame.camera



# Captured image dimensions. It should be less than or equal to the maximum dimensions acceptable by the camera.

width = 320

height = 240



# Initializing PyGame and the camera.

pygame.init()

pygame.camera.init()



# Specifying the camera to be used for capturing images. If there is a single camera, then it has the index 0.

cam = pygame.camera.Camera("/dev/video0", (width, height))



# Preparing a resizable window of the specified size for displaying the captured images.

window = pygame.display.set_mode((width, height), pygame.RESIZABLE)



# Starting the camera for capturing images.

cam.start()



for im_num in range(0, 2000):

    print("Image : ", im_num)



    # Capturing an image.

    image = cam.get_image()



    # Displaying the image on the window starting from the top-left corner.

    window.blit(image, (0, 0))



    # Refreshing the window.

    pygame.display.update()



    # Saving the captured image.

    pygame.image.save(window, './pygame_images/image_' + str(im_num) + '.jpg')



# Stopping the camera.

cam.stop()

Here are 8 captured images. Note that the camera position is changed a bit.

4. Building the Background Model

Up to this point, we successfully built a simple surveillance system in which a camera captures images which are saved in the SD card of RPi. We can extend that to automatically detect changes to the scene. This is done by building a background model for the scene. Any change to such a model will indicate a change. For example, if someone is passing through the scene will cause a change to the background.

The background model can be simply created by averaging multiple captured images to the scene background without any object in them. Because we are interested in the color information, the images will be converted into binary. Here is the Python code used to build the background model.

import skimage.io

import os

import numpy



dir_files = os.listdir('./pygame_images/')



bg_image = skimage.io.imread(fname=dir_files[0], as_grey=True)



for k in range(1, len(dir_files)):

    fname = dir_files[k]

    im = skimage.io.imread(fname=fname, as_grey=True)

    bg_image = bg_image + im



bg_image = bg_image/(len(dir_files))

bg_image_bin = bg_image > 0.5



skimage.io.imsave(fname='bg_model.jpg', arr=bg_image)

skimage.io.imsave(fname='bg_model_bin.jpg', arr=bg_image_bin*255)

Here is the background model in both gray and binary after averaging 500 images.

5. Detecting Changes to the Background Model

After building the background model, we can test a new image to check if there is a change to the background or not. This is done simply by converting the new image into binary. Then the number of white pixels is compared in both images. If the number exceeds a given threshold, this indicates a change from the background. The threshold changes from scene to scene. Here is the code used for testing a new image.

bg_num_ones = numpy.sum(bg_image_bin)
 
test = skimage.io.imread(fname="./pygame_images/image_800.jpg", 

                         as_grey=True)

test_bin = test > 0.5

test_num_ones = numpy.sum(test_bin)

print("Num 1s in BG   :", bg_num_ones)

print("Num 1s in Test :", test_num_ones)

if(abs(test_num_ones-bg_num_ones) < 5000):

    print("Change.")

Here is a test image in both color, gray, and binary in which there is a change from the background due to the appearance of an object (person) in the scene.

6. Building a Simple Circuit that Lights a Led When a Change Occurs

As an indication of a change to the background model, we can build a simple circuit in which a led lights when a change occurs. This circuit will be connected to the GPIO (General Purpose Input Output) bins of the RPi. The circuit needs the following components:

• One breadboard.
• One led.
• One resistor (more than or equal to 100 ohms). I use a 178.8 ohms resistor.
• Two male/male jumper wires.
• Two male/female jumper wires.

It is recommended to test the circuit before connecting it to the GPIO pins. This is because if the resistor value was not selected properly, this might lead to not only burning the led but also damaging the GPIO pins. To do the test, we need a battery for supplying the breadboard by power. Here is the circuit after connecting all components correctly.

After that, we can remove the battery and connect the breadboard the GPIO pins of RPi. Based on the breadboard numbering of the GPIO pins, the ground is connected to bin number 20 and the high voltage is connected to the output bin number 22. The following figure illustrates the connections between the breadboard and the RPi. The RPi is also connected to both the charger and the USB camera.

The output GPIO bin is controlled using the Python script given below. Its default state is LOW meaning the led is turned off. When there is a change to the background, the state will be changed to HIGH meaning the led is turned on. The led remains on for 0.1 seconds then its state returns back to off. When another input image is different from the background, the led returns back to on for another 0.1 seconds.

import time
import RPi.GPIO
import skimage.io
import numpy
import os
import pygame.camera
import pygame

#####GPIO#####
# Initializing the GPIO pins. The numbering using is board.
RPi.GPIO.setmode(RPi.GPIO.BOARD)

# Configuring the GPIO bin number 22 to be an output bin.
RPi.GPIO.setup(22, RPi.GPIO.OUT)

#####PyGame#####
# Initializing PyGame and the camera.
pygame.init()
pygame.camera.init()

# Captured image dimensions. It should be less than or equal to the maximum dimensions acceptable by the camera.
width = 320
height = 240

# Preparing a resizable window of the specified size for displaying the captured images.
window = pygame.display.set_mode((width, height), pygame.RESIZABLE)

# Specifying the camera source and the image dimensions.
cam = pygame.camera.Camera("/dev/video0",(width,height))
cam.start()

#####Background Model#####
# Reading the background model.
bg_image = skimage.io.imread(fname='bg_model_bin.jpg', as_grey=True)
bg_image_bin = bg_image > 0.5
bg_num_ones = numpy.sum(bg_image_bin)

im_dir = '/home/pi/pygame_images/'

for im_num in range(0, 2000):
    print("Image : ", im_num)

    im = cam.get_image()

    # Displaying the image on the window starting from the top-left corner.
    window.blit(im, (0, 0))

    # Refreshing the window.
    pygame.display.update()

    pygame.image.save(window, im_dir+'image'+str(im_num)+'.jpg')
    im = pygame.surfarray.array3d(window)

    test_bin = im > 0.5
    test_num_ones = numpy.sum(test_bin)

    # Checking if there is a change in the test image.
    if (abs(test_num_ones - bg_num_ones) < 5000):
        print("Change.")
        try:
            RPi.GPIO.output(22, RPi.GPIO.HIGH)
            time.sleep(0.1)
            RPi.GPIO.output(22, RPi.GPIO.LOW)
        except KeyboardInterrupt:  # CTRL+C
            print("Keyboard Interrupt.")
        except:
            print("Error occurred.")

# Stopping the camera.
cam.stop()

# cleanup all GPIO pins.
print("Clean Up GPIO.")
RPi.GPIO.cleanup()

The following figure is shown one input image which is different from the background image due to the existence of a person. As a result, the led will be turned on for 0.1 seconds.

The following video (https://youtu.be/WOUG-vjg3A4) shows the led state for multiple frames captured using the camera. The led is turned on when the input image is different from the background model according to the used threshold.

For More Details

Ahmed Fawzy Gad, “Building an Image Classifier Running on Raspberry Pi”, September 2018, https://www.dhirubhai.net/pulse/building-image-classifier-running-raspberry-pi-ahmed-gad

For Contacting the Author

• E-mail: [email protected]
• LinkedIn: https://linkedin.com/in/ahmedfgad/
• KDnuggets: https://kdnuggets.com/author/ahmed-gad
• YouTube: https://youtube.com/AhmedGadFCIT
• TowardsDataScience: https://towardsdatascience.com/@ahmedfgad