Image Background Removal Using Shutter Stream and Magic Wand

In two earlier articles, entitled Practical Algorithms for Background Removal and Background Removal Using Shutter Stream and Mobile Devices, we discussed several approaches, our teams at Iconasys and DMMD, have taken to remove backgrounds in images and showed how the Iconasys Shutter Stream software can be used in conjunction with mobile devices to shoot and create background removed images. Shutter Stream offers three types of background removal algorithms: (1) magic wand, (2) removal with a background image and (3) chroma keying (sometimes known as green screen removal). For pure white (not transparent) and when objects are shot against close to white backgrounds, Shutter Stream also offers a colors levels-adjustment option. This article takes a closer look at the first option, removing backgrounds using the magic wand tool. Future articles will discuss the other background removal tools.

The magic wand tool takes an input color, often determined by the color of the pixel a user clicks on, and labels as background pixels all the pixels that have similar colors and are, optionally, connected to the clicked pixel. Magic wand can efficiently remove backgrounds as long as the following parameters are met: (1) there is a clear color contrast between the background and the foreground object, (2) object shadows are eliminated or minimized and (3) the background is fairly uniform. Next, let’s look at how the three requirements can be achieved using Shutter Stream, Iconasys’ 360 Photography Turntable Acrylic Risers and Iconasys’ Product Photography Lighting Solutions.

Shutter Stream

A compatible camera or mobile device is connected to Shutter Stream. This will provide a live view of the object placed in front of the camera, as shown in Figure 1. With the camera set to fully automatic mode, users have the ability to control three key settings on the camera: (a) aperture, (b) ISO (c) shutter speed. Understanding these three settings is important in order to achieve maximum contrast between the background and the foreground images. For completeness, let’s briefly review the three settings.

Aperture: Aperture measures the size of the opening through which light enters into the camera and reaches the imaging sensor. It has a direct effect on the following three variables: (a) the amount of light entering the lens, (b) the depth of field and (c) the amount of light diffraction.

First, aperture size is given by the f/number, which is the ration of the focal length to the diameter of the aperture:

f/number = f / Diameter

The larger the Diameter, the smaller the f/number. Aperture settings on the camera are usually marked f/22, f/16, f/11, f/8, f/5.6, f/4, f/2.8, f/2.0, f/1.4 and f/1.0 (in Shutter Stream we only show the number and not the “f/” part, so one sees 22, 16, … 1.4, 1.0). In the above sequence the aperture diameter changes by square root of 2. In the above sequence, the amount of light entering the lens changes by the square of the diameter or by a factor of two each time we go from one f/number to the next. For example, going from f/1.4 to f/2.0 decreases the diameter by square root of 2 and the amount of light decreases by 2.

Second, besides controlling the amount of light entering the lens, the aperture also controls the depth of field, which is defined as the distance from the camera where an object is in focus. Depth of field becomes extremely important when shooting micro or long objects. An example of the effects of aperture on the depth of field are shown in Figure 2.

The requirement that a lens is in focus when all the parallel rays are focused in one point can be relaxed to say that a lens is in focus when the parallel lines converge within an error δ of the actual focus. This is shown in Figure 2. The distance between the start and end locations, for which error δ is satisfied, is the depth of field (DOF). For the continuous red rays, the DOF is ?-small and for the dotted red rays, the DOF is ?-big. Thus, a smaller aperture generates a higher depth of field.

Third, a small aperture size causes more light diffraction, which may decrease pixel resolution, especially for higher end cameras. 

Therefore, when setting the aperture, all three conditions: (a) the mount of light, (b) the depth of field and (c) light diffraction should be considered. We recommend that the aperture is set to be as large as possible (i.e. as small as possible f/number) to keep only the object of interest in focus and ideally the background is out of focus.

ISO (Film Speed): The ISO refers to the sensitivity of the imaging sensor. The higher the ISO the more sensitive the sensor is. A good analogy to sensor sensitivity is shown in Figure 3, with the two glasses of water. The glass is the imaging sensor and the water in the glass it the light charge accumulating on the sensor. Each of the two sensors are scaled between empty (or zero) and full (or maximum value). A higher ISO means that less water will fill the glass, while a lower ISO means that it takes more water to fill the glass.

A few things to notice about the ISO and the analogy with different size water glasses. 

First, if both glasses are divided into the same number of sub-divisions, then smaller variations in the volume of water will trigger different levels, more quickly. As a result, higher ISOs images are more likely to appear noisier, while at the same time, if biased correctly, they will also provide a higher dynamic range. In some cases, especially in textured objects where it’s difficult to determine noise, a slightly higher ISO may provide an additional level of realism.

Second, since the smaller glass will fill up faster, higher ISOs require less exposure times than lower ISOs. This makes higher ISOs good for situations when there’s low lighting (i.e. night) or fast action (i.e. things need to be captured quickly). However, this is an artifact of higher ISOs and it’s not really leveraged in product photography, unless capturing action, such as a splashing liquid.

Shutter Speed: The last component that controls the quality of our images is the shutter speed. This refers to the amount of time the shutter is opened. For product photography, shutter speed can, in most cases, be set towards the longer side of time. However, the longer the exposure times, the more prone the resulting images are to camera shake and environment vibrations, which can blur the resulting image. Therefore, even for product photography, minimizing the exposure time is a goal in order to minimize blur.

Acrylic Risers

In Figure 1, our object is pictured on a white background, a common technique in product photography. However, object shadows are clearly visible, making it difficult for the magic wand tool to remove the background. Using Iconasys’ 360 Photography Turntable Acrylic Risers shadows can be significantly reduced, as shown in Figure 4.

Lighting Solution

In Figure 4, the shadows of Figure 1 have been replaced by image reflections. In order to minimize reflections, we recommend using an active light back panel, such as those available from Iconasys’ Product Photography Lighting Solutions. With the active back panel and the acrylic riser, shadows are completely minimized, as shown in Figure 5.

The final step is to adjust the camera settings in order to separate the background contrast from the objects, as much as possible. Here are our recommended steps:

1.    Set the aperture to be as large as possible (i.e. choose the smallest f/number) such that the entire object is in focus. This guarantees the largest amount of light and the minimum amount of light diffraction. Further, keeping only the object in focus guarantees that the background is more uniform (i.e. blurred out), which is a goal of our photography technique.

2.    Adjust the shutter speed and ISO to create the maximum contrast possible. Ideally, try to keep the ISO as small as possible, but for certain objects, increasing the ISO slightly, may add enough dynamic range, where the object may feel more real, even though it may look slightly noisier.

3.    Apply the magic wand tool, as shown in Figure 6 through Figure 9.

In conclusion, this article described how to use the magic wand tool from Shutter Stream, together with Iconasys’ 360 Photography Turntable Acrylic Risers and Iconasys’ Product Photography Lighting Solutions to efficiently achieve background removed images.

Keywords: #ProductPhotography, #eCommerceImaging, #360Imaging, #360ProductPhotography, #BackgroundRemoval, #BackgroundClipping, #TransparentBackgrounds