Machines That See - Part 1

Welcome back, everyone! I'm thrilled to share with you our latest newsletter, part one of a two-part series, focused on the practical applications and intricate challenges of computer vision technology. It's been a fascinating journey to compile insights and predictions for these two editions, and I trust you'll find them as enlightening as I did in authoring them.?

Computer Vision Technologies

Computer vision is a field of artificial intelligence that enables computers and systems to derive meaningful information from digital images, videos, and other visual inputs, and to make decisions or take actions based on that information. In other words, it allows machines to interpret and understand the visual world in a manner similar to human vision.

A diverse array of devices are available for generating the requisite data for computer vision. Below are just a few examples.

• Cameras: The closest counterparts to human eyes, capturing two-dimensional images or videos.
• Lidar: Utilizes laser beams for distance measurement, crafting detailed three-dimensional environmental maps.
• Infrared Sensors: These detect infrared radiation, useful in motion detection and temperature differentiation, especially under low-light conditions.
• Depth Sensors: Technologies like structured light cameras and time-of-flight cameras measure distances to objects, offering a three-dimensional scene perspective.
• Ultrasonic Sensors: By emitting ultrasonic sound waves and analyzing their reflections, these sensors ascertain object distances.
• Radar Sensors: These project radio waves to detect and gauge object speed based on reflections.
• Thermal Cameras: These sensors are capable of imaging based on temperature variances by detecting object-emitted heat.
• 3D Camera Systems: Integrating multiple sensor technologies, these systems provide a comprehensive three-dimensional environmental depiction.

The challenge for engineering and product leaders lies in selecting the most fitting sensor technologies for their specific application needs. While complex solutions like autonomous vehicles may incorporate a wide array of sensors, simpler applications, such as automated vacuum cleaners, use one or two sensors, such as lidar for obstacle avoidance and edge detection.

RightHand Robotics' RightPick 4 system harnesses an advanced 3D camera system to enhance performance and reliability. Designing this system required close collaboration between product and engineering leaders, ensuring that requirements were well understood, followed by an intense analysis and selection of the available sensors at our disposal.

Real-World Challenges

Object detection within the dynamic environment of a warehouse presents a complex array of challenges. There are many different workflows in warehouses where computer vision can provide value. However, we will narrow our focus to order fulfillment, a critical operation where the precision and efficiency of computer vision can significantly impact productivity.

Range of Items: Warehouses store a vast range of items, each with unique physical characteristics. Vision systems must adeptly handle objects that have never been encountered before. Adding to this complexity, product packaging undergoes continuous evolution, altering visual characteristics that our systems must recognize and adapt to seamlessly.

Lighting Conditions: Lighting within warehouses seldom offers ideal conditions for computer vision. The high placement of lighting fixtures, variability in intensity, and potential for shadow-casting obstacles all demand a robust solution capable of performing under suboptimal light.

Product Storage: The nature of containers where product is stored, whether cardboard boxes or plastic totes, introduces another layer of complexity. These containers have diverse attributes:

• Varied colors and dimensions.
• Varied rim widths.
• The presence or absence of flaps in cardboard boxes.
• Design patterns or perforations that could confuse vision sensors.
• Some contain many of one specific item, others contain multiple items, separated by subdividers.

What other unique challenges can you imagine encountering in object detection?

Until Next Time

As we wrap up the first installment of our two-part series about the impact of computer vision technology in warehouse operations, it’s evident that there are many complexities in solving the task of object detection. From choosing the right sensors to navigating the ever-changing landscape of warehouse environments, the path is both challenging and exciting.

Looking ahead, the next issue promises to deepen our exploration into these challenges. Featuring exclusive insights from RightHand Robotics' engineering team members, we will explore real-world applications. Expect to uncover the nuanced decisions behind our cutting-edge 3D camera system and the innovative strides we're making in computer vision and machine learning to refine object detection and handling.

As we say at RightHand, Onward Robots!