Legacy Systems Are Everywhere. How to Build Automations Within Them?

Automating legacy systems is increasingly essential for companies seeking to streamline processes, but these systems often rely on outdated technology that complicates automation efforts. In such cases, UiPath’s Computer Vision offers a powerful toolset to interact with these legacy interfaces. However, to achieve a resilient and reliable automation solution, we often need to make a series of image adjustments, including grayscale conversion, zoom, and additional enhancements to ensure accurate element detection.

First Things First... Do you know what Computer Vision is?

Computer Vision (CV) enables machines to interpret visual information, allowing automation tools like UiPath to "see" and interact with on-screen elements. This is especially useful for automating legacy systems or virtual environments where traditional selectors don’t work. CV uses machine learning to recognize buttons, fields, and menus by analyzing pixel patterns, making automation more adaptable and resilient.

Now, that you are more familiar with Computer Vision concepts, let's keep diving into this amazing feature!

How to use Computer Vision in UiPath?

To use Computer Vision in UiPath, follow these quick steps:

1. Install Dependencies: Ensure the UiPath.UIAutomation.Activities package is added to your project.
2. API Key: Get an API key from Cloud UiPath Orchestrator under Licenses > Computer Vision. This allows access to cloud-based image processing.
3. Use Local Server (Optional): To process images locally, enable "Use Local Server" in the activity settings and install the package UiPath.ComputerVision.LocalServer on your environment. This avoids cloud use, improving security and reducing latency.

After all settings done, you'll see some options available on your Studio. We have so many Computer Vision actions to use, but before using it, we need to map the screen that we want to automate by using CV Screen Scope activity.

Here we have the screen mapped and all of the elements detected using UiPathScreenOCR whithin CV Screen Scope.

Challenges in Automating Legacy Systems

Legacy systems can present numerous obstacles for RPA due to old software interfaces, low screen resolutions, and a lack of accessible APIs. These systems require precise element detection, and even small changes in the screen can disrupt the automation. In these cases, UiPath’s Computer Vision helps by visually recognizing elements, but even with this tool, optimizing images can improve accuracy and stability significantly.

Optimizing Images for Better Detection with Grayscale Conversion

Converting an image to grayscale is a simple yet effective way to improve element detection. By removing color information, grayscale conversion focuses on shapes and contrasts, which can be beneficial when Computer Vision processes the image. This approach can especially help when dealing with interfaces that contain complex backgrounds or low-contrast text.

After running this code to enhance image quality, we can see the difference and improvement.

In the enhanced image of the legacy system interface, we applied several adjustments to improve accuracy for the robot. By converting the image to grayscale and boosting contrast, we removed unnecessary color details, helping the Computer Vision algorithm focus on the main elements like buttons and fields. Additionally, reducing background noise made the interface cleaner and more defined. These improvements make it easier for the robot to accurately identify and interact with specific parts of the legacy system.

Additional Image Processing Techniques for Enhanced Precision

Beyond grayscale conversion, other techniques can further optimize images for reliable Computer Vision detection. Some options include:

• Contrast Enhancement: Increasing the contrast can make specific elements stand out more sharply, helping Computer Vision identify them.
• Noise Reduction: Removing background noise, such as faint grids or watermarks, can eliminate unnecessary details that might interfere with detection.
• Zoom and Scaling Adjustments: Sometimes, zooming in slightly can help focus on specific parts of the legacy interface, especially useful when working with low-resolution displays.

Applying these techniques can make a significant difference when dealing with complex interfaces. You may experiment with these adjustments based on the challenges presented by each specific legacy system.

How Do You Pass the Image Captured by the Computer Vision Activity to Your Code?

Simple! When the Computer Vision activity captures the system image, the content is saved into a variable, which can then be easily manipulated throughout the process.

After processing the images, you can integrate them into UiPath’s Computer Vision activities to build a resilient automation. Here, you’ll set selectors based on optimized images, ensuring that the bot can reliably recognize elements despite visual inconsistencies. Applying these pre-processing steps to the images beforehand ensures that Computer Vision’s element recognition functions with greater accuracy, even under varied conditions.

Final Thoughts

Automating legacy systems presents unique challenges, but by leveraging image processing techniques like grayscale conversion, contrast enhancement, and zoom adjustments, you can significantly improve the stability and accuracy of UiPath’s Computer Vision automation. Each adjustment makes a difference in how effectively the bot interacts with older interfaces, resulting in a more resilient and adaptable solution tailored to handle even the most outdated systems.