Automating everything with Pi
Written by Justin Powell, Senior Edge Computing Engineer at Captur
The goal of any product-based automated test system is to best replicate the customer journey and how that customer interacts with your solution. Here at Captur, we provide a solution that deploys advanced ML models to edge devices; our users mobile phones.
When considering the requirements of our own automated test system we wanted to test the product in the same way our users would, therefore we had the following goals in mind:
The answer? A Pi!
Why Pi?
The Raspberry Pi is a low-cost computer with a small form factor and add-ons for a wide range of automation applications.
A low-cost solution allows us to de-risk the development of our test system without needing large upfront investment while a small form factor ensures that we can easily deploy our setup in any environment.
The Automated Test System
领英推荐
We use a Raspberry Pi 5, chosen as it provides the highest performance available for running our test system. We are using a Samsung S22 Android device installed with Capturs’ latest app and SDK.
Our test system is written in Python using the Pytest framework and we use Appium for controlling the UI of our mobile devices under test.
We use toy scooters and bikes ‘parked’ in front of our Android device to best replicate our users' experience. When a user ends their ride our ML models will determine whether the scooter is placed upright, obstructing anything and ultimately parked safely.
Running a single test takes around 30 seconds allowing us to test early and test often on any future improvements made to our SDK before they are released to customers.
Challenges
While there are many benefits to deploying an automated test system with a real device there are also some challenges.
You must provide a robust setup and teardown for each test as there can be any number of UI interactions that need to be undone to allow the next test to start from a known state.
There is a greater need to control and log details about your environment as devices will attempt to update and patch without user interaction, leading to ‘hidden’ changes between test runs that may change their outcome.
And lastly, scale, while real devices provide real value we must compromise by selecting a smaller number of devices to give the widest possible coverage.
Next Steps
One of the bigger challenges we will tackle next is how we control our test environment such that we can replicate a number of different user scenarios. We have ideas to use a Pi-based camera to provide feedback on the environment and even a robotic arm to manoeuvre our mobile device around our test scooters like a user would - watch this space! ??
Senior Product Manager | Climate & Mobility | Machine Learning | Building better futures
3 个月That's a nice use for a (minced) pi. Would be interesting to think how could you account for real world lighting conditions. We also tested for adversarial cases such as holding a still image or objects - at one point our model determined that baby Yoda was correctly parked!
Product Designer
3 个月?? ?? ??