Testing Is Not An Afterthought

We often overlook the simple, basic, things in life. In the life of a company that creates software or hardware, testing is one of those most basic things. It’s essential to product quality, development, performance, customer satisfaction and ultimately - a company’s profit. 

But testing rarely grabs headlines and QA teams often run short of equipment while working in some far corner of the company office space. The title ‘QA Specialist’ is rarely awarded the same warm glow of prestige as ‘Software Developer’. But it should be.

Instead of creating a cool new feature, QA personnel specialize in how to find the limits and problems with a feature or product. By its nature, testing is a destructive process. This can give developers and QA teams a false sense of opposition to one another. Testing can seem expensive and requires a dedicated team with experience, talent and sharp instincts. 

For those reasons and more, companies often fail to complete sufficient testing - at great cost to their bottom line. The operational efficiency company, Celerity, succinctly explained it like this: 

“The Systems Sciences Institute at IBM has reported that ‘the cost to fix an error found after product release was four to five times as much as one uncovered during design, and up to 100 times more than one identified in the maintenance phase.’ 

To illustrate: if a bug is found in the requirements-gathering phase, the cost could be $100. If the product owner doesn’t find that bug until the QA testing phase, then the cost could be $1,500. If it’s not found until production, the cost could be $10,000. And if the bug is never found, it could be secretly costing the company money and no one could be the wiser... “

At Black Sage, our software team devotes considerable resources to testing. Not only is the QA team seen as equals, they are present for new feature design discussions and they hold the final word when it comes to merging new code into our code base. They sit, quite literally, in the middle of the software developers. There is one QA specialist for every two developers. 

In our QA world, pedantic is not a pejorative.

As our company has grown, one of our jobs as integrators has become testing of new equipment performance claims against its actual performance. We do this to make sure we are integrating with the best hardware on the market. This testing includes ranges, frequencies, fields of view, direction finding, refresh rates and other specifications. 

Testing in a sterile, simulated environment is not nearly enough. We go to great lengths to test our systems and software in the real working environment, be it urban settings with high signal floors or isolated rural and desert settings.

When it comes to hardware vendors, most are interested in having their sensors and effectors integrated into our CUAS solutions. They frequently provide their CUAS sensors and effectors for us to evaluate. Some of these vendors become our vendor partners.

Given our “around the clock” real-world testing capability, our vendor partners get exclusive views into the performance of their systems. We produce test results they sometimes cannot generate themselves, by no fault of their own. Some vendors lack access to the large physical space required for such testing. Others get the opportunity to see their systems installed and operating as part of our larger solutions - offering a mix of signal environments, networking infrastructure, and use cases that are a hard-to-come-by treasure trove of test data. 

As we improve, so do our partners’ products. 

And it is not just about the equipment testing. For our software, we utilize unit testing for code modules and a vast array of developer tools for debugging code as it is being written to find issues at the lowest level possible. We hold design discussions to root out errors before code is ever written. Our continuous integration system runs a suite of tests against each new feature or release build before a QA specialist even bothers to take a look. 

All new code is peer reviewed by our developers and we execute countless smoke tests of the incoming features on production level hardware. We launch automated regression suites for existing features and the team is constantly executing manual tests and digging out corner cases. There are daily flight tests, full field testing and plenty of test reporting that is not for the faint of heart. And there is always more testing to be added to our ever-expanding arsenal.  

Our comprehensive testing serves as a compass as we explore new features and functionality for our counter unmanned threat solutions. Without this compass, our software and integration teams - and our bottom line - would suffer.