I enable frustrated testers to become trusted advisors who drive meaningful quality improvements. Quality Leadership Coaching | International Speaker | Author | UKITB
Stop calling automated testing 'checking'. One of the things you will have heard as a QA engineer is that 'automated tests don’t test'. The idea behind this is a bit nebulous to me, to be honest, but clearly, it has to do with the assumption that no new information can be gained from running an automated test script. In this school of thought, we’re told that only active human intervention gains this new information. I’ve forever been baffled by this. Automated tests are created by humans, who (hopefully!) think about what risk they are covering, and why we need the information this test provides us with. Because knowing that your signup flow is functional isn’t more or less relevant whether a human checks it or a machine does.
Georgina Shute
10 个月
Hehe robots. What term should we use instead do you reckon?
Oleg D??
10 个月
Interesting and valid point. I’m one of those who preaches that Automated Testing should be rebranded to Automated Checking for the purpose of distinguishing the value brought by a human and the type of testing that they can do utilising critical & lateral thinking vs automated tests that a human creates. This “value” provided by a human I’ve seen diminished by many companies, replacing Testers/QA/QEs with SDETs and SDITs roles. I agree though under the hood that automated tests is just another way to surface information, risks or unknowns and test ideas/hypothesis/assumptions. Do you think there is a better way to fight the human value battle? Especially in the day and age of GenAI
Callum Akehurst-Ryan
10 个月
I understand the division being problematic, however I see it used as an education piece for the industry of needing to test within certainty vs. within uncertainty. A lot of the industry still sees all of testing as confirmatory (or checking) where a test will look for a pre-defined result. This means that the result must be pre-defined, something that not all organisations can do well. Agile teams frequently write more vague asks, leaving a lot unwritten (lots of uncertainty) which needs to be tested for. Having different terms is a way of saying "sure, you need to check what you've asked for BUT there's all this uncertainty you need to test for too." A way of showing that confirmatory testing (where there's certainty) isn't 100% of all testing you need. kind of like how we say Front end and Back end developers... it shows different skills a team needs to be full stack :D https://cakehurstryan.com/2021/08/25/why-checking-an-ac-isnt-enough/
Michael Posluszny
10 个月
Theory: Test Automation serves exactly one purpose, to validate that established behavior isn't broken (aka a regression) because of new changes. Reality: Test Automation is a numbers game that upper management plays so they can report to the C-Suite how many automated tests there are and how often they are executed. Proof: How often does senior management review actual tests, results, and asks about the cause of failures? In my experience, over the past decade and a half, exactly zero.
Angandeep Kr Chatterjee
10 个月
Katja Obring I couldn't agree more with your perspective on automated testing. Your insights truly shed light on the value and relevance of automated tests. Keep sharing your valuable thoughts!
Brijesh DEB
10 个月
I am not sure why are we still debating this? The statement 'automated tests don't test' seems counterintuitive. After all, aren't we still checking functionality with automation? The key difference lies in the level of human involvement. While exploratory testing requires active human judgment throughout, automated tests leverage predefined checks. This doesn't diminish their value. I like to think of it like this: a well-designed automated check is a highly efficient and precise check, ensuring consistent functionality across regressions. It frees up valuable human expertise for higher-level tasks like exploring edge cases and user experience, which are difficult to automate effectively.
Jeff Sussna
10 个月
Bravo! I think that view ignores automated test design as a thoughtful creative software development activity like any other. It may reflect tests written after the fact (like cleaning up after breakfast) instead of within the flow of development.
James Walker
10 个月
I agree. There is an element that the automation script is only as valuable as the assertions it contains which I imagine is the basis of most counter-arguments. It'll be interesting to see how AI can facilitate more 'advanced' assertions, probably similar to a human tester where thousands of implicit assertions (yet not explicitly defined) are performed as part of an execution.
Ryan Millar
10 个月
This whole discussion and contention only exists because 'automation testers' (developers under a different name essentially) are put on a pedestal, and it's created this over-simplified perspective of what testing is outside the testing community. Until those two things are resolved I'm firmly in the camp of sticking to the testing vs checking terminology.
Director of Engineering @ Ada Health ? Innovating & Evolving Leadership ? Engineering, Quality, Regulatory, Agile, Culture, Strategy, Coaching ? Mental Health First Aider ? International Speaker
10 个月Still feels like this is such a hot topic... But looking through the replies, people are still focused on the terminology and not the intent. We'll never get to the end of this debate until people start to focus on the intent. I.e. Rather than talk about whether something is an algorithmic test or an investigative test, talk about the purpose of the test instead, in relation to what you're trying to do about information... - Is the test trying to confirm information that's setting an explicit expectation? - Or is the test trying to confirm information thats related to an implicit or tacit expectation? - Or is the test trying to uncover some information about scenarios related to unknowns (risks, variables, perspectives, etc) you already have an awareness of? - Or is the test trying to uncover a new unknown that you are currently unaware of? Starting to think about the intent of a test helps to mould the approach to take related to that intent (which of course can be blended!), and makes the terminology about whether it's a "scripted, algorithmic test" , or an "exploratory test" less important to worry about