FRS theorem of test automation
A biggest lesson from creating test automation suites for over 10 years across different stacks is, that you cannot have all of these 3 facets in the same test.
Fidelity: How closely does a test simulate end user behavior
Reliability: How much trust a test can provide for its results
Speed: How fast does a test finish execution
Taking inspiration from CAP theorem from computer science, this theorem posits that a test cannot have all these three facets.
FRS theorem
It is improbable to create test automation that is fast, reliable and realistic.
Speed and reliability without fidelity
Well written unit tests are blazing fast and reliable but not really what an end user would see. You can typically execute 1000's of them within minutes and unless there is real issue, these test rarely fail.
These are bottom of the pyramid tests which form the foundation for great software.
Fidelity and reliability without speed
UI tests are the closest to end user experience and they are typically very slow, they take about 10-30seconds to finish booting up and running a test. It takes a lot of effort to make them reliable but you cannot speed them up.
You can sacrifice fidelity a bit more and mock the data and achieve reliability and speed with component tests. These are my favorite tests and most underutilized, they still run through the UI but with pre-determined and test-controlled data.
Speed and fidelity without reliability
A test suite that is has artificial time delays(sleep), improper cleanup, order dependent. You do not want to inherit these tests. They are harder to maintain and stops to give confidence.
These are what you typically call flaky tests. E2E tests usually become flaky over time.
Conclusion
So, next time when you are coming up with a test strategy ask yourself (and the team) what are the facets you are optimizing for.
Likely it will be all of them, and yes that's a perfectly valid answer, that's why you should have all kinds of tests that are intended for achieving their goals.
What do you think of this FRS theorem, does it sound legit? have you ever had tests that contradict this?