Software Testing in the SAFe Environment

tl:dr

Agile frameworks put an emphasis on teams and roles. I'd like to argue for the recognition of personal accountability in an organizational environment labeled with "Built-in Quality" as it is called in SAFe.

To have some fun I will compare a SAFe environment to an utopian society, because of a famous question of Thomas Morus himself, hearing from the ways of the people of "Utopia". This question could also be asked about the people working in a scaled up agile organization.?

We should base our organizational design of software production on this question. I call this fundament the concept of "Unlikely Collaboration".?

Preface

There is a moment in every QA Managers life were he or she is willing to write about software testing in a SAFe context. And full disclosure at this point: I don't work in a SAFe context. So my views are not based on my own experience. To be honest, I like to think about organizations from a surveyors point of view, taking first hand reports from others into account. That provides me with enough distance. Being a part of a group always means that there is a risk of losing the ability to describe ones own situation. Also as it is most probably noticeable, I'm not an English native speaker, so please indulge my tries to reach more people, than I would do by using German.

The SAFe work environment as an utopian society - The Concept of "Unlikely Collaboration"

Since Platos politeia, and even before that, people always try to come up with fantasy organizations. In the beginning there was a trend to picture ideal organizations or whole societies often somewhat separated from every other known society or even far in the future but maybe just far away ("Utopia" = no place). Over the years and different authors, things changed to the worse. In my day and age we liked to tell stories of "dystopian" (= bad place) futures often extrapolating terrifying developments of our presents. In parallel the ideas of social improvement (in this words sense the political ideas) toned down from revolutionary quarrels to the realization that the ability to describe the social life is not necessarily the best basis to change it (looking at you Marx).? This realization of the difficulties of changing organizations as fast as possible has often not yet prevented the management desire to do exactly that. And in the case of this text we'll look on a specific context, meaning the collaboration of Software Developers and Software Testers in a scaled up agile framework. The tale of a better organization is here called "Built-in Quality" and it's thankfully available for everyone, being part of the SAFe epos, which is liked and received very well, myself included.? But of course it's a scheme, it's not a real place (got it?) and unfortunately everyone has to implement it and there the real work begins. So in this moment you just got told about this society of developers, testers, product owners and all the shiny value they are about to produce all day long. Like Thomas Morus got told of the Utopians by a Portuguese explorer. And interestingly, before the real report of Utopia begins, Morus starts with interviews done by his associates and himself with an Portuguese explorer in which they start to debate fundamental differences between the utopian society and the European monarchies. They all know Platos politeia so they of course ask about the philosophers as leaders and penalties for certain crimes etc. before ending at the question of property. They are told the Utopians don't know any concept of personal property to which Thomas Morus gives his now famous answer, questioning the personal assurance of members of the utopian society about all the other members doing their job.?

At mihi inquam contra uidetur, ibi nunquam commode uiui posse, ubi omnia sint communia. Nam quo pacto suppetat copia rerum, unoquoque ab labore subducente se! utpote quem neque sui quaestus urget ratio, et alienae industriae fiducia reddit segnem.

("On the contrary," answered I, "it seems to me that men cannot live conveniently where all things are common. How can there be any plenty where every man will excuse himself from labour? For as the hope of gain doth not excite him, so the confidence that he has in other men's industry may make him slothful.)

So were is the relation to the SAFe society might be asked? Every employee gains something out of his or her work and it's at least money. But it this a direct gain? Our employment is only indirectly connected to the value the company is producing (and also this isn't always the case) but the outcome of our personal work often is not. We don't get paid by the number of bugs we've found. In fact we made a deal to exchange our efforts (sometimes more our presence in the Office) with money. We can rearrange this deal when we can convince others that our work is more valuable, but there are several ways to do this and that doesn't necessarily includes to work harder. As a matter of economical fact, when personal gain can't be expected, it would be rational to work as less as possible to get the already agreed upon exchange of money with time.

Let's take an even more enhanced look on this indirect relationship of personal efforts and value. Agile frameworks rightly visualize a value chain and distribute roles and teams on it. In this context everyone might experience the dilemma of either being a top performer in a team or quite the opposite. And the more the team is agile (which often also means cross functional) it's even harder to assess individual performance. Software development is a highly collaborative effort these days. This leads to the difficulty of assessing individual participation. Also in these utopian society, everyone has his or her role, everyone is placed in the value chain. When everyone does his or her Job properly it works. And in the opposite case it sometimes also works.?

So how can I trust that my colleagues actually want to do their job? I can't and this means I should always think about any organization under the guiding principle of "Unlikely Collaboration". As any design of organization not be driven by this principle will, in my opinion, prevent the successful implementation of any scaled up agile framework. Agile software development is especially prone to an expected high motivation of collaboration. But there is really no reason to expect this. Teams are not intrinsically motivated to collaborate, especially in a situation of indirect personal gain (of which a lack of personal property is certainly an utopian extreme, but look on the state of new work offices).?

In the context of SAFe, I'll think about the implementation of the "Built-in Quality" approach. And once again we're blessed by the openness of the SAFe Framework providing us with five approaches to reach this "Built-in Quality". But in contrast to SAFe I'll now try to apply the principle of "Unlikely Collaboration" to them.

Agile Testing - the new Deal

SAFe seems to want everything, every tried and tested agile testing principle baked into a super best practice. A collectively owned, left shifted and fully automated pipeline designed way before the software development even started to really "Built-in Quality". A magnificent building, a symbol of sovereignty over the flaws of men. A vision to pursue or more "bought-in" (depends on the agility of the consulting company) than "built-in". But the idea of everyone pursuing this is unlikely, it's "Unlikely Collaboration", nobody wants to do the work of somebody else, unless this would gain him or her an advantage.

The SAFe "Built-in Quality" principle basically promotes to test early (shift left) and often (automated) and together (Pairing, collective Ownership), which is fine. So let's combine these principles and apply them in to an incentive boosted collaboration. It could be pitched like this: You want your software development and test teams to collectively built-in quality instead of test it in afterwards? Then don't just embed some test people in the development teams, but integrate testing into development! But how exactly will that work you might ask. Here are some simple steps to follow:

1. Take a significant part of testers out of the development teams again.
2. Let QA people develop automated tests machines which can be used by the developers right before they merge code. These test machines will then gradually get enhanced to increase coverage, improve their result reports or their running times. (Beware: executing this many tests will transfer a responsibility of operations to the QA team and alter their skillset).?
3. So the developers basically are testing now, while the old testers design and develop a testing toolbox which helps a developer to assess the impact of his or her merge. When there is a negative impact (for example on performance) the developer immediately knows that his or her new code is the reason. This produces direct feedback and gets rid of the bug cycle (testing, transfer a found bug back to the developers, debugging, fixing, testing) because the root cause for the bug has to simply be in or at least related to the newly introduced code.?
4. Now you will prevent bugs so watch out for "we found this many bugs" KPIs and adjust them. Devs will get addicted to these test machines and trigger them more and more until the daily complains about an unbearable CI becomes a common thing.?But at the end of the day this will increase the likeliness of collaboration by being dependent on another.?
5. Don't let yourself dragged away by this way of testing. Manual testing still can and should really push a product, it's almost wasted by just pushing a release.?

Agile Test Managment - a working Combination?

There is no test management role in SAFe schemes. And the need for traditional test management seems to be decreasing from huge test centers of excellence to rather small manual test teams, maybe not even that. Sometimes there are different test organizations living in peaceful harmony at the same time (embedded QAs, solution QAs, integrated QAs, dedicated QAs, QAs in Ops, Process QAs, the list can get long). Nowadays this is often not designed. Most test teams find themselves to be managed by anyone but a test manager. They can be attached to the development for example. Sometimes some kind of project management organization coordinates them as they are a part of a project release and their test phase has to be scheduled somehow. Not seldom test teams are a specific species, a product of the organizational evolution of a software company. How these teams are developed, should actually be part of a QA strategy because there are a myriad of possibilities to ensure and improve product quality and few should be picked depending on the product. Unfortunately software QA still changes and evolves dramatically. So making up ones mind about the right strategy to keep up with this needs a lot of specific experience and skills. And (coming back to the beginning) to really implement the principle of "Built-in Quality" requires a bit of a visionary strategy, because it won't just work tomorrow or next month. Pursuing a QA strategy can take years, so the goals of that strategy should be safe bets. In my opinion this is only achieved by making someone?accountable as a test manager and here are again some steps to follow:?

1. Find a specific kind of software tester, to be precise, the kind of tester who manages catastrophes. (I know QA should prevent that and we all know the other kind of testers, who always say "told you so" when things went south).
2. This person has learnt to adjust how things are to be done in the future. And I'm afraid, that's a necessary skill. To be able to do that this person will most likely anticipate not just what can go wrong but also what has to be done in this case. Let them present these ideas to you.?
3. Give them the opportunity to build organizations (not just choose people for different roles) and take credit as well as?accountability by officially appointing them.?
4. Wait the maximum possible amount of time. And please expect trouble. But please also make sure that this person is able to handle it.?
5. Enjoy having someone who's is able to tell you what's up with the quality of your products and is also able to tell all the others what to do about that.??

To come to an end here I have to admit that there is a lot more to tell besides just these two presented points. So I might continue, whenever anyone is interested in me doing that. I just wanted to point out that the recognition of "Unlikely Collaboration" as well as the recognition of personal responsibility (especially in my domain of software QA) are the most beneficial ones.?