Some Test Organizational Challenges Faced - It's Never Been Too Late To Learn!

One part of what we do, at Last Mile is to meet with organisations, and spend time with their testing teams (leadership to execution folks) with a view to getting insights into the issues and problems they face. These discussions are usually open discussions, spent understanding their organisation, and sharing our views and observations, having helped the testing functions in large System Integrators, Captives and smaller product organisations in overcoming different challenges.

Recently, we had a chance to speak to the test leadership team of a key program, in one of the largest investment banks, (as also one of the Top 20 Private Companies) in the US.

The free-wheeling discussions of that day with some very mature and sharp testing minds led us to put down a few insights on the typical problems faced and our solutions to them:

• In their rush to get products out to the market faster, and under the pressure of fortnightly sprints, product companies are focusing less and less on customer experience led e2e tests. Their tests are more about assuring that functions work in the way typical users would use them (what I term the happy path), rather than thinking through different and diverse scenarios. Fail fast and fix faster are the two mottos that keep them going. Their challenges usually start from the time customer experiences begin to fail, and they then turn around to trying to understand why the test function didn’t catch those issues on time! What they need to do, early enough, is to align their tests to how the customer journeys across their application or product.
•  Most large organisations are dissolving large QA teams and integrating them into smaller more focused product teams. The challenges that they face, are those of “Agile” alignment, taking end to end QA ownership and driving change across not just the QA function, but across the business and development functions too. So how do they effectively integrate, how do they take on that ‘test’ responsibility, how do they manage the test effort better, is where our expertise comes in and for which we evolve solutions.
•  The changing role of the tester, and the expectations on the role being more “technical”, more “automation” aware constantly keeps coming up. Our observation was that most testers do what the functionality (or the developers) tells them to do. They don’t look beyond what the functionality is NOT supposed to do. The adjectives attached to testing – non-functional, end-to-end, integration etc etc, make testers draw silos around what they actually do. No one is a “tester” any more! And here is where, honing the tester’s capability, by bringing the tester focus back on effective test design, and thinking holistically about the product, is where we add value. Lakshmy Usha articulated this well in her article here.
•  Test automation and the focus on it led to another observation… Knowledge of Selenium or any other tool, doesn’t an automation mindset make. Most automation testers lack the ability to think about what should be automated, and why, and how early they can get involved in the automation. Our experience has been that someone from a development background, with sound knowledge of programming practices and a deep appreciation of testing, is a better automation architect. And this is the quest we help some of our clients in.
•  A lot of companies, especially the large ones, are still struggling with aligning releases. Unlike telcos or even retail organisations, which release end to end functions, financial institutions still make releases of front end / back-end / middle-office systems. Not all of them align themselves to end to end functions. How they could align to integrated end to end releases, and thus gain improved end-to-end ustomer experience led test focus is an area involving major effort and change management.
•  Test metrics are still an area of concern. At one level, most organisations track test coverage or test defect metrics and compare it with subsequent test stages, without really comparing it to end customer feedback. So, whilst most test organisations claim great Defect Removal Efficiencies, their customer feedback scores are still showing little or no improvement. So how effective have these test metrics been in aligning to the business goals? The focus of metrics as a way to align with true business intent and not generic quality measures would be a better of way of seeing the true worth of the test organization.
•  Lastly, companies that drive highly efficient development & test teams, have a way of aligning their teams to the customer goals with razor-sharp focus on those goals. One of the largest TPAs in India, we met, had a team of developers and testers so well aligned, that I spent most of my ‘pitch’ time understanding what they did, and setting up time to meet them to learn more!

We believe, no customer’s problem is the same, the adaption to the organization or program or product context, always makes it unique, and is a huge learning for us too! It is this wealth of accumulated learning and experience that we try to bring to the table when dealing with any customer’s problems.. Tough problems, innovative solutions !

 Drop us a line, and we'd be happy to come by and meet up with your teams too.

This article also appeared on this Last Mile Insights Page