Auditing Agile Projects & Agile Auditing- A primer

Agile was always used as a synonym to fast, efficient and highly adaptive way of doing things.Many audit departments have started the transformation journey to make their audits Agile. The concepts of daily stand-up meetings lasting 15 minutes, continuous engagement with the stakeholders during the field audits, managing sprints to release observations/findings to the auditees, agreeing on the proposed corrective action plans as part of the releases, audit schedules getting converted to audit road-maps etc are getting more and more popular with audit teams that adopt Agile methodologies.

But what I am focusing below is not on how audit departments are becoming agile. I am sharing my experience on how auditors are getting prepared to audit agile project methodologies.

When I started auditing IT projects, Phones were still being used only for making calls, Cookies were something to eat and Clouds just meant that rain  was not far away (feels this was a century ago !!!).

Back then life was simpler for a projects’ auditor. Starting from the strategic alignment of the project to business goals, reviewing sign-offs of the scope documents, validating the structured project plans, reviewing the test cases against the documented scenarios and finally assessing whether the end product (Project) had achieved the envisaged business benefits. All followed the seamless tranquility of still water rather than the rough and tough feel of a waterfall which was what the process was called those days.

Later on, when Agile methodologies became more and more popular among IT colleagues, the traditional IT Auditor in me started doubting them, which was nothing but an act of resisting change. Accusations and questions like “you guys do not have adequate documentation”, “where is your project manager?” where are the PMI defined project management processes? ”, “this process is going to create weaker controls”, “aren’t these repetitive iterations a waste of shareholder’s money?” etc reverberated in my mind whenever I heard about a new technology project following the Agile/Scrum methodology.

All this until I myself learned Scrum to understand what the guys are really talking about.

Now I understand that an effective auditing of agile projects is definitely possible and will add value to the organisation, if and only if the auditor understands clearly what he is auditing. (This is true for any audit for that matter). Here are some of the steps I recommend to follow when auditing a project that follows Scrum methodology.

1. Validate how the Project strategically aligns with the business goals.
2. Interview the Product Owner (he is your man!!) and the Scrum Master to understand the project and the methodology. Ensure that consistent application of agile practices were followed.
3. Review how the business needs have been translated into product vision and then into road-maps.
4. Assess the effectiveness of sprints and review the release plans.
5. Look into how the backlogs are managed.
6. Risk rank the User stories and deep dive into high-risk controls. Focus more on those sprints.
7. Review the selection of team members and the common understanding of the methodologies.
8. Interview stakeholders to assess their involvement, especially with respect to the acceptance criteria.
9. Keep an open mind to see the changes in scope throughout the life cycle of the project. The scope creep which was a dreaded thing in traditional methods is just a BAU (Business As Usual) in Scrum.
10. Review other critical aspects of the project like cost, contracts and quality.

Even after all these learning and adoption of the new methodologies, I will still be a bit apprehensive to take “User Stories” as my base documents for the assessment. This is simply because we as auditors have been hearing lots of “stories” to justify the control gaps and weaknesses from our auditees that anything referring to as a “story” still triggers the sceptic in us.

But we will get there one day, because we are second to none in embracing changes and mastering the new terminologies.

Biju Nair

Dubai, 13 June 2019