How to handle production issues?

Issue management is of 3 stages.

1. Issue control and application stabilization.
2. Fixing the issue
3. Post-mortem of the issue

Stage 1 and Stage 3 are very important in the issue management process flow. Let us see how to do the stage 1 in this article. We shall see the post-mortem process in the next article.

?

First and foremost, the immediate action is to set the players. This should be done within a few minutes of the issue being reported. Ideally, this should be decided before the release and be on standby.

?The 2 players that need to be set immediately are

1. Technical Analyst
2. Issue Co-ordinator

?The above players can be an individual or group of members depending upon the criticality and size of the issue.

?Technical Analyst -TA:

?TA are those who have worked on the issue related area. This role is generally played by a senior developer with technical and functionality expertise and/or the developer who actually worked on the module where the issue is reported.

?

Issue Coordinator -IC.

This role will be the technology team spokesperson who will communicate and provide timely updates to senior managers, support, stakeholders and other concerned parties.

This role is very important for the TA to be very effective in solving the issue. Due to the pressure, TA might get disturbed by constant status updates about the progress of the issue. Sometimes the same update needs to be given to multiple parties separately which distracts the TA from analysing and fixing the issue. This is where the Issue Coordinator plays a big role. This IC role is generally played by the Project Manager, Team lead or Support member. Typically, the person should be aware of the process, stakeholders, and product and should have good communication.

?

Responsibilities of Issue Coordinator

• Give timely updates to all the stakeholders.
• Collect the impact reported and consolidate the impact received from different channels
• Manage and execute the necessary process during the Issue management.
• Should be the only person who should be talking with the TA to get the latest updates.
• Come up with workaround solutions, if possible.
• Join the calls and meetings as and when required by different parties related to the issue.
• Get the QA team to replicate the issue in UAT or QA environments
• Get the DevOps on standby to release a fix or perform a rollback of the application.
• Procure the necessary approvals in the meantime.
• Narrow down the user impact and communication should be sent only to the target user. Wider communication might cause panic and add load on the application.

?

Responsibilities of TA

• Understand the issue and ask more questions to get more information about the issue.
• Connect with the SME if needed.
• Allocate a resource to debug the issue in the dev environment.
• Analyse to predict the next occurrence of the issue.
• Start analyzing the root cause of the issue.
• Once the RC is found, estimate the effort required to fix and prioritize it with the IC and Product owner.

?

The primary goal of managing an issue is not jumping into fixing the issue. There are a few other important activities that you may have to do even before fixing the issue.

1. Do an Impact analysis to understand the scope of impact.
2. Start the damage control so that the issue doesn’t happen again and create more impact. Damage control can be done by rolling back the app/module, bringing down the service that has the issue, and informing the users to stop using the module that has the issue.
3. Collect the historical details about the issue. Though the issue might be reported today, sometimes the issue would have started building before the current date.
4. Find any workarounds that can be given until the issue is fixed. If no workaround is available, it's better to roll back.
5. Back up the logs and necessary metrics for future analysis.

Please do not panic, everything is solvable.

Do not pressure the team, help them to analyze and fix it.

Do not hesitate to ask for help.

?

Happy Technology Management !!?

?#InnovationMinds hashtag#DigitalPioneers hashtag#LeadersOfTech hashtag#FutureTechLeaders hashtag#TechInnovators hashtag#LeadershipInTech hashtag#StrategicTechLeaders hashtag#DigitalVisionaries hashtag#TechLeadershipJourney hashtag#LeadingTechChange hashtag#TechInfluence hashtag#EmergingTechLeaders hashtag#LeadershipTechStyle hashtag#TechLeadershipInsights hashtag#InnovateAndLead hashtag#TechStrategists hashtag#LeadershipByTech hashtag#TechTrailblazers hashtag#FutureForwardLeadership

?

?

?

?