How can you categorize and prioritize IT Operations incidents and changes?

Changes should not be responded or resolved, they have to be prepared and controlled. Changes can sometimes generate incidents but these one will be managed by incident management.

Another perspective on Change Management and its planned vs unplanned nature is the case of hot fixes or software patches. Those changes can sometimes result from Releases or other integration changes and are often classified as Urgent or Emergency changes depending upon the magnitude of customer or service impact.

?? It is definitely important to have clear concepts as a starting point, in the world of IT Operations an INCIDENT is an atypical event that generally occurs without foresight on the part of the team, it can have different levels of Impact and Severity, therefore it is important to be able to categorize possible incident scenarios. ?? Changes in IT operations arise due to improvements, requirements and/or correctives that will be deployed on the ecosystem in some component of the architecture. What is really important is to have a change management process, aligned to some framework of good practices. Although both make modifications to the ecosystem, their connotation is different and the impact of their occurrence are opposite poles.

Actually I disagree with this section based on some factual errors on how ITIL deals with Incident priorities and how ITIL classifies changes. First the purpose of Incident Management is" to minimize the negative impact of incidents by restoring normal service operation as quickly as possible". Incidents priorities are established by impact and urgency which create a matrix of priority levels, usually 3 to 5 priority levels. The purpose of Change?is maximize the number of successful service and product changes by ensuring that risks have been properly assessed, authorizing changes to proceed, and managing the change schedule. For ITIL, Changes don't have priorities but are classified as Standard (the lowest risk), Normal or Emergency.

While ITIL definitely has guidance on the urgency classification of Incidents and Changes, it's often best for the IT Service and Business Owners to consult and formulate their own guidelines around their priority model. It's important to note that IT services is a broad term and depending on the nature of the service, different type of failure could mean different urgency for a given service. For instance, high latency in a Payments API service could be critical for their service offering while the same may not be true for File storage, which may tolerate some level of latency issues. Hence, it's imperative for service owners to design priority model based on their services and the customer needs.

The classification of incidents hinges on two primary factors: Impact and Urgency. Based upon the impact and urgency the incident priority is selected. P1 - Always classified as Significant impact & and urgency, P2 - High Impact & urgency etc. Conversely, changes are primarily categorized as Standard (Low Risk), Normal, and Emergency, with additional sub-categories like Expedited Problem Fix, Incident Fix etc.

It is imperative to take the best of each good practice framework, such as ITIL, ISO20000, COBIT, ISO27001 and ISO 27032 or any other framework that provides a structure for categorization. For the categorization of incidents I recommend applying this step by step: 1. Create a Prioritization system. 2. Create a rating system. 3. Enable an online knowledge database. 4. Set Severity thresholds. 5. According to the existing SLA and OLA, group with clear criteria. In a change control scenario there are: a. Standard changes and b. Emergency changes. From this point, it is important to identify each one and deploy its internal process efficiently, ensuring that the emergency change committee is in place with decisive empowerment.

Impact Assessment: Determine the potential business impact of each incident or change. Severity Classification: Assign severity levels based on technical impact. Time Sensitivity: Assess the urgency of the issue and whether it requires immediate attention. Stakeholder Input: Involve key stakeholders to consider business priorities. Historical Analysis: Use past data to identify recurring issues and prioritize root-cause solutions. Automation: Automate routine tasks to free resources for critical issues. Risk Evaluation: Consider potential risks associated with each incident or change. Continuous Improvement: Regularly review and adjust prioritization criteria to align with evolving needs.

Consider utilizing a weighted scoring system. Assign numerical values to each factor (e.g., business value, customer satisfaction) and use an algorithm to calculate a composite score. This approach offers a more granular and objective way to prioritize incidents and changes in IT Operations.

When communicating IT changes, it is crucial to consider the audience (include relevant stakeholders), clearly define the affected systems, provide details about timelines and expected downtime, outline fallback procedures, address support changes, and follow up with post-change communication.

Communication is crucial for both incidents and Changes. Communicate Incidents: Need to target the right stakeholders, business domain, Customers, and Vendor(s) if need be. A very high-level message, to the point, succinct message either via a text or email, or both. You communicate according to your audience. Try to avoid technical jargon. Communicate Changes: This needs to be communicated either via text, email, or both. Ensure to include impacted Services or Configuration items (CIs) and all the relevant high-level change details. Outage time should be communicated clearly followed by the Business & Technical verification resources.

It is important that each component of the ecosystem is defined in a matrix, what hierarchical level it has on the business and the ecosystem, the dependencies and the critical areas that would be affected by an incident and/or change control. These actors involved within the component that would be the subject of an incident or change control, being on the map, will have their contact information and therefore, according to the escalation levels and the categorization of the event, new actors are included in the chain. Communication. It is imperative to use traceability platforms and these contain in their characteristics automated sending of messages with business rules and escalation.

Research finds that high performing teams can do multiple deployments per day, with low lead times with MTTR of under 1 hour. Their change failure rate is between 0-15% It’s found that teams that breakdown work into smaller chunks achieve higher efficiency with good change closure. Yet, the ITIL service transition has specific check points and classifications for changes as controls that can hinder change management. The solution can be the team’s ability to converge. We can’t forget that frameworks and methodologies (Agile, ITIL) are in place to help professionals level and surpass them. So the success in handling service operations or service transition is proportional to the maturity level of the professionals practicing them.

I agree with all these measuring metrics but to improve the IT Operations incident and change management, I would recommend a debrief post every Major incident resolution where we need to discuss, Gap, Learning across the People, Processes, and Technology. What went well and What did not etc. For Changes, there should be a separate debrief, where the Change Manager would run a debrief to review how the change went, and if there was any feedback (positive/Negative, etc) from involved Support teams during the change.

In IT change management, conducting a post-implementation review is vital following a major change. This review aims to extract valuable insights from significant deployments, and the identified learnings are documented and integrated into process development whenever applicable.

Other metrics like MTTF (mean time to failure), and MTTA (mean time to acknowledge) can help tech teams understand how often incidents occur and how quickly the team bounces back from those incidents.

Large service organizations often have a gamut of internal and external IT services co-existing and interplaying. This brings up both opportunities and challenges in implementing a sound and capable IT Service Management practice. Opportunities are around achieving standardization of service quality so that the collective interplay of those services is a reliable and solid performant business service. Challenges exist around the need to accommodate for varying needs and divergent objectives of these services which are often built in silos by different business units. Many organizations have now considered tailoring the ITSM practices and allowing a hybrid model of a mix of baseline metrics and custom service metrics for individual services.

In an experience providing a regional transaction processing service, we found that a component external to ours had a crash and that this left all the ACQ - Acquirers that were dependent on this processor inoperative, in the first place in the incident they did not have a DRP implemented, the communication and escalation chain was not effective, there was no knowledge base to consult and shorten classification times and therefore improve the intervention plan. Being such large companies, you can have everything documented, however without drills or practical exercises the times will not be better than if you had nothing, everything must be complemented and practiced.