What do you do if your network goes down and you're the only engineer on duty?

Check the following: 1) recent configuration changes or planned works 2) recent physical site access logs 3) power outages 4) fiber outages 5) anomaly events (SW updates, External systems outages, Gaming or Streaming events ) This is covering 90% of the network outages.

Time is of the utmost importance, and our primary goal is to minimize any potential downtime. Upon confirming an issue, I will follow the established escalation communication matrix and promptly notify all node owners. Simultaneously, I will initiate isolation techniques using the Network Management System (NMS) to scrutinize for alarms, node failures, potential cyberattacks, recent modifications, and other relevant factors until the root cause is identified. This will provide me with an indication of the outage severity. I will take immediate action by providing a solution or workaround if the issue falls within my expertise domain. If not, I will provide remote access (if allowed) or on-site support until the appropriate personnel arrive.

See if the failover mechanism of switching network to other Service Provider is Auto-activated, if not, I will use manual intervention and ensure the service is restored and hardware/servers are up Then I shall contact the vendor to lodge a priority complaint and share the preliminary logs and analysis asking them to get involved and post us with regular updates. Thereafter I will follow the below Incident Management steps : 1. Assess the Situation: 2. Check Hardware 3. Review Logs 4. Isolate the Problem 5. Troubleshoot Methodically 6. Communicate: Keep stakeholders informed about the situation and provide regular updates 7. Seek Help if Needed: 8. Implement Temporary Workarounds 9. Document Everything 10. Perform Post-Mortem Analysis

, I'd swiftly assess the outage's scope, check configurations and hardware health, and investigate environmental factors or errors in logs. Employing temporary fixes if necessary, I'd communicate updates to stakeholders and escalate if unable to resolve independently, all while maintaining thorough documentation for future reference.

Well first and foremost your duty is to make sure you are NOT the only person on the NetEng team that knows about this. Once you have notified the CoC (Chain of Command) that there is a potentially serious issue you can work the problem. In order of priority. 1.) How bad is it? Total outage, partial outage, serious degredation of service(s). 2.) What applications and users are currently being impacted? 3.) Check of NetMon systems. What do your tools tell you is going on? 4.) Checking logs and device current status. 5.) Failure recovery. Can this be recovered within the timeframe agreed before BCP or DR steps kick in? 6.) Restoration of normal operations. 7.) Post mortem. 8.) Mitigation process/project including ITIL docu updates, etc.

, I'd swiftly assess the outage's scope, check configurations and hardware health, and investigate environmental factors or errors in logs. Employing temporary fixes if necessary, I'd communicate updates to stakeholders and escalate if unable to resolve independently, all while maintaining thorough documentation for future reference.

Using the network diagram, I search for the nearest offline network device, connect to its online neighbor and examine the interface to the offline device and its own logging. I look at log files from offline network devices.

When the network is managed by multiple vendors, the key point is access to proper, up-to-date documentation. Based on it is sometimes easy to find in which vendor responsibility is the issue probably located. If it is not possible to identify location of the issue based on the documentation, only option is to open multivendor troubleshooting online session and identify possible root cause + it source/location.

I look for DDOS I look for any network changes that have be initiated by any of my colleagues. I would rollback any new updates But starting at the bottom, I would look for one node attempting to torrent and shut it down.

If Layer 1 is cool, start at the bottom like running MTRs from source to destination and then reverse to look at the path to see if it's only you or a bigger issue like BGP. Then work your way up. 25 years and the lower levels I have found can be the root cause more often than not. Unless it's a LAN issue, then do pretty much the same.

The best way to find the roor cause of a network failure, is to cleary identify what is the exact symptom the network problem is causing. Generic sympton′s like everything′s is out do not let the people focus on fast finding the cause, since they have to check everything on the network. More precise sympton′s , let us focus on where the problem is. Like only the finance transctions to a specifc bank/card are not going through, wiche is very diferent of finance transactions are not working. Since is only one specifc bank or card, i can focus on that part of the network that provides this services.

Depending on the result of the analysis, various measures are required. The on-site contact may be able to reset the circuit breaker at the distributor, carry out a cold start of devices and check cable connections. It may be necessary to deploy a network technician on site with the required access, who must have suitable replacement optical transceivers and network cables with him, and possibly also a replacement device and measuring device, if known. I inform the responsible Engagement Manager, who can take over the customer information and documentation.

To test the network, you should check to see if the routers are all up with pings to and from each one. Test the firewalls for correct configuration and blocked/allowed IP addresses, and make sure there is internet connectivity. Finally, debug devices and check the logs to be sure there are no more issues.