What are the best practices for communication during a data center disaster recovery event?

Everything should be scripted or planned in detail as much as possible because once a disaster hits people will need guidance and you have to assume they are working in isolation. The other critical thing is to actually test the processes. Yes it's expensive. Yes it's painful. Yes it's going to impact staff. But all the above are a drop in the ocean compared to what will happen WHEN a disaster arises. Get on top of it now, wear the small pain now and save your sanity later.

Meticulous planning is key. One often overlooked yet critical aspect is validating and rehearsing call escalation lists. I always prioritize having an accurate plan in place, complete with alternate contacts. During holidays, I've encountered situations where reaching an executive proved challenging because they assumed their direct report wouldn't need to contact them. Or that they were high enough up the ladder they would never get called. Outdated contact details or employees no longer in a department can also pose hurdles. Lesson Learned: Not having the right contacts when needed is akin to not having a password into your system, a potentially costly oversight. How do you ensure your contact lists are up-to-date and effective?

Heed these sage counsels: Swift Dispatch: Send messages as Mercury's winged sandals, swift & sure. Crafted Playbook: Prepare a script, a playbook, to banish confusion from the stage. Herald's Truth: Choose a herald, a voice like thunder, proclaiming transparent truths. Learned Council: Gather a council versed in data's mystic arts, a bulwark of knowledge. Electronic Ravens: Establish an aether network, e-ravens swift in message-bearing. Regular Parleys: Hold regular conclave parleys, where updates flow like a nurturing river. Redundancy and Resilience: Invoke redundancy & resilience, sacred guardians of communication's flame. In disaster's shadow, these practices, like stars in tempestuous seas, guide the ship to recovery's shores.

Communication is one of the important pillar during the disaster recovery process. There should be a central team and communication manager who should communicate each stake holder, key business owners, all impacted downstream and upstream systems and partners on the progress of the recovery process. Key updates ( with high level details on the current state and pending works) should be communicated via email in regular intervals. In case of crisis and issues, the manager needs to bring key people to a call and discuss the issue in detail and ensure right team needs to engage and participate and help in mitigating the crisis.

Plans shall be done for a generic ?disaster” as what will happen most likely is different to any super-detailed plan. So, start with detailed assets inventory (remembering people are assets in this context). Then create dependency tables. For each critical assets, evaluate how long business can survive without it, looking into three options (inconvenience, loss of customers and money, loss of business altogether). Then plan for people and technology reliability options. Consider plan for nearly physical communication tree (proximity of staff home locations). Plan for responsibility delegations (decision makers need to be available) and for external communication and create communication triggers. Print, distribute and test.

Ensure that all internal teams are on the same page. Miscommunication within the team can lead to inefficiency and errors in the recovery process. Keep records of all communications during the disaster. This documentation can be invaluable for post-event analysis and for improving future response strategies.

First of all, the standard operational procedure in case of such an eventuality needs to be well documented with templates, scenarios, distribution lists, and such, so that if it happens, people do not need to panic and can leverage the templates only varying some key data points and send out the communication as fast as possible. In my experience, it is important to not just use multiple communication channels but to also tailor the message based on who it is being communicated to. The level of technical detail to be included for a technical audience may not be required for business users, for whom the business impact needs to be outlined more.

Multiple communication channels is ok but ensure each team needs to be in sync to avoid the confusions. For business owners and higher management, the communication and updates should be crispy and to the point. For technical teams, the details should have more information.

In my personal experience, being honest and transparent about the problem in hand can really increase the probability of resolution. Ownership must be taken and roles and resposibilities should be identified as early as possible. If proper communication is done, people can understand what is at stake and how wide the area of impact is.

It is very important to be transparent and honest for any issues and work. Avoid issues will not solve the issue however by saying that there should be some buffer time and right time to pass the message. Sometimes its ok to wait for sometime while the system recovery is in progress and provide the update in right time. Giving bad news in quick time is not always good.