Operational Resilience – Lessons from a Tsunami

san ten ichi-ichi

Hong Kong – March 2011

At 1.50pm on Friday 11th March 2011 I was on the Hong Kong trading floor, lunch had just finished and my thoughts were already turning towards beers after work and the coming weekend.

Suddenly there was a commotion, as urgent messages started coming through on the squawk boxes and telephones. The Tokyo office was violently shaking in an earthquake. Whilst tremors were not unusual this one was sufficiently alarming for our Japan based staff to alert Head Office in Hong Kong, whilst simultaneously ducking for cover.

A magnitude 9.1 earthquake had struck in the Pacific Ocean, 72 km east of the mainland. The most powerful ever recorded in the region, it has since become known as Higashi Nihon Daishinsai (the Great East Japan Earthquake) or as San Ten Ichi-Ichi (after it’s date)[i].

Within an hour we were watching the TV screens with horror as pictures showed a tsunami engulfing towns along the east coast of Honshu. Unknown to us at the time, the waves were also overwhelming the defences of the Fukushima Daiichi nuclear plant, ultimately triggering partial meltdowns in three reactors.

After a weekend of crisis management meetings and emergency calls it was concluded that the best approach would be for two of us to fly down to Tokyo and work alongside local management.

With the markets due to open again, huge volatility and volumes expected, married with the aftermath of the tragedy, aftershocks and an evolving nuclear crisis; the best way to help was to be on the ground, not second guessing decisions from afar.

Fukushima – March 2011

Whilst we were arriving in Tokyo, 220 km to the north in Fukushima the problems were escalating. ?The tsunami waves generated by the main quake had also damaged the backup generators at the nuclear plant.

The loss of power caused cooling systems to fail and mounting heat within each reactor’s core triggered the fuel rods to overheat and melt down. Material fell to the bottom of the containment vessels, boring holes in the floor.

Explosions resulting from the build up of pressurized hydrogen gas took place in the outer containment buildings enclosing the various reactors on March 12th, 14th and 15th .

Tokyo – March 2011

In the Tokyo office the TV monitors relayed a stream of news on the tsunami death toll and damage, against a constant backdrop of the Fukushima nuclear plant, and the occasional ominous puff of smoke or release of steam.

Given that the epicentre of the earthquake and flooding was a distance from Tokyo, the premises, technology and infrastructure of our company remained fully operational.

The principal concerns were therefore people related, including:

• Were all the staff and their families safe, where were they now, and what was their preferred next move. Did some want to travel further away from both Tokyo city and Fukushima further to the north, and how would we help them in that;
• What was the mindset of all our employees and those who needed to be in the office. How do you balance the primacy of welfare and understandable fears on the one hand, with (from some) a potential sense of bravado;
• Would the decisions of the ultimate parent company (with whom we co-shared premises and other facilities) impact both our ability to continue and the outlook of our own staff;
• What was the impact of the disaster on the psychology of investors and market participants, and how would that effect the volumes, volatility and ultimately our ongoing operations[ii][iii].

All these factors were being juggled whilst the office periodically swayed from side to side, as the aftershocks rumbled on, and the tectonic plates found their way back to relative stability.

Fukushima Lessons

Whilst the initial prompt for the Fukushima Daiichi nuclear incident was a natural phenomena, there were a range of human factors that exacerbated the final outcome.

In the words of Kiyoshi Kurokawa, Chairman to the 2012 Investigation Commission report[iv]:

“The earthquake and tsunami of March 11, 2011 were natural disasters of a magnitude that shocked the entire world. Although triggered by these cataclysmic events, the subsequent accident at the Fukushima Daiichi Nuclear Power Plant cannot be regarded as a natural disaster. It was a profoundly manmade disaster - that could and should have been foreseen and prevented.

Our report catalogues a multitude of errors and wilful negligence that left the Fukushima plant unprepared for the events of March 11. And it examines serious deficiencies in the response to the accident by Tepco, regulators and the government.

For all the extensive detail it provides, what this report cannot fully convey - especially to a global audience - is the mindset that supported the negligence behind this disaster. What must be admitted - very painfully - is that this was a disaster "Made in Japan."

Its fundamental causes are to be found in the ingrained conventions of Japanese culture: our reflexive obedience; our reluctance to question authority; our devotion to 'sticking with the program'; our groupism; and our insularity.

The report concluded that replacing people or changing the names of functions or institutions would not solve the problems, and that unless the root causes were resolved, preventive measures against future similar accidents would never be complete.

Furthermore, the underlying issue was noted as the “social” structure that resulted in "regulatory capture," and the organisational, institutional, and legal framework that allowed individuals to justify their own actions, hide them when inconvenient, and leave no record in order to avoid responsibility.

The resilience of the Fukushima Daiichi Nuclear Power Plant was compromised by the human factor.

Business Continuity Lessons

Acknowledging that memory is patchy and partial, I have no recollection of anyone in our company reaching for the (doubtless) well crafted and detailed Disaster Recovery or Business Continuity plans, as the crisis unfolded.

Six inches of paper looks impressive when the auditors or regulators visit, but to co-opt the well known quote:

“no (disaster) plan survives first contact with the enemy”.

Granted the specific nature of the event; leaving our premises, technology and operations intact, lent the situation a different flavour from one where essential infrastructure is damaged or destroyed.

However, the critical path of our resilience and response was paved with the psychology of:

• A quorum of empowered people who could act quickly and decisively;
• The trust of employees that their best interests were held paramount, and that the reasoning behind decisions and actions was transparent;
• The motivations and interests of other parties – whether group companies, clients, regulators, exchanges, or suppliers.

In Japan, organizations often provided hard hats and emergency ("Bosai”) kits for employees to use in case of earthquakes or other natural disasters. These contained food, water, medicines and other emergency supplies[v].

But perhaps more important than the provision and content of these crisis kits was the willingness of people to step into a situation where one may be needed.

The resilience of the company was reinforced by the human factor.

Other Disaster (and Disastrous) Lessons

The centrality of human behaviour to organizational resilience is unsurprising and consistently reinforced.

Yet even the most technically advanced of organizations, with big budgets and technical expertise, repeatedly fall foul. Notable examples include:

• NASA - The Space Shuttle Challenger disaster of 1986 serves as a poignant instance of how human factors can lead to catastrophic failure. The launch proceeded despite warnings about the cold weather's impact on the O-rings, which ultimately led to the shuttle's explosion. The independent report[vi] after the accident highlighted significant cultural failings. Despite this 17 years later in 2003, when Columbia disintegrated on re-entry, the investigation once again was scathing on the organizational and behavioural traits that led to disaster [vii]. ?
• BP - The Deepwater Horizon oil rig explosion in 2010 resulted in one of the largest environmental disasters in history. Poor decisions, miscommunication, and inadequate training all played a role in the catastrophe. The 2011 National Oil Spill Commission investigation[viii] identified an array of operational behaviours and underlying safety management problems that were deemed causal[ix].
• Boeing – The 737 Max crashes and subsequent issues are a complex interplay of design flaws, regulatory challenges, corporate culture, and manufacturing issues, married with internal pressure to keep pace with Boeing’s chief competitor, Airbus[x]. As one employee said of the new version 737 Max, ?before it went into commercial operation - “I’ll be shocked if the FAA passes this turd.”

In many ways these repeated patterns contradict the words of the Chairman to the Fukushima Investigation Commission report quoted above, who identified the root cause of the disaster as resulting from specific behaviours and conventions within Japanese culture.

Siloed mentality, (a lack of) psychological safety, impression management, a reluctance to grapple with commercial and organizational conflicts of interest are prevalent in many corporations across the world and are not unique to Japan.

So I would amend the phrase used by Chairman Kurokawa, to now say:

What must be admitted - very painfully - is that these are disasters "Made by Humans."

Operational Resilience in Banks – The Theory

My reflections on operational resilience and the lessons of Fukushima were triggered by listening to a recent presentation by Kathleen Heppell-Masys[xi] (from Atomic Energy Canada Limited) who led the human and organisational contribution to the independent IAEA report into the Fukushima Accident[xii].

She talked of the imperative to understand the entwined human, organizational and technical factors critical to resilience and safety.

Of course operational resilience is already a hot topic in banking, with a myriad of global initiatives and new regulations, including:

• The Basel Committee on Banking Supervision (BCBS): New principles for operational resilience issued in March 2021[xiii];
• United Kingdom: New rules and guidance from the Bank of England on operational resilience introduced in 2022, and required mapping and testing by March 2025[xiv].
• USA: In 2020 the Federal Reserve released a joint regulatory paper on sound practices to strengthen operational resilience[xv]. In March 2024 the acting Comptroller highlighted the importance for critical banking services[xvi]
• European Union: Digital Operational Resilience (DORA) is a EU regulation that impacted financial service firms will need to be compliant with in 2025[xvii].
• Similar resilience requirements from Australia, Canada, Hong Kong, Singapore and other jurisdictions[xviii].

With all this focus on operational resilience from the regulators, looming deadlines and the commensurate projects, resources and budgets of banks, are we comfortable that a balanced and comprehensive approach has been taken?

Operational Resilience in Banks – The Practice

Some regulators have attempted to call out the importance of the human factor, albeit these pronouncements can give the impression of an after-thought, rather than a central (and enforced) requirement. As examples:

• In the UK the PRA notes a requirement for bank boards to “articulate and maintain a culture of risk awareness and ethical behaviour for the entire organisation, which influences the firm’s operational resilience”[xix].
• In Canada OSFI highlights the importance of “establishing a culture that promotes and reinforces behaviours that support operational resilience and proactively managing culture and behaviour risks that may influence resiliency”[xx]

To test the reality, at a recent conference in London[xxi] I asked an audience, drawn from within Financial Services, the following two questions:

1) What do you view as the most significant resilience risk facing your firm?

The answers were as follows:

• 39% - Technology – e.g. Outdated systems, stability, integration, digitalisation
• 27% - Cyber – e.g. Malware, denial of service, social engineering
• 20% - Third Parties – e.g. Vendors, suppliers, outsourcing, intermediaries
• 7% - External – e.g. Climate, geopolitical
• 4% - Data
• 4% - People – e.g. Key person, skills, behaviour, concentration risk

2) To date, where has your firm focussed most of its resources in building operational resilience?

The answers were as follows:

• 46% - Technology?
• 46% - Governance and Frameworks
• 8% - Policies and Procedures?
• 0% - People

I acknowledge that the questions may have been better worded had they allowed for a ranking of importance and effort across multiple answers.

Nonetheless it is clear that resilience is still being considered as primarily a technical and organizational challenge, and the critical role of human behaviour and decision-making is not yet a priority.

Furthermore, those who are looking at the cultural factor in operational resilience are often conflating two different interpretations of the challenge, being:

• A more limited one - does my firm take the topic seriously at all levels , is there sponsorship and engagement? Seeing the subject through the lens of risk culture communication and awareness.
• A more expansive one -? does my firm (also) properly understand and monitor on an ongoing basis the behavioural drivers that are key to maintaining (or undermining) resilience? Seeing the subject though the lens of continual analysis, supervision and action.

Conclusion

There is the possibility that banks and their regulators are ahead in their thinking, and less liable to fall into the traps that have repeatedly caught out the Japanese Nuclear industry, NASA, BP, and Boeing, to name but a few.

However, their track record should temper any over-optimism.

The 2023 events that overwhelmed SVB, Signature Bank and Credit Suisse were as much about behavioural risk as they were about traditional financial and liquidity risks.

Whilst those examples were more about core business resilience rather than operational resilience, the centrality of human action and inaction still applies.

And even if we take a purely operational lens, problems with core processes and systems in banking are still common:

• A study in 2019 found that six of the UK’s biggest banks had at least one IT failure every two weeks[xxii];
• In 2022 TSB was fined ￡49m for a systems meltdown that caused chaos[xxiii];
• Last week both Ethiopia's largest commercial bank[xxiv] and the UK’s Nationwide[xxv] were blaming the ubiquitous technology “glitch” for their own problems.

But if we see both the main resilience threats and defences as primarily technical in nature, able to be solved through engineering solutions, then we may be falling for the comforting balm of false certainty and linearity.

And we may be building our fortifications for the last wave, not the one coming.

??

Postscript notes

• 13 years have taken their toll on my memory so apologies if my recall is patchy. All errors are my own.
• Respect to all those I worked with in Japan during those few frantic weeks, especially Keith Takakuwa . I had the advantage of returning to the relative normality of Hong Kong, and the tenacity and quiet dedication of those Tokyo staff who remained throughout, and in the following weeks and months, was impressive.
• For the purposes of narrative I have highlighted those areas that went awry in the Fukushima incident and that went well in our Company. In reality there was also massive heroism and selflessness from many at the nuclear plant, and snafus with us. But the central role of human behaviour and decision making remains throughout.

#OperationalResilience #OperationalRisk #RiskCulture #BehaviouralRisk #BehaviouralScience #BehavioralScience #BusinessContinuity #DisasterRecovery #BehaviouralLens

??

References

[i] https://en.wikipedia.org/wiki/2011_T%C5%8Dhoku_earthquake_and_tsunami.

The earthquake was the most powerful ever recorded in Japan, and the fourth most powerful earthquake recorded in the world since modern seismography began in 1900. It moved Honshu 2.4 m (8 ft) east, and shifted the Earth on its Axis between 10 cm (4 in) and 25 cm (10 in)

[ii] Hood, M., Kamesaka, A., Nofsinger, J., & Tamura, T. (2013). Investor response to a natural disaster: Evidence from Japan's 2011 earthquake. Pacific-Basin Finance Journal, 25, 240-252. https://www.sciencedirect.com/science/article/abs/pii/S0927538X1300067X

[iii] The Guardian March 17th 2011. Japan crisis sees yen hit record high against dollar. https://www.theguardian.com/business/2011/mar/17/japan-crisis-yen-surges-nikkei-falls

[iv] Fukushima report: Key points in nuclear disaster report: https://www.bbc.co.uk/news/world-asia-18718486. The official report of The Fukushima Nuclear Accident Independent Investigation Commission - Executive summary https://reliefweb.int/report/japan/official-report-fukushima-nuclear-accident-independent-investigation-commission

[v] Disaster Prevention Information - Tokyo Metropolitan Government

[vi] Report of the Presidential Commission on the Space Shuttle Challenger Accident 1986 https://www.nasa.gov/history/rogersrep/genindex.htm

[vii] Columbia Accident Investigation Board 2003 https://web.archive.org/web/20120426005255/https://caib.nasa.gov/default.html

[viii] National Commission on the BP Deepwater Horizon Oil Spill and Offshore Drilling 2011 https://cybercemetery.unt.edu/archive/oilspill/20121210200431/http:/www.oilspillcommission.gov/final-report

[ix] Reader, T. W., & O’Connor, P. (2014). The Deepwater Horizon explosion: non-technical skills, safety culture, and system complexity. Journal of Risk Research, 17(3), 405-424. https://www.tandfonline.com/doi/abs/10.1080/13669877.2013.815652

[x] Herkert, J., Borenstein, J., & Miller, K. (2020). The Boeing 737 MAX: Lessons for engineering ethics. Science and engineering ethics, 26, 2957-2974. https://link.springer.com/article/10.1007/s11948-020-00252-y

[xi] 1LOD Conduct and Culture 2024 - https://www.1lod.com/deep-dives/cc-deep-dive/cc-agenda

[xii] IAEA - The Fukushima Daiichi Accident. https://www.iaea.org/publications/10962/the-fukushima-daiichi-accident

[xiii] Basel Principles for operational resilience – 2021 - https://www.bis.org/bcbs/publ/d516.htm

[xiv] PRA Statement of Policy on Operational Resilience https://www.bankofengland.co.uk/prudential-regulation/publication/2021/march/operational-resilience-sop

[xv] SR 20-24: Interagency Paper on Sound Practices to Strengthen Operational Resilience https://www.federalreserve.gov/supervisionreg/srletters/SR2024.htm

[xvi] Acting Comptroller Targets Operational Resiliency for Critical Banking Services – 2024 - Acting Comptroller Targets Operational Resiliency for Critical Banking Services (ktslaw.com)

[xvii] EU Digital Operational Resilience Act (DORA) https://www.eiopa.europa.eu/digital-operational-resilience-act-dora_en

[xviii] Operational resilience: Regulation Around the World https://www.nortonrosefulbright.com/en-us/knowledge/publications/3215deb7/operational-resilience-regulation-around-the-world

[xix] In the PRA paper https://www.bankofengland.co.uk/-/media/boe/files/prudential-regulation/statement-of-policy/2021/operational-resilience-march-2021.pdf?la=en&hash=908CF0854077E5F466D512BFB904C6EA4503F54B

[xx] In Operational Resilience and Operational Risk Management - Draft guideline (2023) https://www.osfi-bsif.gc.ca/en/guidance/guidance-library/operational-resilience-operational-risk-management-draft-guideline-2023

[xxi] I was at XLOD London November 2023 https://www.1lod.com/xlod

[xxii] UK banks hit daily by IT failures halting payments, says Which? The Guardian: March 4th 2019. https://www.theguardian.com/money/2019/mar/04/uk-banks-hit-daily-by-it-failures-halting-payments-says-which

[xxiii] TSB fined ￡49m over IT system meltdown – BBC News - 20 December 2022

https://www.bbc.com/news/business-64036529

[xxiv] Commercial Bank of Ethiopia head warns 'no escape' for clients who profited from glitch – BBC News - https://www.bbc.co.uk/news/world-africa-68615717

[xxv] Nationwide payments back to normal after IT glitch caused delays – BBC News - https://www.bbc.co.uk/news/technology-68635180

?