Rogue User Account Swapped Microsoft Consumer Encryption Keys, Months Before Japan's Classified System Were Breached in the Fall of 2020

Prepared by Lee Neubecker and William Slater.

August 16, 2023

A forensic analysis of ineffective security audits, encryption Certificate Authorities and the rotation of private keys provides startling insights into how hackers could have accessed the email accounts of the American Ambassador to China, the U.S. Military, and key ally Japan.

Evidence suggests that a suspect user impersonated a legitimate member of the National Security System (NSS) team and tricked NSS in pushing out new 3.54 NSS package containing Microsoft’s 2017 Trusted Root Keys on June 27, 2020. The evidence is bolstered by the disappearance of critical Microsoft firewall logs and an apparently inadequate security audit.?

The implications are mind-boggling. Users throughout the world depend on the root keys to secure their computers. Without valid, properly configured digital certificates, communications via the Internet cannot be considered secure.?

What Happened??

In July 2023, according to Reuters, "Microsoft?(MSFT.O)?said on Friday that Chinese hackers misappropriated one of its digital keys and used a flaw in the company's code to steal emails from U.S. government agencies and other clients."

The compromised MSA Consumer encryption signing key allowed for unlocking the email accounts of the American Ambassador to China, Nicholas Burns, the U.S. military, and allies such as Japan. The highly sensitive information was attributed to a compromise of Microsoft MSA Consumer key that allowed for bad actors to forge tokens to bypass security on targeted user accounts. The result was that Chinese hackers were able to access the email accounts of more than 25 organizations, including U.S government agencies and our allies including Japan.

The Washington Post recently reported that Japan’s most sensitive computer networks were compromised in the Fall of 2020.? The compromise of Japan’s networks in 2020 came months after the Microsoft 2017 Root Certificate Authorities were rotated by a suspect user and released on June 27, 2020.?

Now come disclosures that Chinese military hackers penetrated the networks of Japan, according to the Washington Post. “Chinese military hackers penetrated Japan’s most sensitive computer networks, the National Security Agency discovered in the fall of 2020, just as the United States was reckoning with the landmark SolarWinds hack — and then China’s intrusion continued through the transition to a new president,” the Washington Post reported. https://www.washingtonpost.com/politics/2023/08/08/chinas-hacking-japans-defense-networks-was-bad-shockingly-bad/

Our forensic analysis and audit of the 2017 Microsoft ECC & RSA Trusted Root Certificate Authorities

In response to the previous reporting, we performed a forensic audit pertaining to two of Microsoft's old 2017 ECC & RSA Trusted Root Certificates.?

We discovered an apparent suspect user account “jcj” that appears to have been impersonating a legitimate NSS team member. The suspect user account “jcj” appears to have succeeded in tricking the NSS team into pushing out new versions of Microsoft’s 2017 Trusted Root Keys for ECC and RSA on June 27, 2020. [NSS to 3.54 #3286 package release] The “jcj” user account last reported activity on December 3, 2020 and was disabled sometime on or after that date. The suspected imposter user account that issued the request to rotate Microsoft’s Root Certificate Authorities shows a picture of a dog.

The presumed imposter "jcj" user requesting the changes

The real J.C. Jones from Lets Encrypt shows the authentic NSS team member

The change request showing the imposter account Commit Date of June 12, 2020

The change request to add a new version of Microsoft's 2017 Root Certificates was made by the imposter user ID “jcj” and committed changes on June 12, 2020.

The suspect user account “jcj” appears to have succeeded in tricking the NSS team into pushing out new versions of Microsoft’s 2017 Trusted Root Keys for ECC and RSA on June 27, 2020. [NSS to 3.54 #3286 package release]

Investigation of Microsoft TLS Firewall Logs indicated deletion activities on July 21, 2020

Bug 1658995 (security restrictions were added to access this information requiring a Bugzilla user account with multi-factor authentication since the initial publication - they must have read our report!) indicates a manual review of the firewall audit logs was conducted on July 29,2020. The bug report indicates that Microsoft’s TLS logs reported missing records on July 21, 2020. This was roughly one month following the dissemination of the suspect NSS package released on June 27, 2020 in the month following the release and distribution of these replacement Microsoft Root Certificate Authorities 2017 (RSA & ECC).

Days following the reported missing Microsoft TLS Firewall log entries were purged on July 21, 2020, the original Microsoft 2017 RSA & ECC Certificate Authorities were removed on July 24, 2020.

The Bug 1658995 failed to indicate the the suspect account user “jcj” was the source that submitted the change request to add a new Microsoft 2017 ECC & RSA Root Certificate Authorities to [NSS to 3.54 #3286 package release] on June 11, 2020 that was approved by the NSS team and later released on June 27, 2020.

These questionable Microsoft 2017 Root Certificate Authorities submitted by the suspect “jcj” remain in the public trust and NSS release packages today.

The most recent activity for the suspect "jcj" user account indicates the account was disabled sometime on or after Dec 3, 2020 when the user closed D97337 Bug 1675523.

What Certificate Authorities are Trustworthy?

In seeking to answer that question, we searched for and reviewed the published PKI documents on Microsoft's website. Furthermore, we also searched for the unique Docusign envelope ID listed and discovered multiple documents having different contents but the same unique Docusign Envelope ID listing alternate versions of Microsoft's approved Certificates Authorities. Docusign envelope IDs are unique and change with each revision instance. This indicates fabrication of one or both of the alleged BDO Microsoft PKI audit documents. The lack of a valid digital signature by the alleged auditor BDO only adds to the confusion.

The existence of a website linked to the suspect account that references "Phabricator" and "BDO" on the login screen only raises further concerns about Microsoft BDO PKI Audit reports authenticity.

June 23, 2023 BDO Microsoft PKI Audit Reports - which one can you trust?

Two versions of Microsoft’s PKI Audit Reports allegedly performed by BDO report a signature date of June 23, 2023 and both list the same Docusign Envelope ID [D297A988-9557-42AC-8346-913DD1BACD8E] which indicates at least one of these Microsoft PKI audit reports are fabricated.

https://www.microsoft.com/pkiops/docs/Content/seals/microsoft%20pki%20acs%20wtcs%20br%20indp%20acct%20opinion%20and%20mgmt%20assertion%20june%202023%20-%20final.pdf - 11 pages

5367F20C7ADE0E2BCA790915056D086720C33C1FA2A2661ACF787E3292E1270?

The 11 page PKI report uses a 2020 certificate, whereas the 16 page report uses the older 2017 certificates (likely the consumer certificates.)

https://www.microsoft.com/pkiops/docs/content/seals/microsoft%20pki%20wtca%20indp%20acct%20opinion%20and%20mgmt%20assertion%20june%202023%20-%20final.pdf - 16 pages

Microsoft ECC Root Certificate Authority 2017 and Microsoft RSA Root Certificate Authority 2017

1. FEA1884AB3AEA6D0DBEDBE4B9CD9FEC8655116300A86A856488FC488BB4B44D2
2. 358DF39D764AF9E1B766E9C972DF352EE15CFAC227AF6AD1D70E8E4A6EDCBA02
3. ECDD47B5ACBFA328211E1BFF54ADEAC95E6991E3C1D50E27B527E903208040A1
4. C741F70F4B2A8D88BF2E71C14122EF53EF10EBA0CFA5E64CFA20F418853073E0

Should we trust a NSS Update Package sent by Anonymous User?

If the suspect “jcj” user account was a bad actor and retained the private key to any of the certificate authority changes made, that bad actor would be able to exploit the U.S. Military, Japan and any other entity that trusts the rogue Certificate Authority and would allow for exploitation and interception of Azure email and so much more. This could have much broader implications since Linux, IOT and other devices also receive these packages.

The reported timing when Japan was compromised by China as discovered by the NSA in the Fall of 2020 as reflected in the Washington Post and yesterday’s White House Press briefing, further suggests the Microsoft Root Certificate Authority changes pertaining to the old 2017 RSA and ECC versions may have been a root factor in the recently reported security breaches.

Impacts of Untrusted and/or Poorly Managed Digital Certificates

The scary thing about digital communications is that connected Internet Protocol (IP) hosts will exchange digital messages with or without the presence of properly configured, valid Digital Certificates. Without valid, properly configured digital certificates, there can be no trust and therefore reliable communications via the Internet. The public Internet that we have known and used since its inception in the 1980s, was built for reliability, performance and interoperability between disparate platforms. It is incumbent upon all those who provide technical support and security services for the organizations who want to be part of gigantic Internet Diaspora, to act with responsibility.? This requires deliberate intent to ensure that every Internet Asset that participates in Internet activity is safe, secure, and that this can be regularly monitored and confirmed by qualified cyber security experts.?

Furthermore, perhaps the legendary American author, Mark Twain said it best about functioning in a World where Trust is a rare commodity:??

“It is easier to fool people than to convince them that they have been fooled.”

Conclusion and a Call to Action

This brief paper has shown a case study in what can go horribly wrong with the management of Digital Certificates that are required to provide trusted digital communications on the Public Internet. Just as a chain is no better than the weakest link, the strength of a Security Architecture based on substandard Digital Certificate Management we consider nonexistent and not trustworthy. This type of poor management of cyber investigations as found in the response to missing Microsoft TLS firewall logs reported in Bug 1658995 may have led to the recent compromises reported in the news, allowing the Bad Guys to easily exploit sensitive networks and critical systems that are connected to the Internet.?

We challenge all professionals who are entrusted with the security of Internet-connected assets to view this as a clear call to try to do better and to diligently strive to understand how to manage their Digital Business environment. Maintaining a secure cyber security position requires ongoing review and analysis of your supply chain and everything in the chain of trust.? Routinely monitoring your systems, firmware, logs and security patch updates for anomalies is a continuous process with no easy simple software solution. If that challenge can not be adequately met, hire some professionals who have deep knowledge and experience in this critical area of Infrastructure security management to assist.? The NSS team may need qualified outside auditors like us to help identify other breaks in the chain of trust.

We hope other security researchers will review our findings and provide feedback.

Security researchers may email us to request further technical details.

The Authors Contact Information and Timeline Analysis Follows

Lee Neubecker, CISSP

President of Enigma Forensics, Inc.

Security blogger at leeneubecker.com.

312-668-0333

[email protected]

William Slater, CISSP,?CISA, CISO, vCISO, SSCP, PMP, U.S. Air Force Veteran

https://billslater.com/interview/?

312-342-2626

[email protected]