Microsoft has released the results of its investigation into how Chinese hackers breached its email systems and gained access to US government emails.

Microsoft came under fire after Chinese hackers were able to acquire Microsoft account consumer signing key, giving them access to email systems, including those used by the US government. The company was called to task for paywalling critical security features behind its highest paid cloud plans, making it nearly impossible for customers of lower-tier plans to know they had been impacted by the breach.

The company has been investigating how the breach occurred and has finally released its findings:

Our investigation found that a consumer signing system crash in April of 2021 resulted in a snapshot of the crashed process (“crash dump”). The crash dumps, which redact sensitive information, should not include the signing key. In this case, a race condition allowed the key to be present in the crash dump (this issue has been corrected). The key material’s presence in the crash dump was not detected by our systems (this issue has been corrected).

We found that this crash dump, believed at the time not to contain key material, was subsequently moved from the isolated production network into our debugging environment on the internet connected corporate network. This is consistent with our standard debugging processes. Our credential scanning methods did not detect its presence (this issue has been corrected).

After April 2021, when the key was leaked to the corporate environment in the crash dump, the Storm-0558 actor was able to successfully compromise a Microsoft engineer’s corporate account. This account had access to the debugging environment containing the crash dump which incorrectly contained the key. Due to log retention policies, we don’t have logs with specific evidence of this exfiltration by this actor, but this was the most probable mechanism by which the actor acquired the key.

Only time will tell if Microsoft’s explanation for the breach will assuage concerns. The Department of Homeland Security’s Cyber Safety Review Board has said it will investigate the breach, and Tenable CEO Amit Yoran has called Microsoft’s Azure security “grossly irresponsible.”