Close Menu
Canadian ReviewsCanadian Reviews
  • What’s On
  • Reviews
  • Digital World
  • Lifestyle
  • Travel
  • Trending
  • Web Stories
Trending Now

Liberals limit debate on assisted dying bill as Conservatives call for changes

Pokémon Go’s Most Cursed Crossover Officially Revealed

The iPhone Duo feels far more intuitive with Apple’s new face unlock

Sombr just went on a vintage shopping spree in Toronto, Canada Reviews

Turkey, Texas, and 5 More U.S. Towns Made for Thanksgiving

The Onion CEO Ben Collins seeks truth, one fake news story at a time | Canada Voices

A colourful meteor shower peaks over Vancouver tonight and the skies are clearing up

Facebook X (Twitter) Instagram
  • Privacy
  • Terms
  • Advertise
  • Contact us
Facebook X (Twitter) Instagram Pinterest Vimeo
Canadian ReviewsCanadian Reviews
  • What’s On
  • Reviews
  • Digital World
  • Lifestyle
  • Travel
  • Trending
  • Web Stories
Newsletter
Canadian ReviewsCanadian Reviews
You are at:Home » Inside Microsoft’s giant Outlook outage
Digital World

Inside Microsoft’s giant Outlook outage

17 November 20266 Mins Read

At 9AM last Thursday, everything seemed normal. Microsoft employees were arriving on campus, ready for a regular day of emails and meetings. But barely an hour later, one of Microsoft’s worst outages in history started to unfold, leaving employees, businesses, and many governments without access to Outlook emails for nearly 24 hours.

It all started in Switzerland, according to a source familiar with the outage. Exchange Online servers that power Outlook started to fall offline. The problem quickly spread to Mexico, and then to other countries around the world, as Microsoft’s failover systems tried to shift email traffic elsewhere. Within a couple of hours, more than 80 percent of Exchange servers were unresponsive.

Sources tell me that Microsoft marked the outage as “sev0” in its internal incident tracking system, the highest level, which indicates a catastrophic level of service disruption. Hundreds of engineers began trying to figure out what had gone wrong, and how to restore service. It’s not unusual for servers to fail under certain circumstances, but Microsoft has a variety of systems that are supposed to route traffic toward healthy servers to keep the broader system online. In this case, the failover systems did their job, but they also spread the problem.

In the early hours of the outage, engineers were analyzing crash reports, trace logs, and kernel dumps to try and find clues to the giant outage. Some servers had entered into maintenance mode and became unresponsive due to unusually high CPU loads. Engineers tried to bring servers back online, but they were almost immediately overwhelmed by traffic waiting to access the servers.

Exchange servers boot up and run an OS over a network (Preboot Execution Environment), a process that allows Microsoft’s support staff and engineers to manage OS updates remotely. I understand that the sudden burst of traffic from servers being rebooted was overwhelming the systems monitoring them, causing a form of reboot loop where they kept trying and failing to come back online.

As engineers continued to hunt for the root cause, Microsoft was even struggling to roll back servers to a healthy state using its usual processes of reverting recent changes. As the outside world began to wonder if this outage was related to a cyberattack or sloppy AI coding, Microsoft’s engineers were already investigating potential attack vectors and liaising with DDoS specialists to understand whether this could have been an attack.

I’m told crash reports and logs quickly ruled out the possibility of an attack, and after recent rollbacks failed, Microsoft’s engineers began to take a closer look at the networking stack of the servers and discovered an unusual bug. It was unusual because the bug had been sitting on Microsoft 365 networking gateway servers since July without any impact to servers.

“A latent defect caused servers to become overwhelmed and unresponsive under certain load conditions,” wrote Charles Lamanna, the leader of Microsoft’s Copilot, Agents, and Platform (CAP) team, in a post on X. The teams behind Office, Outlook, and Microsoft Teams all started reporting to Lamanna last month after the head of Office and Teams, Ryan Roslansky, left the company.

Microsoft’s preliminary post-incident report describes the bug as one affecting select Windows servers “due to lock contention inside an internal Windows process.” The bug doesn’t affect regular versions of Windows since it was the result of a fix that was prepared specifically for Microsoft 365 networking gateway servers.

Microsoft didn’t find the bug until this major outage, and it admits that was because of the traffic mix and hardware used in its testing labs. “The contention issue is worsened on machines with more cores and newer hardware; whereas our load testing tended to focus on weaker hardware with fewer cores,” Lamanna wrote.

The bug became obvious because of two other incidents that increased load and reduced server capacity. Fifteen hours before this Outlook outage began, Microsoft Teams failed over from an alternative networking gateway to the main gateway that supports Exchange Online. “This increased the load on the networking gateway and changed its traffic patterns,” Lamanna wrote. Ordinarily this wouldn’t have been a problem, but at the same time around 14 percent of server capacity for this gateway was “out of service due to an unrelated issue from the previous day.”

The trifecta of a bug, Microsoft Teams failing over, and reduced server capacity all caused the network gateway to trigger the problem that had been lying dormant since July. It took Microsoft nearly 24 hours to get its Exchange servers stable again, and I’m told overlapping interventions from teams trying to fix issues complicated coordination at times.

Microsoft engineers had a running bridge call the entire time, and at one point it hit the maximum of 1,000 participants. That shows you how many teams were involved with trying to find the root cause of this huge outage, which started at the peak window of traffic in the US.

“Recovery took longer than usual because as servers returned to service, they were immediately overwhelmed by waiting traffic and became unresponsive again,” Lamanna wrote. “Our normally automated recovery processes were not designed to restore service from this broad unresponsive state. Additionally, since the main code defect was a latent bug from several months previous, our normal rollback, flighting, and feature flags were unable to quickly revert the change.”

Microsoft is now working on a final post-incident review that will be published later this week. Even without that full review, it’s clear that the company will need to invest more into its testing processes to ensure that when something like this happens again, it can catch the problem a lot sooner. Outages happen daily, but this kind of cascading failure is a rare moment for Microsoft to learn from its mistakes and make sure the same sequence of events doesn’t bring down Outlook for millions of people again.

Notepad will return in December after next week’s Thanksgiving break.

I’m always keen to hear from readers, so please drop a comment here, or you can reach me at notepad@theverge.com if you want to discuss anything else. If you’ve heard about any of Microsoft’s secret projects, you can reach me via email at notepad@theverge.com or speak to me confidentially on the Signal messaging app, where I’m tomwarren.01. I’m also tomwarren on Telegram, if you’d prefer to chat there.

Thanks for subscribing to Notepad.

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

  • Tom Warren

    Tom Warren

    Posts from this author will be added to your daily email digest and your homepage feed.

    See All by Tom Warren

  • Microsoft

    Posts from this topic will be added to your daily email digest and your homepage feed.

    See All Microsoft

  • Notepad

    Posts from this topic will be added to your daily email digest and your homepage feed.

    See All Notepad

  • Tech

    Posts from this topic will be added to your daily email digest and your homepage feed.

    See All Tech

Share. Facebook Twitter Pinterest LinkedIn Reddit WhatsApp Telegram Email

Related Articles

The iPhone Duo feels far more intuitive with Apple’s new face unlock

Digital World 17 November 2026

Get ready for a nationwide emergency alert test at 2:20PM ET today

Digital World 17 November 2026

This E Ink photo frame can double as your family’s digital dashboard

Digital World 17 November 2026

The Verge Best of 2026

Digital World 17 November 2026

Nvidia’s board is reportedly ‘not considering’ hiring Jensen Huang back

Digital World 16 November 2026

Apple is adding face unlock to the iPhone Duo — without new hardware

Digital World 16 November 2026
Top Articles
Why Should a Couple in Love Visit an Escape Room?

Why Should a Couple in Love Visit an Escape Room?

30 September 2025165 Views
The Mother May I Story – Chickpea Edition

The Mother May I Story – Chickpea Edition

18 May 2024106 Views
How to Keep Your Business Finances Organized All Year Round

How to Keep Your Business Finances Organized All Year Round

3 October 202598 Views
LearnToTrade: A Comprehensive Look at the Controversial Trading School

LearnToTrade: A Comprehensive Look at the Controversial Trading School

28 April 202484 Views
Demo
Don't Miss
Lifestyle 17 November 2026

The Onion CEO Ben Collins seeks truth, one fake news story at a time | Canada Voices

Open this photo in gallery:Ben Collins spent more than a decade as a journalist in…

A colourful meteor shower peaks over Vancouver tonight and the skies are clearing up

’s Travel Gift Guide For Christmas 2026, Canada Reviews

17th Nov: 20th Century Women (2016), 1hr 58m [R] – Streaming Again (6.65/10)

About Us
About Us

Canadian Reviews is your one-stop website for the latest Canadian trends and things to do, follow us now to get the news that matters to you.

Facebook X (Twitter) Pinterest YouTube WhatsApp
Our Picks

Liberals limit debate on assisted dying bill as Conservatives call for changes

Pokémon Go’s Most Cursed Crossover Officially Revealed

The iPhone Duo feels far more intuitive with Apple’s new face unlock

Most Popular
Why You Should Consider Investing with IC Markets

Why You Should Consider Investing with IC Markets

28 April 202435 Views
OANDA Review – Low costs and no deposit requirements

OANDA Review – Low costs and no deposit requirements

28 April 2024370 Views
LearnToTrade: A Comprehensive Look at the Controversial Trading School

LearnToTrade: A Comprehensive Look at the Controversial Trading School

28 April 202484 Views
© 2026 ThemeSphere. Designed by ThemeSphere.
  • Privacy Policy
  • Terms of use
  • Advertise
  • Contact us

Type above and press Enter to search. Press Esc to cancel.