Wired Calls Exchange Server a Security Liability

In March 2021, during the middle of the Hafnium attack against Exchange servers, I said that the attack gave impetus for on-premises customers to move to the cloud. Eighteen months later, things have not improved. If anything, they might be even worse despite Microsoft’s best efforts to mitigate attack vectors and close off holes in the venerable Exchange code base.

Recent day-zero vulnerabilities underline the point. But you know things have reached a critical point when Wired magazine calls Exchange server a security liability in a well-argued article by Andy Greenberg, summarized with “it’s time to say goodbye to on-premise(s) Exchange.”

Greenberg points to a stream of vulnerabilities (like ProxyRelay, described by researcher Orange Tsai), ongoing hacking campaigns, and the difficulty organizations often experience when they try to patch Exchange servers. These are facts. Exchange Server is a huge target for attackers because it was such a popular on-premises email server. Although the number of servers is gradually reducing, there’s still a lot of servers out there for attackers to pursue.

Five Reasons Why Exchange Server is Vulnerable

Customers might have some questions about why Exchange Server is so vulnerable. I think it boils down to five factors:

The age of the code base. No version of Exchange Server was designed to deal with the kind of threat-filled environment now prevalent on the internet. The web components in the current release (Exchange 2019) use an architecture laid down fifteen or more years ago (arguably for Exchange 2003 but definitely for Exchange 2007). The dependency on IIS and the difficulty in configuring web virtual directories to make sure that OWA is secure reflects thinking that wouldn’t happen today.

The reluctance of the installed base to move to new server versions. Even if Microsoft closes holes and improves the security of Exchange Server, their efforts are worth precisely zero if customers don’t upgrade their servers. Exchange has always been a slow application in terms of moving forward, largely because of the requirement for new hardware.

For example, Exchange 2019 mail servers have a recommended memory of 128 GB to allow Exchange to cache “hot mailbox data” and improve performance. Many organizations run Exchange on virtual machines hosted on VMware or Hyper-V, introducing another complication in the upgrade process.

The fragility of the upgrade/patch process: It takes too long to apply security updates or the regular quarterly cumulative updates. Servers must be taken offline, and everything must be right (including Windows patches) before an update will install. Even then, odd things can happen to prevent an update from completing. These things work out in the end, but update difficulties have caused too many grey hairs for Exchange administrators, as I know to my cost.

The detachment of the Exchange engineering team: There was a time when the Exchange engineering team was very connected to its customers. I don’t believe this is the case any longer for two reasons. First, the most experienced engineers have moved to new positions within Microsoft in roles that concentrate on the Microsoft 365 substrate or Exchange Online. These folks knew the product inside out. They also had great connections with customers and MVPs and a genuine appreciation of operational challenges. The current engineering team responsible for Exchange Server just doesn’t possess this background. They are talented people, but their connectivity with the real world is not the same as it was.

The lack of connection with their base is compounded by the demise of the in-person Ignite conference. Traditional Ignite conferences allowed engineers to interact with customers in a very personal manner. People brought technical issues to the conference to debate with engineers. It was a perfect two-way learning conduit that hasn’t happened since 2019. The recent “hybrid Ignite” is not the same. The rumor is that Microsoft will try and bring back an in-person Exchange Conference (MEC) in 2023. That might help, but I suspect that the content will focus on Exchange Online because that’s where Microsoft’s future lies.

Microsoft’s use of Exchange Online: Microsoft eats its own dog food, but the current dog food is cloud-flavored. If Microsoft used Exchange Server as its mail server, we might see more aggressive action to harden the server, close holes as they emerge, and introduce new software solutions to improve updates and patches. For instance, Microsoft might have been as assiduous in removing basic authentication from Exchange Server as they have been for Exchange Online.

Microsoft generates $100-plus billion annually from cloud services, and that’s where its focus will remain. The number of resources Microsoft assigns to on-premises server engineering will only decrease over time.

Exchange Online

Exchange Online is very different from Exchange Server, and the gap widens all the time, not least because of the central role Exchange Online plays for the Microsoft 365 substrate. Although Exchange Online spans over 200,000 physical mailbox servers, the risk of compromise is much lower than for any on-premises environment because of the security resources Microsoft dedicates to protecting its cloud infrastructure. Quite simply, few other companies could afford to erect and manage the same kind of defenses.

Time to Take the Migration Pain

I don’t know how many Exchange servers remain operational in on-premises organizations. The FBI found tens of thousands of servers to patch last year. Given that Office 365 has more than 345 million paid seats (almost all of whom use Exchange Online), the bulk of the migration from on-premises Exchange is over.

Some organizations that remain absolutely need to run on-premises (military servers are the classic example, including those on submarines). Many of those organizations have the security smarts to be able to defend their infrastructures. Others don’t, and that fact is obvious because of the ongoing level of attacker interest in exploiting Exchange flaws. Putting the special cases to one side, any regular commercial implementation of Exchange Server must ask if things have become so bad that they should migrate ASAP, even if they follow Paul Robichaux’s security principles for Exchange Server.

Different circumstances affecting companies will influence the decision, but at this point, it just makes sense to remove themselves from the target list and migrate. Migrations are painful and costly, but a compromised Exchange server is so much worse (as people discover on an all-too-frequent basis).

Meet Tony Redmond and other Microsoft MVPs at The Experts Conference 2022, December 6-7.

100% Free and Virtual. Get world-class AD and Office 365 training, plus earn 10 CPE credits.

Learn More

About the Author

Tony Redmond

Tony Redmond has written thousands of articles about Microsoft technology since 1996. He is the lead author for the Office 365 for IT Pros eBook, the only book covering Office 365 that is updated monthly to keep pace with change in the cloud. Apart from contributing to Practical365.com, Tony also writes at Office365itpros.com to support the development of the eBook. He has been a Microsoft MVP since 2004.

Comments

  1. Denil

    Support must need to know atleast why the mailbox’s and it’s spam filters were acting strangely.., it’s a basic., they simply saying tell to your recipient to add our address into trusted sender., and They don’t even know about SCL which was mentioned in the documentation. we can able to say 1 or 2 clients/recipients to add my email address in your contact list., and it’s hard to say 1000+ Clients., and for that we need to call everyone to add my email address in your trusted address and it takes atleast 1week or maybe more then a week., and we don’t even know all of them will answer or not., I hope you’ll understand. And please, I request you to degrade/update the Outlook Mail Servers and its Exchange Internal Headers. I Hope you’ll soon solve these issues.

    Thanking You

    1. Avatar photo
      Tony Redmond

      As no one here works for Microsoft, we can’t help. You need to work the issue with Microsoft support.

  2. Denil

    After the recent Exchange Update, Mails were going to Junk Folder, Even shortText Mails/Personal Mails., Or even if am gonna send mails to Clients/Recipients, They were receiving Emails to Junk Folder, within Organization Outlook/ No in Organization Outlook. Filters were working strangely.. And also with Recent Updates while using SMTP Client Submission LocalHost Submitting IP/ Public IP has been removed and showing only Microsoft IP’s. Even Customer Support (CS) don’t know issues properly. Exchange Server Year 2019 15.02.0529.005 far better then latest Exchange Server.

    1. Avatar photo
      Tony Redmond

      You’d be better off complaining to Microsoft Support about issues like this than expressing them in comments here. Formally documented issues are the only ones that Microsoft will action, and that means reporting the problem to support.

  3. saibabu

    Hey! Nicely written , Intresting to read

    Thanks & Regards
    V.saibabu

  4. kosmik technologies

    Hi, Thank you for sharing this beautiful blog….

  5. Hayden Greaves

    Hi Tony,
    Thanks for the article. I personally believe that there is still a strong argument for on prem Exchange as a Hybrid server, for EXO management in synced AD scenarios (still the majority of enterprise) and simplified mail relay for multifunction devices (scan to email) and application SMTP.

    Otherwise you are faced with creating scripts for every scenario such as hiding an object from the GAL, mail aliases, etc. And RBAC becomes that much more of a nightmare when delegating specific rights to service desks etc.

    Fact is, to simplify BAU management, less experienced generalist in-house engineers need a GUI.

    Just don’t publish endpoints externally, or use Azure AD App proxy for conditional access layers of protection, and a lot of risk is mitigated.

    1. Avatar photo
      Tony Redmond

      I suspect you might be an experienced Exchange admin, so you’re probably very comfortable with the maintenance operations needed to keep the server healthy. I wasn’t really addressing you in the article. I’m really after the folks who have on-premises servers and struggle to manage them. Maybe it’s too little time. Maybe too little knowledge. Whatever it is, they struggle. And we see this every time an attack emerges. Those servers should be decomissioned and their mailboxes moved to the cloud. Pronto.

      The folks who know what they are doing can make their own minds up. But I think you’d find that the cloud is a pretty good place to be in most circumstances…

  6. Dude

    In a world of fake sysadmins, texts like these can pass as revelatory, but it’s just a long exchange online sales pitch.
    Man, on prem exchange is fine. Don’t expose it to the Internet in any way (outer perimeter for email is on linux) and for owa you have a reverse proxy. That exchange is untouchable. And Exchnage is easy to maintain and patch. It’s a joke. And you are in full control. But IT WILL COST YOU.

    1. Avatar photo
      Tony Redmond

      There’s no sales pitch here. As a matter of record, I have documented the development of Exchange Server since 1995 over a series of books (like Microsoft Press Exchange 2010 Inside Out) and have strong and enduring relationships with many people who have worked on the product. It pains me to make such a recommendation, but it’s the right thing to do at this point. It is possible to run Exchange Server in a secure manner. Unfortunately, many instances exist when the server is badly maintained or not maintained at all. We see this evidence in the success of attackers in finding vulnerable servers to compromise. By all means stay on-premises, but do so in the full knowledge that you’ll need to work hard to maintain security against ongoing threat.

      1. David

        I think you both have valid points. Those that want/can invest in the dollars to maintain on-prem Exchange, for whatever reason, should be able to. Though Microsoft actually needs to reconnect with these organizations to understand the use case better, including dealing with low bandwidth deployments, like mentioned in the article.

        Those companies, that don’t have the money, or want, to invest in the resources to keep on-prem Exchange updated and secure, should absolutely migrate to the cloud. I would lean toward Tony being right that the majority of on-prem Exchange installs should probably be migrated to Exchange on-line now.

  7. Dale

    One of the primary reasons we still have on-premises Exchange servers is for mass mailings, since Exchange Online won’t work for that. All of our mailboxes are in the cloud. What’s your recommendation for us to be able to send out mailings to our entire customer base if we remove onprem mail servers?

    1. Avatar photo
      Tony Redmond

      Mass mailings is an appliance-type activity rather than the norm of hosting user mailboxes. You could keep an Exchange Server for mass mailings. As long as it’s not exposed to the internet (no OWA, EAS, etc.), it should do just fine.

    2. Joe Stocker

      Use SendGrid or MailChimp for mass mailings

Leave a Reply