The Curious Case of the Quarantined Azure AD Identity Protection Weekly Digests

Exchange Online Protection (EOP) is often criticized for it’s effectiveness and accuracy. I have no doubt that some customers are finding EOP’s performance to be disappointing. But I also think that EOP, like any other email security service, will not work well for every single customer “out of the box”, and often requires some effort to tweak and tune the configuration to achieve a satisfactory outcome.

Recently, I’ve been noticing my Azure AD Identity Protection weekly digest emails getting quarantined by Exchange Online Protection.

At first, it was a minor annoyance. Why would Microsoft block their own service notifications? I clicked “Report as Not Junk” and “Release to Inbox” and got on with my day. But after a few weeks it was starting to bother me, and I spent a few minutes digging into it further.

When you’re investigating a spam false positive/negative, the best place to start looking is the message headers. There’s a few ways to get the headers. If the message is in quarantine, you can access the headers in the Exchange admin center.

If the message has already been released to the recipient’s inbox, you can retrieve the message headers in Outlook. Once you have the message headers, copy and paste them into the Message Analyzer in the Remote Connectivity Analyzer. If the Message Header Analyzer add-in is already installed in Outlook, you can perform the header analysis from within Outlook. I prefer doing it on the RCA website in my web browser though, which is easier to read.

There’s a lot of info in the header analyzer output. Some of it is fairly straightforward, looking at things like SPF results. Some of it is more difficult to interpret on its own, and you’ll need to refer to the anti-spam message headers documentation.

In the header analyzer, look for the Forefront Antispam Report Header, which provides a neat summary of the EOP verdict on the email. In my case, there’s nothing immediately suspicious about this email. The spam confidence level (SCL) is only 1, the IP filtering verdict is okay, the bulk complaint level is low (as you’d expect for a system email from Microsoft), and the phishing confidence level is 0. All looks fine. As a side note, if you’re looking at headers for a message that has been released from quarantine, the SCL will be -1 and the spam filtering verdict will be SKQ.

After looking through the headers I noticed that the language was displayed as “ja” (you can see this in the screenshot above), which is the language code for Japanese. Now, this email is definitely not written in Japanese. But something is making EOP think this email is written in Japanese. Interestingly, the same digest email goes to my Outlook.com mailbox, and an analysis of the headers from that message show no country/region or language information.

Why are Japanese emails getting quarantined as spam in my Office 365 tenant? Because I have international spam filtering enabled.

I can’t read languages other than English, and some non-English spam was getting through to my mailbox, so I decided to just filter out foreign language emails. It never occurred to me at the time that this might cause system emails from Microsoft to get quarantined (who would anticipate such a thing happening?).

Anyway, since I have no way to control the characteristics of the Azure AD Identity Protection weekly digest email, that leaves me with a few options to fix this problem (aside from reporting the matter to Microsoft, which I’ve already done). I could remove Japanese from the international spam filtering list, but that’s not ideal, as the problem could shift to another language.

I could add the sender email address of the weekly digest emails to the allow list in my spam filter policy. In fact, that’s what I did at first. Then it occurred to me that I might be exposing myself to spoofed email from that address. But surely Microsoft would not allow one of its own notification email addresses to be used in that manner, right?

I ran some quick tests, sending emails using PowerShell from my own network, as well as some other networks I have access to. Sending from my own network, I am able to spoof the identityprotection-noreply@identityprotection.microsoft.com email address successfully. The emails are delivered to my inbox, and the headers show a “Spam Filtering Verdict” of SKA, due to the allow list entry that I added.

Clearly it is not a good solution to add that email address to the spam filtering allow list in EOP. After I removed the allow list entry, I repeated my spoofing tests and the emails were not delivered, but were quarantined instead. This brought me back to square one, and I needed to find another way to allow the Azure AD Identity Protection emails to be delivered.

So I looked into it some more. The next thing that caught my attention was that the emails come from an email service called Sparkpost. If you are a developer creating an app or service that needs to send emails, you can use Sparkpost instead of writing your own email service or managing your own email servers. A perfectly normal approach for developers to take, although I’m not sure why Microsoft feels the need to use them for Azure AD Identity Protection emails.

The sender address for the Azure AD Identity Protection emails is identityprotection-noreply@identityprotection.microsoft.com, so I looked up the SPF record for the identityprotection.microsoft.com domain. I found that a redirect is being used, which effectively means that the SPF record for _spf.e.sparkpost.com is used as the SPF record for identityprotection.microsoft.com.

So I looked at the SPF record for _spf.e.sparkpost.com, and found the following.

The use of an “exists” mechanism is not something I see very much of, so I decided to try validating the SPF policy using this tool.

The result was a pass, meaning Sparkpost’s IP addresses can send emails from the identityprotection.microsoft.com domain.

However, if you take another look at the SPF record for _spf.e.sparkpost.com above, you’ll see that is uses “~all”, which is a SoftFail. A SoftFail says to the receiving server, “the domain owner discourages the use of that IP address as a sender, but you can decide for yourself whether to block it or not”. What’s interesting to me is that an SPF SoftFail seems to be enough to get my spoofed emails quarantined (but not rejected), but once I put the allow list entry in, the SoftFail did not cause my spoofed emails to be blocked.

This gave me the idea that there is a potential solution by using mail flow rules, instead of using EOP filter settings. I created a mail flow rule to allow emails from identityprotection-noreply@identityprotection.microsoft.com only if the SPF result is a pass.

The outcome is that legitimate Azure AD Identity Protection emails will now be received in the inbox instead of getting quarantined by EOP. Any spoofing attempts will continue to be rejected or quarantined by EOP based on my spam filter settings.

To summarize:

• Azure AD Identity Protection emails are being treated as Japanese language by Exchange Online Protection.
• If you are using international spam filtering in EOP, the Azure AD IP emails will be quarantined or rejected, depending on your spam filter policy.
• Adding the Azure AD IP sender email address to the EOP allow list exposes you to spoofing of that email address, so it is not an ideal solution.
• Using a mail flow rule that bypasses spam filtering for the sender address only when the SPF check passes will prevent the quarantine/rejection, without exposing you to spoofing.