Noticed an increase in legitimate email going to the junk mail folder lately? You’re not alone.
I’m personally seeing the same high rate of false positives, as are some of my customers. The junked emails have a safety tip inserted by EOP that says:
This sender failed our fraud detection checks and may not be who they appear to be.
I’ve previously written about this safety tip here. The solution then was to fix your SPF records. The same advice applies here too, but it’s a little more complicated now, as you’ll see below.
The common element of each customer where I’ve seen this happening is that they have Office 365 ATP licenses in their tenant. I also have ATP licenses in my tenant. This commonality wasn’t immediately obvious until some background discussions in the last 24 hours, and the posting of this Office roadmap item.
The anti-spoof capabilities referred to in that roadmap item are different, but related to, the ATP anti-phishing policies I wrote about recently here. Anti-phishing policies look for lookalike domains and senders, whereas anti-spoofing is more concerned with domain authentication (SPF, DMARC, and DKIM).
You’ll notice that the roadmap item was just added in the last 24 hours, and was immediately listed as “rolling out”. That means the feature is in production. However, there’s no public documentation or other announcements available to let customers know what to do about it. The reference to “policies” in the roadmap post likely mean there will be admin controls released for this behavior in the near future.
Some tweets from my fellow MVPs explain what’s happening.
This change is one of those “good idea, bad execution” situations that could have been handled a lot better from a communication perspective. But it’s out there now, so what do we do about it?
The first and most obvious step is to review and fix your SPF, DKIM, and DMARC records. I’ve covered SPF in the past here. You can also look at Microsoft’s documentation for SPF, DKIM, and DMARC. That should solve the issue for emails sent from your own domains, for example externally hosted apps that use your domain when sending to your Exchange Online mailbox users.
Once you have your own house in order, the question is then what to do about other senders getting junked when they send email to you? That’s a little tricker, because you can’t fix someone else’s domain authentication problems. You can certainly help them to diagnose the problem and suggest the fixes, but these situations often degrade into a “it’s not our end” finger pointing exercise. If it helps, point them at this advice from Microsoft, particularly this quote:
The newest anti-spoof features help protect organizations from external domain spoof. Office 365 honors emails from external domains having proper SPF, DMARC, and DKIM authentication settings enabling them to pass authentication, and junks messages that fail this authentication. The challenge occurs when external domains do not have these settings properly configured…. (We recommend) admins of sender domains into Office 365 update SPF, DKIM, DMARC configurations so emails can pass the stricter authentication rules.
If all else fails, it may be necessary to apply some whitelisting of domains in EOP for important senders that you want to receive email from. That’s not ideal, as it will also cause other mechanisms in EOP to be bypassed.
On the bright side, the longer the other domain owners ignore the problem, the worse the situation will get for them. Eventually, if they want to reliably send to Office 365 customers, they’ll need to fix their domain authentication problems.
This is still an issue in 2021, there’s nothing remotely intelligent about Microsoft’s “spoof intelligence”
Basically 90% of emails that have a valid SPF/DKIM records, but no DMARC record fails their comp auth and ends up in in the users junk folder. These are companies using third party senders like mailchip, sendgrid, shopify, etc..
Yet temp errors to SPF and DKIM with no DMARC = compauth pass.. um what?
There’s no point submitting to Microsoft because every single time the verdict is “should have been blocked”
Meanwhile my tenant allow/block list grows by the day that it almost seems pointless having spoof intelligence enabled.
I’m regretting moving from Google Workspace to O365 that’s for sure.
You my friend seems pretty much fed up with 0365
Hey Paul,
our connector is setup to run all emails internally (RouteAllMessagesViaOnPremises : True), and still some of emails end up in EOP quarantine. We are getting SPF soft fail and SPF fail error when I lookup headers. Any hints?
Thanks,
M
Microsoft recommendation for SPF settings should look like below if your mail is hosted in O365.
Will this allow a domain hosted in O365 to spoof another domain that is also hosted in O365?
v=spf1 include:spf.protection.outlook.com -all
I’ve been seeing that type of spoofing, if sent from another domain that uses office365 then SPF will pass if it’s set up correctly (spoofers/phishers often do).
To combat this i created a rule to check for the presence of “dkim=fail” in the Authentication-Results header. Legit inter-domain emails won’t have a DKIM signature if sent through office365, but spoofed inter-domain emails will have a DKIM signature for the originating domain. This will cause DKIM to fail.
Create a rule that looks for that header for emails sent FROM domain TO domain, if the header matches dkim=fail then it’s likely a spoofed email.
– If sender domain = example.com
and
– If recipient domain = example.com
and
– Authentication-Results header includes dkim=fail
then
– Quarantine and send Email incident report.
We are running a newsletter service that is sending mails on behalf of our customers. We get a lot of complaints that mails to Office 365 / outlook.com are marked as spam. When analyzing the headers of affected emails we find that all authentication checks have passed. This is an example:
authentication-results: spf=pass (sender IP is 88.198.181.214)
smtp.mailfrom=n2g35.com; microsoft.com; dkim=pass (signature was verified)
header.d=zsvr.org;microsoft.com; dmarc=pass action=none
header.from=zsvr.org;compauth=pass reason=100
What else can we do to find out whats the reason for these mails being marked as spam and to avoid that ? Is there any technical support site that we can contact ? When going to outlook.com support there is only support available for the product itself but not for technical questions like these.
A lot (75% to 80% of incoming spam are false positives) of incoming (and legitimate) mails are falsely marked as Spam and moved to the spam folder. We already set this Spam Filter Rule https://imgur.com/a/1nZFEfc but mails are still moved to Spam. Office 365 support does not help. They say ‘we do not support Exchange Online Admin Console’. We have tried several times. Management is furious, we have over 500 users. What can we do?
The Real Person!
The Real Person!
Escalate the ticket. It’s true at all that they don’t support the Exchange admin center, that is rubbish. You’re paying for Office 365 support and that includes support for spam filter configuration issues.
Note also that those aren’t the only spam filter options. There is also the international options and the advanced options you can see there. Some of the options are only surfaced in the Security and Compliance Center (the phishing controls for example). A setting you can’t see in the EAC might be causing the filtering decisions. Also, inspecting the headers of the false positives and comparing with the antispam heaer documentation on TechNet should help you narrow down what is causing the filtering to happen.
I’m an IT Manager for a company and we’ve been experiencing the same thing. We are on GSuite and after extensive research, I found that almost all emails going to Office 365 accounts were being marked as spam. SPF, DKIM, and whatever else come back clean. Finally got a Tier 2 technician with Microsoft and was told: “well, that’s what happens now.” It’s impossible to ask every Office 365 user in the world to whitelist our domain. Anyone found some way to fix this?
The Real Person!
The Real Person!
Have you seen headers from Office 365 showing why your messages were junked?
This is very bad and in practise disqualifies Office365 as mail service. We cannot have important mails from our customers being held back at random. As we are using an external MX preprocessor before turning mails over to Microsoft, SPF will always fail. But there is no option to disable SPF?
The Real Person!
The Real Person!
It’s a net positive. Microsoft is being aggressive about it, but other mail providers are heading in the same direction. So this is going to be a problem for anyone not handling their domain auth requirements properly.
If you have an MX in front of Office 365 then Microsoft has guidance for those scenarios. Start here, and then check the links at the end of the article: https://blogs.msdn.microsoft.com/tzink/2018/06/05/if-your-mx-record-doesnt-point-to-office-365-how-do-you-disable-spam-filtering-in-office-365/
We have seen an a dramatic increase in this over the past two weeks across our client tenant that have ATP, also including our own tenant.
I raised this with Microsoft support and they found a corrupt Junk Email rule on my mailbox which I never created, this was also apparent on a number of other mailboxes on our tenant (yet to confirm this on our client tenant)Microsoft’s only suggestion was to use MFCMAPI.exe on each mailbox exhibiting these behaviors and remove all Mailbox Rules.
I dubious of this ‘fix’ but wanting to know if you have seen any improvement in this Junk Email problem across your tenants or still seeing it largely? I have added DKIM to our domain (already had SPF), but the receiving emails going are coming from a vast array of people who we have no control over their SPF/DKIM/DMARC.
The Real Person!
The Real Person!
There happens to be a similar issue going on right now that is published in the Service Health Dashboard. Messages containing “proofpoint.com” in the message, body, or HTML markup are being incorrectly marked as phishing.
We ran into this issue too re “ProofPoint Spam”. However it was a little different (maybe related) as messages were being sent to Quarantine at O365.
The original issue of messages being sent to the Junk mail folder at Outlook still exists at our tenant.
I’m interested in this MFCMAPI.exe tool. Wondering if that will resolve it on our end as well.
In order to qualify for an SPF=Pass, does the SPF record need to specify a “Fail” mechanism? Will the SPF=Pass also be generated if a “SoftFail” mechanism is defined?
In other words, will both of these qualify for an SPF=Pass as long as they are generated from O365?
v=spf1 include:spf.protection.outlook.com -all
v=spf1 include:spf.protection.outlook.com ~all
The Real Person!
The Real Person!
Yes, a pass is a pass, no matter what mechanism you’re applying to “all”.
Has anyone experienced an issue with accounts being blocked from sending outbound email to be more aggressive then before? Something has changed recently in the filtering of outbound spam, in the past week we have had two accounts that have been blocked for “sending spam”. both cases the emails were legitimate emails (single email with over 100 recipients)
Anyone know the changes that were made on the Microsoft protection system that is making this so aggressive?
The Real Person!
The Real Person!
I’d suggest that your issue is unrelated. A single email with over 100 recipients is suspicious to me. You may consider it “legitimate”, but that sounds likely to be bulk/marketing email. If you think they’re being blocked incorrectly, open a support ticket with Microsoft.
Is it possible to turn of ATP for he whole tenant?
And do it fast?
– and will this solve the problem?
The Real Person!
The Real Person!
If you have ATP licenses, then this change applies to you. If you want to remove ATP entirely from your tenant, then I recommend you contact Microsoft support for guidance.
But I doubt that will “solve the problem” for you. These changes in handling of mail from domains that are poorly configured for SPF etc are going to become more widespread across Office 365 and the entire industry.
This is a response from our support ticket with Microsoft.
“The compauth result is only stamped for users with ATP license.
This stands for composite authentication and is Office 365’s combined SPF, DKIM, DMARC, and all other internal results. If a message fails explicit authentication (DMARC quarantine/reject), the reason code will be 000. If a message does not publish a DMARC record and Office 365 has no valid signals on the message, the reason code will be 001
You may Whitelist the sending address, Domain or IP If you think these are legitimate sender for you. “
Hello,
This kind of issues happens all the times with EOP. 100% legit senders got marked as spam or phishing even if they have SPF applied. One of the Polish service providers domain has been marked as spam which shouldn’t happen.
Does it make difference if the domain only has SPF or it has, both SPF and DKIM?
How to mark sender as safe without whitelisting it?ž
Thanks,
Dorian
We have previously had this check enabled and had no issues however, from Friday, it was noticed that we had large quantities of emails (including purchase orders) going to Junk folders. when I looked at what O365 was reporting and what the email headers contained, O365 was identifying the incorrect source IP address so even customers who have SPF enabled were failing and going to junk! This has now been identified by MS and being worked on.
For those with ATP in their tenant, a “workaround” to disable this new authentication check:
The parameter that should be turned off is called EnableAntiSpoofEnforcement and the default value for this is True.
The commandlet that we have to run in order to disable this new feature is the following:
New-AntiPhishPolicy -Name “Test Policy” -EnableAntiSpoofEnforcement $False
Also, this policy MUST be enforced trough a rule and the rule can be created with the below command:
New-AntiPhishRule -Name “Anti Phish Rule” -AntiPhishPolicy “Test Policy” -Enabled $true -RecipientDomains “*”
Hi, the second command is wrong
The last option should be RecipientDomainIs
New-AntiPhishRule -Name “Anti Phish Rule” -AntiPhishPolicy “Test Policy” -Enabled $true -RecipientDomainIs “*”
Thank you for this! My response from contacting Microsoft was “Based on your query I would like to inform you that anti spoofing protection cannot be disabled.” After informing them that there is indeed a way to disable it they then sent a follow up: “As informed to you earlier there can be some work around to disable it. Microsoft do not recommend to disable this functionality.”
Love it.
Also, if you already have a policy set, and need to disable the Anti-spoof enforcement, you can use:
Set-AntiPhishPolicy -Identity “Anti-Phish Policy name” -EnableAntiSpoofEnforcement $false
More options here: https://docs.microsoft.com/en-us/powershell/module/exchange/advanced-threat-protection/set-antiphishpolicy?view=exchange-ps
This is poor execution right in the middle of tax season, my users are quite angry right now. It seems to me that this change kicked in a week or two ago, although I haven’t investigated it too much. The problem is that we can’t disable this feature at all through the anti-spam “spoof intelligence”. I would like the option to append to the subject line the same as spam, so they at least receive these legitimate client emails to their inbox.
Their support solution to me yesterday was to whitelist the sending domain which is completely impractical, there are hundreds going to Junk Email now.
I changed my spam settings yesterday to append to the subject line and I notice these “spoofed” emails are going to Junk Email without the subject line change. I can’t think of anything to do other than tell the users to check their Junk Email folder often…
On the bright side we know which clients could use some IT services.
Very poor execution really and total lack of communication and technical readiness from Microsoft 🙁
Email security world is mainly a matter of TRUST and REPUTATION: releasing new “features” and managing changes in this way definitely lower the trust in Microsoft EOP and ATP from customers.
The Real Person!
The Real Person!
What I’m seeing with my customers is a lot of the mail they get from other small businesses (and most of their dealings are with other small businesses) are getting junked, because a lot of small business just use whatever email is provided with their web hosting. And most of the time the web host doesn’t help them get set up with SPF, let alone DKIM or DMARC.
All messages that i have examined and that were send to the Junk mail folder have value that i dont recognize. Does anybody knows what this means?
compauth=fail reason=001
1 Authentication-Results spf=none (sender IP is 23.83.215.29) smtp.mailfrom=technischburodunning.nl; alfa.nl; dkim=none (message not signed) header.d=none;alfa.nl; dmarc=none action=none header.from=technischburodunning.nl;compauth=fail reason=001
The Real Person!
The Real Person!
Compauth has just been explained to me recently. It means “composite authentication” and it’s basically a rollup of other signals. A fail is not good. In your case there it seems the lack of SPF record (spf=none) for the sender domain is the problem. This is consistent with what we’re learning about this new ATP spoofing protection. SPF is now very, very important.
Just wonder why some of Microsoft own domains are treated as Junk?
e.g. the automatically e-mail sent from Microsoft as “Your Office 365 group ‘NAME’ expires in 15 day(s)” with sender address msonlineservicesteam@ssgm.microsoft.com neds up in Junk and also seen some of those message sent with msonlineservicesteam@microsoftonline.com.
Little frustrating for the Site owners if their groups will expire and deleted if the miss the e-mail ending up i Junk folder.
The Real Person!
The Real Person!
I assume the domains aren’t set up correctly. All I can suggest is you raise support tickets for it, and I’m sure the team managing this ATP feature will address the issues promptly.
Hi, we have similar issue and found that EOP is identifying the wrong source IP address therefore fails SPF even when this is set up correctly
Like Kenny, I can see the same things.
Thank you Paul for this article. I think we must change retention policy for junk folder and inform our customers.
If need, I’ve got some emails in my junk folder from.. MS Support. Here is a header part:
16 Authentication-Results-Original spf=pass (sender IP is 40.107.4.95) smtp.mailfrom=microsoft.com; outlook.fr; dkim=pass (signature was verified) header.d=microsoft.com;outlook.fr; dmarc=pass action=none header.from=microsoft.com;
17 received-spf None (protection.outlook.com: microsoft.com does not designate permitted sender hosts)
@Terry Is there something of a absolute minimum to set up in terms of SPF/DKIM/DMARC in order to avoid being junked? We have a customer that have a problem that their mails are being junked since a week or so. They do have a valid, and correct, SPF record, but haven’t got DKIM or DMARC set up just yet.
The culprit for our customer has been identified. A http link to the company website in their users’ mail signatures seem to be the problem. If the link is omitted, the mails are not junked.
Now the problem is that their web site is hosted by a major hosting company, and the IP of their www A record is pointing to the hosting company’s load balancing server farm.
The hosting company hosts loads of web sites. My guess is that one, or more, of the web sites hosted there are fraudulent.
How can we find out which web site?
Is there a way of asking to “whitelist” a URL? If so, how?
I have seen this mentioned anywhere. Junking in EOP is a function of the Spam policy. Depending on the SCL score, it will be let through, junked on the way in or quarantined. Whitelisting sets the score to -1.
I’m the feature owner of Antispoofing in Office 365. To answer the question, if a user adds a sender to their safe senders list, then it will not apply the antispoofing to move to junk, nor apply the safety tip. However, it also bypasses our other protections. So, if you get a message from that sender and it *is* spoofed, it’ll go to your inbox. This is a vector for phishing attacks, we’ve seen this over and over.
Also, setting up SPF/DKIM/DMARC makes the problem go away (mostly… there are issues with mailing lists and other indirect mail flows). However, the intent behind antispoofing was to implicitly authenticate email even in the absence of SPF/DKIM/DMARC. There is a lot of backend intelligence to look for signals that an unauthenticated message is legitimate, we don’t junk all unauthenticated email. However, the long tail of smaller senders has proven problematic.
That being said, isn’t it better than to tell end users to add a sender to their safe list then to put a global allow in place as Paul mentions? This way if that sender is spoofed it will only get through to that 1 user who has it on their safe list as opposed to the entire tenant because it’s on EOP’s global allow?
Or am I looking at it wrong?
Spoofed email will get through only to the user who has added the sender in their safe sender list and not all the users.
setting a global allow is only advised via transport rule that checks if SPF or DKIM passed,
so basically your in the same problem zone no SPF -> no whitelisting
sure you can whitelist but that a one way ticket to success for spoofers
We have never published SPF records for any of our domains. Now some recipients of emails that are sent from mailboxes residing in O365, through their OWA/Outlook clients IPs are getting the Fraud Detection Banner. This is occurring for known and frequent correspondence, but only recently the banner is getting appended. How can Microsoft say that SPF records are not a requirement, when it seems that they are forcing them to be. We have over 30 different third-party hosts that also send mail as some of our accepted domains, which reside on different IPs where it would be impossible to create an SPF record. I would understand if the banner comes from the third-party hosts, but my gripe is that even if email is sent out legitimately from O365, it’s still considered fraudulent!
The Real Person!
The Real Person!
More and more mail services are treating domains without SPF and other authentication mechanisms as likely spam. Some very aggressively. Office 365 is not the only one.
If you want your mail delivered reliably in future, you should adopt the domain authentication standards that the industry is standardising on. If you’re using services that don’t support those standards, you might consider moving to services that do.
I think this is the single biggest challenge we have seen with regard to moving to Office 365 and email spoofing. When we had our on-premise implementation, we had our 3rd party services sending emails with our spoofed address to our edge servers (rather than our gateway which sat in front of our edge servers). Our edge servers were setup to reject any email from the gateway that had our email address on it since no email with our address should ever originate outside of our organization.
Microsoft’s solution is to get email admins to configure thier DNS records confirured correctly. But trying to add all of our third party services (and their ever evolving mail systems) into our SPF records is a heavy lift. Creating a subdomain for our vendors is the best solution I have heard to address this. But it is administratively almost as burdensome as the SPF and can be circumvented by a clever phisherman.
Email gateways are real expensive. And I am not impressed with ATP yet although it is improving incrementally.
@Terry, you may want to relay the local Safe Sender exclusion behavior to the Microsoft Premier Support team. The guidance we previously received is that an Exchange Transport Rule setting the Spam Confidence Level to -1 is the only way to override this behavior, and that the local Safe Sender exclusions are not honored.
This has been a headache for sure this week for us.
Is it safe to assume that if an end user marks a sender as a safe sender it will allow that email in without going to Junk?
The Real Person!
The Real Person!
Maybe not safe to assume, but if you notice that it is relieving the issue please let us know.
It’s anticipated that as ATP learns more about normal send/receive patterns for your users it will get better at making accurate filtering decisions, but I suspect that if the sender has wrong or no SPF/DKIM/DMARC set up then it will just keep junking them.