The process of writing a book or a training course forces you to spend a lot of time with the subject that you’re writing about. For my upcoming course on Office 365 security, I’ve been spending a lot of time playing with Exchange Online Protection.
It’s funny how something you’ve configured many times before suddenly throws a few curveballs at you when you spend hours or days with it. Little annoyances that don’t bother you when you’re doing a one-time configuration, suddenly become quite annoying when you’re trying to create an informative demonstration for your students. In this post I’m going to cover a few of the little annoyances in Exchange Online Protection that have bothered me recently. Some of these will be well known to many of you, but perhaps you’ll see a few new ones in the list.
I’m also conscious of the fact that having spent so much time in EOP lately, I might be overlooking easier workarounds or solutions. So, if you have any input on these, please feel free to leave a comment below.
Which Admin Portal to Use?
Exchange Online Protection (EOP) is included with Exchange Online plans, and is also available standalone for on-premises customers. Initially, admin could be performed via the Exchange admin center (EAC).
Later, Microsoft developed the Security & Compliance Center (SCC) ; a separate admin portal focussed on security and compliance features. The ability to manage Exchange Online Protection was added to the SCC, creating a situation where there are now two places to configure EOP.
As much as I dislike having the same thing available in two different portals, it’s not the end of the world. But it would be nice if the admin experience was consistent between them. Unfortunately, it isn’t. For the most part, I stick to the EAC as I find it less buggy. The only time I venture into the SCC is to configure the bulk email options (a one-time task), or deal with spoof intelligence (very rare).
Here are some examples of the inconsistencies between portals.
In the Exchange admin center, the connection filter policy can be accessed without touching anything else. So, if you want to add an IP address to your connection filter allow list, you can easily do it.
In the Security & Compliance Center, there’s no obvious way to manage your connection filter policy.
If you make the reasonable guess that the connection filter policy is found in the Anti-spam section, and your spam policy uses the standard settings, you still won’t be able to see or manage the connection filter policy.
It’s not until you drill down to the custom settings, and then enable custom settings (if not already enabled), that you’re able to manage the connection filter policy.
In other words, settings that are decoupled in the EAC are coupled together in the SCC. A consistent admin experience would be preferable.
Bulk/Phishing Email Options
Another difference between the EAC and SCC is the options for dealing with bulk and phishing email. In the EAC, if you configure the spam and bulk actions, the UI doesn’t make it entirely clear what will happen to bulk mail. The wording “mark bulk email as spam” could mean treat it as “Spam” (which would move it to the Junk Email folder in this example), or treat it as “High confidence spam” (which would quarantine it in this example).
In fact, bulk email handling is controlled by a setting not visible in the EAC at all. In the SCC, there is an explicit option for how to treat bulk email when bulk mail filter is enabled. Another setting also exists for how to treat phishing email, which is also not present in the EAC at all.
Bulk filtering is enabled by default on new tenants today, but older tenants that preceded the availability of the feature were not turned on by default. Further complicating the admin experience, if bulk mail filtering is not enabled in your tenant, then turning it on via the EAC gives you a result that is inconsistent with your existing spam actions. In the example below, bulk mail filtering is not enabled. The spam action is set to prepend the subject line, and the high confidence spam action is set to delete the message (I’m using these actions to illustrate the point, not because I think they’re optimal settings).
When bulk email filtering is enabled, you’d expect it to adopt the action of either prepending the subject line, or deleting the message. But, if you check in the SCC, you’ll find that bulk email filtering defaults to “Move message to Junk Email folder” instead.
Language/Region Pickers for International Spam Filtering
If the SCC has the advantage for bulk/phishing configuration, it loses that advantage if you need to configure international spam filtering. In the EAC, configuring international spam options gives you a useful picker from which you can select the languages/countries that you want to filter mail from. This makes it trivial to make bulk changes to the list (e.g. adding all languages except for English).
In the SCC, you get no such help. To add anything to the list, you need to start typing characters to perform a search, or know the specific language/country code you want to add. This makes it far more difficult to make bulk changes.
The Standard vs Custom Settings Switch
One of the more irritating UI gripes with the SCC is the Standard and Custom settings switches, which are located on separate tabs on the page. After making some policy changes (i.e. customizing my policy), the Custom switch will be on, and the Standard switch will be off. Leaving that page for a while, and then returning, will show that the Standard switch has reverted to the on position, and the Custom switch is now off again. This is despite the non-default/custom settings (e.g. international filtering) being visible in the policy.
Spoof Intelligence and Action Center
Worth a mention here, the Spoof Intelligence settings (reviewing spoofed senders and allowing/blocking them) is only available in the SCC. Meanwhile, the Action Center is available in the EAC, but not in the SCC area with the other EOP settings.
Moving on from EAC vs SCC complaints, there are further issues in the admin UI for EOP with the terminology used. When you configure spam filter actions, there are two classifications:
- High confidence spam
Elsewhere in the EOP policy settings, different terminology is used for other features. I’ve already covered the EAC bulk email UI text of “Mark bulk email as spam” and how that doesn’t provide a clear understanding of what action will be performed for bulk mail. Here’s some other examples.
In spam filter policy block lists, does “Always mark email from the following senders as spam” mean “Spam” or “High confidence spam“? You need to go to the TechNet documentation to learn that it means treat as “High confidence spam“.
What about international spam? Those options use different terminology, “Filter email messages…“. Does that mean “Spam” or “High confidence spam“? Again, it’s necessary to check TechNet to learn that it means “High confidence spam“.
In the advanced spam filter options, the wording is slightly different again. We already know that “Mark as spam” means “High confidence spam“, but what does “Increase spam score” mean? The UI is unclear.
Again, we need to visit TechNet to learn the answer.
When enabled, these options set the spam confidence level (SCL) of a matched message to 5 or 6, which is considered suspected spam. The action performed on the message will match the Spam setting in your content filter policy.
So, “Increase spam score” means “suspected spam“, so EOP will take the “Spam” action.
Meanwhile, “Filter” and “Mark as spam” both mean “High confidence spam“.
When you know the answers, it’s easy to know what effect your EOP policies will have. For new admins, the inconsistencies are frustrating, and add an unnecessary learning curve.
Limited Flexibility with Spam Actions
Let’s say you wanted to achieve the following outcome in your spam filter policy:
- Tag the subject line of suspected spam (not high confidence spam) with the words “[Possible spam]”
- Tag the subject line of bulk email with the words “[Bulk email/Marketing]”
In the spam and bulk actions, there is an option to “Prepend subject line with text“, which will allow you to tag the subject line of those emails.
However, if you choose that action for two or more types of email, you are still limited to a single string of text.
As a workaround, if you want to tag the subject lines with different text, you can set one of them to prepend in EOP, and other to add an X-header. Then, in your mail flow rules, look for that X-header and apply the other subject tag there.
Why This Bothers Me
I sat on this post for a few days wondering whether I should even publish it. I don’t like to just complain about stuff, but sometimes problems need to be brought to light. Ultimately what made me decided to publish this post was putting myself in the shoes of a new Office 365 customer, or a beginner Exchange Online admin. It’s easy for me to overlook the issues because I have found workarounds for them, such as using one console vs the other, or because I understand the ambiguous terminology being used in the UI. But for someone who is new, it’s a frustrating journey to get through all those things. It’s hard enough to deal with all of the changes in Office 365, without having to deal with all these little inconsistencies as well.
Microsoft is obviously working hard to make customer and administrator lives easier. We see improvements every day in the products. Hopefully some of these little things will get the attention they deserve as well.
Photo by Henry Hustava on Unsplash
Great Article, feeling the pain as well.
You can’t even easy submit a junk or phishing scam message to Microsoft. Crazy!
Create a blank email message.
Address the message to the Microsoft team that reviews messages, as follows:
For junk messages: firstname.lastname@example.org
For phishing scam messages: email@example.com
Copy and paste the junk or phishing scam message into the new message as an attachment.
It’s more than a year since this blog is released but still the UI gripe on the standard/custom switch issue is still there. This is very annoying. Does it mean the custom setting will not apply if the switch is off?
When you have multiple anti-spam policies configured, such as for users @domain.com, domain2.com, domain3.com. I noticed that each policy has its own allow/block list, does that mean that if you wanted to truly whtelist a sending email address that you would have to add it to all the domains you have configured with custom policies?
When analyzing spam email headers to blacklist, I’m pulling the CIP address, sometimes the originating IP address and any available domain names to add to my spam filter block list. I have not been adding any IP’s to the Connection Filter. Should I be adding CIP’s (or anything else) to the Connection Filter? I guess this brings me to my main question: What is the difference between the Spam Filter and the Connection Filter?
I have been and Email Admin for a long time, EOP falls short of any real solutions. Ironport or Proofpoint. After using this for a few months, If I was writing the checks, I would go back to a premium solution. We are constantly getting hit by phishing and spam now that we switched over.
eh, just popped a few Panadol to deal with this flipping UI mess that is not as inconsistent as you say, but much much worse! I won’t bash it this time round (ok why not – Pop up after pop up!, never ending scrolls in admin & drop down showing items per page in eac (more inconsistencies!), scattered UI, DIRSYNC!!!, FLAT FSKNG UI!) more Panadol.
Anyways, I am here because I found this article useful in that I am not the only one sharing these grievances. Everything MS is ARPITA to admin these days. It gets much much worse.
Also, just wondering if there are exchange online powershell commands to block emails coming from server ip addresses in the mail header? Set-HostedConnectionFilterPolicy is not working because we use messagelabs to apply spam policy. I just want to block CIDR ranges if found ANYWHERE in the header. I can’t seem to find a way to do this.
Has anybody any idea what setting wins between these two things:
1.) EXO’s ECP -> Protection -> Spam Filter -> Edit a policy -> Advanced Options -> SPF record: hard fail [On | Off]
2.) S&CC’s -> Threat management -> Policy -> Anti-Spam -> Phishing Email
I actually just found that the SPF record: hard fail setting is actually available in S&CC under “Mark as spam” settings when editing a custom policy. Makes me wonder what all “Phishing email” entails.
Phishing email is emails identified as likely phishing emails (I know right?) by the spam/content filter, based on whatever algorithm they have detecting that for them. I assume it uses SPF and other domain auth mechanisms as signals for that algorithm, but also other things (e.g. links, phrases, etc).
The control for taking action on phishing email is only available in the SCC, not in EXO/EOP. I assume thats because the SCC is where most of the development effort for these policy UI’s is being invested.
I’m looking forward to the time that hopefully comes where they just remove the controls from Exchange Online’s ECP. That’ll make things easier to work with.
simple answer to all this – dont use EOP 🙂 We use Symantec cloud now and have never looked back. the granularity you get is way ahead.
Great post Paul and I also feel the pain of two consoles. I do like the new Quarantine view in SCC vs EAC as it splits out Spam, Transport Rule, Bulk, Phish, and Malware. One thing I wish it would show is the emails that have been released in the view. Also if I’m not mistaken the HTML preview view is safe in SCC vs EAC where I also looked at the RAW HTML.
Note to other readers:
Following the advice Paul has mentioned on this site we noticed a good reduction in Spam/Phishing/Malware/etc. when moving to EOP from our on-prem solution. We took our time and reviewed every option and made sure we had the desired settings already configured when we flipped the switch (MX).
thanks as always Paul. I’m at that phase of deciding whether to go with EOP when I move the MX records to 365 or go to a 3rd party solution. I’m hearing from another 365 customer that a lot of spam gets past EOP and I’ll be crucified if that happens. How do you rate it as a spam filter overall?
EOP is fine. Out of the box it isn’t as effective as it can be. All these systems need tuning to suit an org’s expectations. One of the problems is when people compare the default EOP config to their previous solution that had years of tuning and tweaking applied to it.
Great article – really helps clarify some key settings / options. Thank you!
Great article Paul. Exactly feel the same.
I had the same feeling about EOP and SCC, it’s a little confuse to understand all this features, mostly because you don’t find everything in just one place.
Microsoft could consolidate it in just one place. I’d rather EOP than SCC.
Thanks for sharing with us your thoughts.
Glad to know it wasn’t just me!
You don’t want to hear my gripes about dealing with quarantined email.
I have a ton of rules to intercept phishing emails before they get to users, and trying to find legitimate ones among hundreds of actual phishing emails is a pain in the a**! The GUI is incredibly slow, the email presentation (to see ‘is this a phishing email?’) is near useless and I’m constantly flipping between screens to block or create a new rule based on this phishing email.
Another huge problem with ATP is that it add more jargon onto the link confusing users. Every link appears to users as if it leads to Microsoft.
I’m pushing to get another product that can help, and not hinder work.
The ugly rewritten links in ATP Safe Links are supposed to be going away some time in the near future.
Thank you! It will surely get more attention from Microsoft now that you’ve highlighted it on this stage (and Twitter). Here’s hoping anyway.