Current GUI Causes User and Admin Frustration

The email quarantine process is as old as email itself, and still an essential part of any email service. As far as Office 365 is concerned, Exchange Online is a world-class product and the quarantine mechanism works well, however the current iteration of the Quarantine portal falls short in user experience and performance.

Quarantine, as most of us are aware, occurs when an email or its content is flagged to be potentially harmful to the intended recipient, whether it’s a phishing attempt or a spam email. These emails are automatically corralled and held for review by the recipient or an administrator and then blocked or released.

This protection mechanism, along with the associated user experience, is critical in my opinion because it affects whether one receives legitimate emails or not. Unfortunately, the Office 365 quarantine experience leaves a lot to be desired and deserves some attention from Microsoft.

Management Interface for Admins – Where the Confusion Begins

The quarantine process is hosted by Exchange Online Protection (EOP), a subset of the greater Exchange Online product. EOP performs many of its usual tasks with Artificial Intelligence (AI) and as a result, admins have fewer buttons to push and dials to turn compared to other email protection products. The management interface for Exchange Online Protection may be found within the confines of the Exchange admin center in Exchange Online.

However, the Quarantine settings we need are buried within another portal. As Microsoft works to consolidate the many Office 365 portals, Exchange Online Protection will move from the Office 365 Security & Compliance portal at protection.office.com over to security.microsoft.com, which is currently branded as the Microsoft 365 security center.

While the consolidation of portals is welcome, it does create an extra layer of confusion when admins are trying to solve a problem and can’t find the correct portal to configure a particular policy.

User Interface – The Confusion Compounds

The user experience for email Quarantine isn’t any better, so bear with me as I re-hash the process. I’ll start by logging into Outlook on the Web or OWA at outlook.office365.com. I open my inbox and notice that I have a Spam Notification email (also known as a digest email) which is generated by the Office 365 service and informs me that two items originally destined for my mailbox are now blocked or on hold:

Managing Office 365 Quarantine Confusion
Figure 1: Quarantine email with Block Sender and Review options.

I can either review these messages using the Quarantine page in the Security & Compliance Center, OR I have the option to Block Sender or Review. Note that at this stage we are still unable to release the message, even if we know the sender and are confident that this is NOT a phishing attempt.

*Although the service behaves like this for phishing messages, messages classified as spam can be released immediately by simply clicking the Release button. Microsoft states that “quarantine holds potentially dangerous or unwanted messages,” and, “By default, end-user spam notifications are disabled in anti-spam policies.” For the admin unfamiliar with the default setting for end-user spam notifications in EOP, this means that users may not even notice that emails are held up in quarantine or that they can expire after 30 days.

In this instance, since I know and trust the email address and want to release the message, I click Review, noting that the URL for Review starts with https://protection.office.com//quarantine. My action takes me to Office 365 Security & Compliance portal. Well, it does eventually, after some animation appears, communicating that something is happening:

Managing Office 365 Quarantine Confusion
Figure 2: Security Portal delay animation.

After the Office 365 Security & Compliance portal finally renders, it redirects to the Microsoft 365 security center as the URL changes to security.microsoft.com. There I’m greeted again by the same animation, a loading notification, and then I’m presented with the quarantine view where I notice that there are surprisingly more messages listed along with the one in question I’m trying to interact with:

Managing Office 365 Quarantine Confusion
Figure 3: Message detail in the Security Portal.

At best, this sequence of events lasted well over five seconds, and it was a good 30 seconds before I was able to interact with the message in quarantine. Having used several different vendors for email quarantine functionality, I can say that the time it took between clicking on Review to then arrive at this point, is not on par with other security vendors in this space. Most vendors offer a condensed view with only those emails requiring a review before being released from a performant portal or micro-site.

Now our next step is deciding what to do with the message. I must say, the view of my options is not particularly user-friendly, especially when you consider the non-technical nature of the average person using email:

Managing Office 365 Quarantine Confusion
Figure 4: Quarantine Portal message release options.

Hovering over each option reveals more detail and lets the user know what will happen should they decide to click on it. Not every button shows this extra detail though, which can cause user confusion over what they should do:

Managing Office 365 Quarantine Confusion
Figure 5: Tooltip displayed by hovering over the Remove from Quarantine button.

In my case, I have chosen to release the message, so with that, I click on the Release Message button and the Release message dialogue appears:

Managing Office 365 Quarantine Confusion
Figure 6: Quarantine portal release message options.

A lot is happening with this view, it’s busy and easy to overlook detail such as the option to “Report Messages to Microsoft for analysis,” which is already ticked by default. Noting the accompanying text, we underscore the AI-based nature of the service.

Moving further down the dialog screen, we see our message and see the default option to Release messages to all recipients. As a normal user, I would change nothing here and click Release message. Next, dots will begin to move at the top of the dialog, indicating that something is happening.

After a few seconds or more a notification appears, confirming the message has been released:

Managing Office 365 Quarantine Confusion
Figure 7: Security Dialog screen after message release.

Clicking Done should close the dialog and revert to the grid view, but it doesn’t and returns us instead to the same screen we started with:

Managing Office 365 Quarantine Confusion
Figure 8: Quarantine Portal returns the user to the previous message details screen.

Now, if the user wasn’t confused and/or frustrated before, they will be soon. Not only does this appear to be the same screen we started with, but all the buttons under Details display the same options and without any next steps. Since there’s no indication that the message was already released, many users will assume there was a glitch in the process.

The user is then presented with three options, but only able to select the third option (which would make sense as the message has already been released) but again, this could be confusing for the average user:

Managing Office 365 Quarantine Confusion
Figure 9: Release message dialog options following initial message release.

Finally, I click Close, since that seems to make sense, and I see a grid view that includes the release status of my message:

Managing Office 365 Quarantine Confusion
Figure 10: Quarantine Portal Grid view following message release.

After far too many steps, it appears that I’ve now successfully released the message. And while some of these options in the various dialogs do make sense, I still find the overall experience cumbersome and more painful than it needs to be.

Administrator Grid View – Peak Frustration

If we logon to the Quarantine portal in an admin role, we notice that it is identical to the user view but with the ability to see messages held for all users. The messages may be sorted according to personal preference using the Filter option:

Managing Office 365 Quarantine Confusion
Figure 11: Quarantine Portal Filter Options screen.

Unfortunately, the default grid view for admins does not display the intended recipient of the held email. The Modify Columns button gives a brief glimmer of hope, but ultimately increases frustration by only offering seven modification choices such as Direction and Message ID and Policy Name. Since the grid is sortable based on columns, having more than these seven arbitrary options available while triaging reasons for the email quarantine would be incredibly useful.

Additionally, the view options do not allow me to remove previously released items from the list, which also limits the view to 100 items at a time. It’s important to note that choosing a large message count and releasing them at once often causes the Bulk actions dialog to eventually error out. This can make bulk actions very exhausting for customers who need to triage a large quarantine view.

Make Your Life a Little Bit Easier with PowerShell

More often than not, I avoid the Quarantine grid view entirely and move to PowerShell so I can efficiently execute bulk actions. The Quarantine cmdlets are available using the Exchange Online PowerShell V2 module. They support modern authentication, and feature well-documented examples:

Delete-QuarantineMessage
Export-QuarantineMessage
Get-QuarantineMessage
Preview-QuarantineMessage
Release-QuarantineMessage

Here’s a quick example of the cmdlets in action, displaying the first 1000 unreleased messages for a scoped domain:

Get-QuarantineMessage -pagesize 1000 | where {$_.SenderAddress -like 
"*@*.practical365.com" -and $_.ReleaseStatus -eq "NOTRELEASED"}

If I trust this domain, I can combine a bulk release action. Note that I would not do this idly, but only once I understand the reasons why all email from a particular domain would be blocked:

Get-QuarantineMessage -pagesize 1000 | where {$_.SenderAddress -like 
"*@*.practical365.com" -and $_.ReleaseStatus -eq "NOTRELEASED"}
| Release-QuarantineMessage -ReleaseToAll

Solutions for Both Users and Admins

To minimize user frustrations and pain points, admins have a couple of not-so-great options. For one, they could allow more spam and low confidence phish emails to flow into spam folders, have users deal with them there. Chances are Outlook will also classify the item as spam as it flows into the mailbox and with that method, users won’t experience two different delays, but their mailbox will be more cluttered with spam.

Another option is to train users on the Microsoft 365 Security portal, which would hopefully enable the user to comb through their quarantine queue for items that appear to be missing. While a more tech-savvy user is adept enough to successfully navigate the portal experience, support staff should still expect frustration from many users.

Conclusion

Microsoft Exchange Online, like its on-premises predecessor Microsoft Exchange Server, tends to function pretty smoothly when built and configured according to best practices. However, some aspects of the Exchange Online user experience are less than pleasing, sometimes downright painful, and the admin experience around email quarantine definitely falls into the latter category.

Not to underscore the need for Quarantine, because a security net that filters out potential phishing emails is a necessary business function – I just personally find the Office 365 experience confusing and unnecessarily technical. A big part of the problem is the fact that the Quarantine portal is also being utilized by normal users who are not accustomed to navigating such a technical UI, which is intended more for seasoned admins.

Compared to all the other vendors in this crowded space, the Exchange Online Protection Quarantine experience is sub-par and disappointing. Actions are slow and often time out, especially during bulk operations. The user experience is complex and not performant at all. Administrators have powerful options but are often hamstrung by the same slow portal experiences which users suffer under.

This is an area which requires some attention and a significant investment from Microsoft, both for product performance reasons as well as obvious flaws with the user experience. Improving the user and administrator experience in the Quarantine portal, as well as streamlining the release workflow, would be changes well overdue and welcomed by all.

About the Author

Nicolas Blank

Nicolas is the founder, as well an architect, author and speaker focused on Office 365 and Azure at NBConsult Group in South Africa, England and Hong Kong. Nicolas is a Microsoft Certified Master for Exchange and Office 365, Microsoft MVP (Most Valuable Professional) for Microsoft Office Apps and Services since March 2007. Nicolas has co-authored “Microsoft Exchange Server 2013: Design, Deploy and Deliver an Enterprise Messaging Solution”, published by Sybex and available on Amazon; as well as “Azure Site Recovery: IaaS Migration and Disaster Recovery”, published by Pluralsight. Nicolas is a co-founder of and a contributor to thearchitects.cloud podcast and YouTube Channel, blogs at blankmanblog.com, tweets as @nicolasblank

Comments

  1. Bill

    It’s a year later, and things have actually gotten worse, not better.

  2. Dillon Burke

    How did you get the notification to arrive in the morning? Mine has been Arriving at night.

  3. Dillon Burke

    Is there a way to get a report of how many emails in quarantine were released?

    1. Bash

      Easy.. use powershell

  4. Adam Kuhn

    Very helpful article. Thanks for the review.

  5. Joshua Bines

    Fun fact… if the user selects ‘Block Sender’ the sender added the bad senders on the mailbox and then any future emails are sent to the user’s Junk Folder instead of the Quarantine. As I understand it this to prevent the same sender always remaining in the spam digest.

    ‘If you would like to delete these messages on arrival instead of quarantining them, use mail flow rules’
    https://docs.microsoft.com/en-us/microsoft-365/security/office-365-security/use-spam-notifications-to-release-and-report-quarantined-messages?view=o365-worldwide

    1. David Gorman

      Yes, this is really odd design. “Block the email but you’ll still get the email”.

  6. Keith Privigyi

    I use it in admin only. It’s a nightmare. I referenced all of the O365 help pages and none of them explain exactly what the different buttons do, or what processes are kicked off in the investigation process. It’s a complete mess.

  7. JoshButler

    With the newly released GUI, they have removed the ability to see Policy Name for a quarantined message. This makes it difficult to determine which transport policy was used to make a quarantine decision.

    1. JoshButler

      They added the policy name column back 2 weeks later.

  8. David Anderson

    How do I stop this review email from being sent to users?

  9. Dave Land

    Apparently, a redesign of the O365 quarantine review screens happened in the last two days. Astonishingly, they made it even MORE opaque and impossible to use. I can’t make head or tails out of it now.

    1. Adam Abbate

      Seems like they have improved the interface (marginally) for users – it is definitely quicker now…but made it harder to use for admins!

  10. Mark

    We haven’t found any possibility for a delegate to release a (false-positive) quarantined message to his/her colleague’s mailbox (during vacation, illness etc.). This seems to be another huge problem with Microsoft’s concept for the quarantine.

    1. Adam Abbate

      I’ve found that if the delegate has “Read and Manage” access then they can use the spam notification email to release messages. Still not ideal though!

  11. Joel Oleson

    Very clunky UX. It’s sad that this is the current state. Was really hoping this would have been addressed by now. It’s easy to see why there are so many partners in the EOP space and people are willing to pay.

  12. MikaelJones

    You hit the head on the nail here – I agree on ALL points. I wonder if Microsoft is doing anything about it. With all the investments in the new Security portal they should!

Leave a Reply