Why User Submissions of Suspicious Email Delivered to Exchange Online are so Valuable

Improving the User Submissions Feature

User Submissions is a Microsoft 365 feature that allows users to report bad emails that they receive. Simply put, this feature is valuable because it enables a ‘boots on the ground’ defense for email. Malicious actors constantly look for new avenues to attack organizations through tricks such as social engineering, URL links, and attachments. One potential attack surface is an organization’s end users, trained or not, who will receive Phishing or Junk email at some point in time. With User Submissions, end users have the ability to report these emails to IT or Microsoft to protect against this type of attack. Reported messages will be analyzed by Microsoft, your IT department, or both depending on the configuration, and the results are added to Microsoft’s mail hygiene engines to help reduce false positives and identify these types of emails.

Additional articles on the User Submission process written by Tony Akers can be found here and here.

You may be familiar with the Report Message button option in Outlook, but did you know that there is more to be configured and that the backend interface has moved to Settings > Email and Collaboration > User Reported Settings in the Microsoft 365 Defender portal? You may have configured settings here in the past, but Microsoft moved User Submissions to the Settings section of the Defender portal. It’s here where you can customize the user experience in terms of branding, messaging, and information provided to the end user. In this article, we explore these changes and explain why you should make more use of this feature.

Recent Changes in User Submissions

Microsoft constantly changes cloud features by either adding enhancements or changing the GUI. Here is a quick summary of the changes:

• As mentioned above, the configuration for User Submissions moved from Policies and Threats to Settings, which seems to be more of an operational change rather than functional.
• The option to use the Integrated Report button in Outlook or the Report Message / Report Phish Outlook add-ins. However, this option is transitional, as the add-ins will stop working at some future time (Figure 1).

• Custom messages for when an email is reported – IT can provide links to a FAQ or contact number for the Help Desk.
• Identifying which type of message is being reported, for example, Phishing or Junk email.
• Company Branding can be added to the admin review email, an email sent back to a user confirming the analysis of an email as malicious/safe or junk/not junk.

Customizing User Submissions

User Submissions can be configured in such a way that organizations can do the following:

• Set expectations of your end users: When can they expect a response, who can they reach out to for support, and provide a FAQ page and phone numbers, if needed.
• Provide transparency: Thank them for reporting the message, as it helps everyone at the company.
• Alleviate mistrust: Some users may mistrust Microsoft, and if the communications use the Company Branding integration with User Submissions, it may raise their confidence that the report will be actively looked at and resolved.

User Submissions is disabled by default, so the first step is to enable it. Once this feature is enabled, it is recommended that administrators use the default slider button (see Figure 2), enabling the built-in Report button as that will be the default option, and the Outlook add-ins will be deprecated.

Next, we choose who the message is reported to. The message report defaults to Microsoft, but we can choose a custom mailbox and Microsoft or just select a custom mailbox. Once that is done, we can work on some customizations to publish to users:

• Custom Reporting Message
• Custom Post Reporting Message
• Custom Admin Review Message
• Company Branding

Custom Reporting Message

Customized pop-up messages when reporting emails can provide IT an opportunity to educate and inform users of their part in the process of reporting malicious emails. With only 300 characters to work with, keep the message short, concise, and informational such as in the example below.

Click <b>Report</b> below to complete the process.<BR><BR>Thank you for reporting this %type% email to IT Support. We will review this message and provide a response withing 72 hours. Visit our <a href="http://intranet/emailsupport.html">FAQ</a> or contact IT Support @ 312-555-1212 with questions.

As you can see, we layout critical pieces of information:

• Thank the user for reporting the message so they feel like part of the process.
• Providing next steps – IT response in 72 hours (based off an internal SLA possibly).
• Providing a link to Frequently Asked Questions (FAQ) for follow-up.
• Providing an IT Support phone number for further support.
• Visual cue provided with the ‘%type%’ exposes which message type is being reported.

As shown above, the text can include HTML coding such as bold text (<b></b>), hyperlinks (<a href=””></a>), and line returns (<BR>) to generate an informational message for the end user.

Custom Post Reporting Message

A second message can be configured that acknowledges the completion of the reporting process, which can also be used to thank the end user, as seen in Figure 3. However, the message appears so briefly that the user reporting the bad email may never notice it.

Custom Admin Review Message

After an administrator reviews the reported message (in the Submissions section of the Defender portal) and decides what to do with the message, an email about the decision is sent to the user to tell them what happened. There are three types of message that a user could receive – No Threats Found, Phishing, and Junk – and the message text should be similar to the other messages, thanking the users for their involvement, providing the next steps, and any contact information needed for follow-up. Once a user is notified, an Administrator should then report any malicious emails to Microsoft for analysis and choose Phishing, Malware, Spam, or Trigger an Investigation.

The message is sent once an administrator reviewed the email, determines the email’s status, and picks an option from the drop-down shown in Figure 4.

When creating the Automated email response for Phishing, Junk, or No Threats Found, a couple of pointers for this block of text are:

• HTML code is available for use.
• There is an 1113-character limit, and the text box is expandable for ease of use.
• The Footer box is used for all three types of messages, so tailor this as generic as possible.
• If using a custom logo, with a recommended size of 280×60 pixels or about 10kb, make sure that the image used (from Custom Themes) is uploaded and not a URL link. Also, the image appears differently in OWA vs. Outlook (much larger).

End User Experience

Once settings are configured in the Microsoft Defender portal, Exchange Online publishes the changes. After a short delay, the updated messages are available to users. Afterward, users see the customized text if they report a message, when an email is analyzed, and when an admin reviews the submission. Figure 5 shows an example of the custom message sent when an end user reports a message:

If an email is reported as Junk, it will be moved to their mailbox’s Junk Mail folder. If an email is reported as Not Junk, then it will be removed from the Junk Mail folder and placed back in the Inbox. For phishing emails, the email is removed from the users’ mailbox completely. These actions are performed by Exchange Online Protection (EOP). Additionally, once an email has been reviewed by an administrator, the end user receives information about the review. If the email is a Phishing email, a red bar with the word ‘Phish’ is present in the email (Figure 4), an email with no threats has a green bar with the words ‘No threats found,’ and finally, an email classified as junk has an orange-colored bar with the word ‘Junk.’

Start Configuring Today

User Submissions are not a splashy feature in Microsoft 365, but it is something worthwhile to configure as it leads to a better end-user experience and assists Microsoft in providing email defense with Exchange Online Protection by updating their filters against future malicious email campaigns. For Microsoft 365 administrators, the takeaways are that it’s possible to apply branding and custom messaging for User Submissions. IT Support can provide useful, succinct information to users and provide confidence that actions are being taken. By enabling User Submissions, administrators allow users to participate in the defense process for their organization and raise awareness around bad emails that arrive in user inboxes.