Attack Simulation Training: RBAC and End User Notifications

Helping End Users Recognize Dangerous Email

Attack Simulations are Microsoft’s foray into a crowded field of competitors who provide a service that trains users to recognize dangerous email with simulated Phishing or malware-infested messages. Microsoft has continually added features and functionality since they released Attack Simulations including additional simulation types, different payloads, custom payloads, customizable training and more. The most recent upgrades are Role Based Access Control (RBAC) permissions and end user notifications.

Role Based Access Control

A welcomed addition to any Microsoft 365 feature, there are two roles initially available in Azure AD: Attack Simulation Administrator and Attack Payload Author. The roles allow designated users to manage simulations while others concentrate on creating attack payloads, but what is missing are roles to handle the training setup and end user notifications as administrators don’t usually handle these tasks in medium to large organizations. You can assign the roles through the Roles and administrators’ section of the Azure AD admin center. By default, these roles are not assigned to the Global Administrator role.

The RBAC roles are also manageable through PowerShell (the Exchange Online management module). After connecting to Exchange Online, connect to the compliance endpoint:

Connect-IPPSSession

You’ll then be able to see details of the two management roles for Attack Simulation by running the following:

Get-ManagementRole | where {$_.Name -like '*attack*'}
 
Name                            RoleType
----                            --------
Attack Simulator Admin          AttackSimAdmin
Attack Simulator Payload Author AttackSimPayloadAuthor

Although you can perform actions in PowerShell like seeing who is assigned these roles, you can’t perform any attack simulation features in PowerShell such as creating a new notification message.

Read More: How to Report on Exchange RBAC assignments.

Configuring End User Notifications

Attack Simulation uses messages called End User Notifications to communicate with the end users who use attack simulations. The notifications include praise for a job well down when end users recognize and report a problem message, and reminders for needed training. 

Currently, only praise notifications are available. Templates for both types of notifications are available through the End User Notifications tab in the Attack simulation training section of the Microsoft 365 Defender admin center. Two choices exist: Global Notifications and Tenant Notifications. Global Notifications tab shows Notifications created by Microsoft which are not editable.  The Tenant Notifications tab is where an Administrator creates custom notifications.  To create a custom notification based on a Microsoft notification, highlight the notification and click ‘Create a Copy’ (Figure 1):

Administrators manage custom notifications through the Tenant Notifications tab, and there are two types of Notifications:

• Positive Reinforcement Notification: When an end user reports a message, they receive a positive reinforcement email.
• Simulation Notification: Communications sent when a user is enrolled in Attack Simulation Training and has been phished.

Positive Reinforcement Notification

First, we will create a Positive Reinforcement email. When creating employee communications, it’s important to create a message with appropriate and insightful text (the HR or marketing department might be able to help frame suitable text, if needed.)

Begin by creating a new notification under Tenant Notifications. Select the type of notification, and then enter a name and description (Figure 2):

Next, we’ll need to insert the content of the email to be delivered (Figure 3):

Creating the content for a custom notification requires some forethought and planning, and there are also required details you’ll need to provide that appear when a user reports a malicious/suspicious email:

• Display Name: Provide a name for the sender. Ideally, the sender should be familiar to end users.
• From Email Address: This email address will be visible to the end user. If you want to allow replies, you need to insert a real email address. Otherwise, an email address such as noreply@practicalpowershell.com or noreply@<your email domain> is appropriate.
• Language: Chose one of twelve available languages: Chinese (traditional), Chinese Simplified, English, French, German, Italian, Japanese, Korean, Portuguese, Russian, Spanish, and Dutch.
• Subject: While this field is prepopulated with ‘Thank you for reporting a phish!’, it can be customized to an approved subject.
• Email Body: The greatest customization options are available here, including options to use Text, Code, Dynamic Tags as well as including content from Microsoft’s Default Templates for the Positive Reinforcement notification. Figure 4 shows a sample of a customized notification:

Although the tenant-specific notification was created from a Microsoft Template, it can be customized further to meet an organization’s communication needs.  After adding the content you’d like to include in the notification, Click Save, Next and Submit to complete the creation process:

Simulation Notifications

Simulation notification emails are sent to end users as reminders of training assigned because they clicked on a link or otherwise failed for the simulated attack. Microsoft includes an email to handle this type of communication, and like the Positive Reinforcement email we can create our own version.  The creation process is exactly like the Positive Reinforcement Notification with the obvious difference that the text imported from the Microsoft template reflects the need to inform the recipient about the dangers of malware and phishing.

After creating custom Tenant Notifications, we can add the notifications to a new Simulation Training.  Currently, this is only possible for the Positive Reinforcement Message.

On the Notifications configuration page, three options exist for how to handle these notifications:

• Do not deliver notifications: No notifications will be sent to end users.
• Microsoft default notification: Microsoft created notifications will be sent to end users for both success and failure of a simulation.
• Customized end user notifications: If custom notifications were created, these can be sent to end users if this is a selected option.

If we choose to not send Notifications, Microsoft pops up a message recommending not to choose that option. Obviously, if you run a simulation, you should let people know about the results and choosing to deliver notifications also provides the opportunity to deliver a Positive Reinforcement message to end users.  For that, there are three options available:

• Do Not Deliver
• Deliver After Campaign Ends
• Deliver During Campaign

Next to the Delivery option is the ‘Action’ column. When clicked this generates a preview of the communication to the end user (Figure 7):

Caveats

• During the creation of a new Attack Simulation, we can only select Positive Reinforcement Notifications and not the Simulation Notification emails.
• Positive Reinforcement emails are not delivered immediately when a user reports an email.

End User Experience of Attack Simulations

When a campaign starts, an end user has several options for responding to the threat: ignore the email, delete the email, report the email, or click on the suspicious content.  If the user is trained well, then option three will trigger a Positive Reinforcement email if the Simulation is configured to do so, and communication may look something like Figure 8:

If the user does not report the message and instead clicks on suspicious content, first they’ll receive an email notifying them of the additional training required to help them understand the threats that might lurk in inbound email (Figure 9):

In addition to the above training notification, the user should receive a Simulation notice. However, this is not an option yet and will be a future feature.

A Good Start

Attack Simulation Training follows the pattern of most Microsoft cloud features: the product appears with a good starting set of features and the product develops over time to add capabilities.  Microsoft has now added RBAC and End User Notifications to its Attack Simulation Training, both of which add value. However, like many offerings in this space, Microsoft can do better.

Here are some future enhancements I would like to see:

• More RBAC roles such as End User Notifications and Reporting.
• Simulation emails do not pass through the Exchange transport pipeline, which means that transport rules don’t process the messages. Rules often apply branding or warnings to messages, which would make the simulation more realistic in terms of message appearance.
• Simulation Notifications appear to be missing from the Simulation configuration itself.
• Better support for Graph API and PowerShell to manage and configure aspects of attack simulation.

At the end of the day, these two additions to Attack Simulation Training are a great incentive to deploy and adopt this functionality, as End User communications are the key enhancement that make this feature worthwhile for an organization.