Using Adaptive Scopes with Microsoft 365 Retention Policies for Users and Groups

Moving from Static to Dynamic Locations

Microsoft 365 retention policies allow organizations to control how information is retained and removed for multiple workloads. Until recently, retention policies have been static. In other words, once you create a policy, the target locations for the policy remain unaltered until an administrator updates the policy to add or remove locations (mailbox, sites, and groups). For example, a new executive joins the company, and an administrator amends the retention policy applying to executives to add their mailbox and OneDrive for Business account.

Flagged for availability in August 2021, it’s taken Microsoft a little longer to make adaptive scopes for retention policies available in public preview. Now that the roadblocks are removed, tenants with Office 365 E5 licenses, or Microsoft 365 E5 compliance licenses, should see adaptive scopes available in the Information governance section of the Microsoft 365 compliance center. General availability will follow in due course, probably in early 2022.

What’s an Adaptive Scope?

An adaptive scope is a filter used to identify a subset of target locations. This isn’t breakthrough technology because Microsoft 365 already uses filters in features like Exchange Online dynamic distribution lists or dynamic Microsoft 365 groups (and teams). What’s different is that you can associate one or more adaptive scopes with a retention policy. When this happens, Microsoft 365 resolves the filters contained in adaptive scopes against Azure AD to identify the set of target locations for the policy and, critically, does so on an ongoing basis to include or exclude locations which meet or do not match the scope conditions. In other words, adaptive scopes are automatic location management for retention policies. They’re well suited for dealing with applying retention policies in scenarios like:

• All Azure AD accounts belonging to a specific department or located in a certain country.
• All SharePoint Online sites of a particular type.
• All Microsoft 365 groups holding product information.

In fact, anywhere you can identify a user, group, or site with a custom property, you can use an adaptive scope to find and apply a retention policy to that location.

In this article we’ll discuss how to use adaptive scopes with retention policies for Users and Groups. Adaptive scopes for retention policies targeted at SharePoint Online and OneDrive for Business use the same fundamental approach but have their own quirks, so I cover them in a separate article.

Adaptive Scope Requirements

Before we go into the details of how adaptive scopes work, two important points need to be understood. First, you can’t update an existing retention policy which uses static locations to use an adaptive scope. Instead, you create a new retention policy with an adaptive scope which finds the same set of locations and use it to replace the old static policy. To ensure a smooth switchover, the old policy should remain in place until the new policy is active for all target locations.

The second point is that although static retention policies are covered by Office 365 E3, adaptive scopes require Office 365 E5 or Microsoft 365 E5 licenses. Or rather, any account which benefits from an adaptive scope requires an E5 license. This is to be expected because the licensing follows Microsoft’s normal approach whereby any automatic management within Microsoft 365 requires E5 licenses.

Scope Types

Adaptive scopes are managed through the Information governance section of the Microsoft 365 compliance center (or Records management if you’re already using that solution) and come in three types:

• Users: Applies to mailboxes, OneDrive for Business accounts, and the compliance records captured in Exchange Online for Teams chats conversations and Yammer user messages (when Yammer networks are in Microsoft 365 native mode). This type of scope doesn’t filter for Teams and Yammer messages explicitly. They are processed along with other user mailbox content.
• Sites: SharePoint Online sites and OneDrive for Business accounts (when identified by URL, name, or a refinable string).
• Microsoft 365 Groups: This type of scope covers the Teams channel conversations and Yammer community messages captured as compliance records in group mailboxes.

Although you can’t mix location types (like sites and mailboxes) within an adaptive scope, you can include multiple adaptive scopes within a single retention policy to ensure that the same retention settings apply across multiple workloads. You can also include multiple adaptive scopes of the same type (for instance, three adaptive scopes for users) in a single retention policy.

Obviously, the mixing and matching of adaptive scopes within retention policies allows great flexibility. And because Microsoft 365 evaluates the filters against Azure AD on an ongoing basis, the limits which exist for locations included in non-org-wide static retention policies (like 100 SharePoint Online sites or 1,000 mailboxes) don’t exist. An adaptive scope could cover 500,000 users or 100,000 sites.

Creating a User Adaptive Scope

Because it works somewhat a dynamic Azure AD group, the user adaptive scope type is the easiest to create and understand. In the Information governance section of the Microsoft 365 compliance center, choose Adaptive scopes (preview) and create a new scope. Give the scope a name and description, and then move to define the scope type. At this point, you choose one from Users, Sites, and Microsoft 365 Groups. Choose Users.

The compliance center opens a query builder where you can define the properties and values used in the filter (Figure 1).

The query builder is straightforward (simple) and doesn’t need much description. The query builder supports four operators:

• is equal to.
• is not equal to.
• starts with.
• not starts with.

Remember that the query is against Azure AD, so not all mailbox properties can be included in a query. You can use the properties shown in Figure 1 (the fifteen single-value custom attributes are usable; the five multi-value custom attributes are not). If you want to check against multiple values for the same property, use multiple or tests (Figure 1 features a check for several variations of job title).

The advanced query builder is no more than a field to input a query in OPATH syntax. For example, to find users in France who belong to the IT department and job title is “Architect” and have “IC” in custom attribute 4, the query is:

Department -eq “IT” -and CountryOrRegion -eq “France” -and Title -eq “Architect” -and CustomAttribute4 -eq “IC”

Advanced queries can use a wider set of operators (such as like and notlike string comparisons). For instance, this query finds users in any department beginning with Sales whose job title contains Manager:

Department -like 'Sales*' -and Title -like '*Manager*'

User queries users can also find mailboxes to process based on the mailbox type or state. An example of a mailbox type query is to find shared or resource mailboxes. This query creates an adaptive scope to process shared mailboxes based on the IsShared flag used to mark these mailboxes. Remember that these mailboxes will need Office 365 E5 licenses if processed by retention policies based on an adaptive scope.

IsShared -eq 'True'

Inactive mailboxes are an example of a mailbox state. Mailboxes reach this state if they are within the scope of a retention hold or include items with retention labels when deleted. The query to find inactive mailboxes for an adaptive scope uses the IsInactiveMailbox flag.

IsInActiveMailbox -eq 'True'

Although advanced queries are more flexible, outside mailbox states and types, the Azure AD account properties used in queries remain limited to the set presented in the query builder. As an example, I tested with an OPATH query used in a dynamic group to find accounts licensed with Office 365 E5 (below). Azure AD can resolve the query to calculate group membership. But it doesn’t work for an adaptive scope:

user.assignedPlans -any (assignedPlan.servicePlanId -eq "2f442157-a11c-46b9-ae5b-6e39ff4e5849" -and assignedPlan.capabilityStatus -eq "Enabled")

Validating Scope Results

Microsoft already has a bunch of processes used for retention processing. According to sources, a new process resolved the queries for adaptive scopes to determine what locations the scope will find. The process runs daily but might need several runs to find every possible location. the query builder doesn’t resolve queries or have any way to show the expected results, so there’s no immediate way to prove that a scope will return the expected set of accounts, sites, or groups. Instead, you must wait before the background job processes the scope and makes its results available. According to Microsoft, this could take up to five days (it’s usually faster for users and groups), which makes the deployment of adaptive scopes quite frustrating at times. You don’t have to wait to see the set of calculated locations for a scope as you can use a scope immediately after creation in a retention policy. However, it’s obviously better to have confidence that a scope finds the right accounts before using it in a retention policy.

To see the set of target locations calculated for a scope, select the scope, and then Scope details in the fly-out pane (Figure 2). If you see “No data available,” it either means that the query didn’t find anything, or a background process has not resolved the query yet.

It’s also true that you might see incomplete results for a query. I have checked scope results after a day or so and found a couple of mailboxes and checked again after another day to find the full set I expected. Microsoft 365 is a very distributed environment spread across hundreds of thousands of servers, so it’s understandable that processing can take some time. Nevertheless, given that it’s possible for Azure AD to calculate dynamic group membership much quicker, it’s puzzling why adaptive scope results are so slow.

Once a set of scope results appear, you can compare the set of locations returned for a user adaptive scope by running the Get-Recipient cmdlet with the same query. For instance, the equivalent Get-Recipient command for the OPATH query:

CustomAttribute4 -eq “IC” -or Title -eq “Consultant” -or Title -eq “Architect” -or Title -eq “Senior Consultant”

is:

Get-Recipient -RecipientTypeDetails UserMailbox -Filter {CustomAttribute4 -eq "ic" -or (Title -eq "Architect" -or Title -eq "Consultant" -or Title -eq "Senior Consultant")} | ft displayname, customattribute4, title

DisplayName                 CustomAttribute4 Title
-----------                 ---------------- -----
Marc Vilas                  IC               Architect
Brian Weakliam (Operations) IC               Architect
Eoin Redmond (Ireland)                       Architect
Oisin Johnston              IC               Consultant
Ben James                                    Architect

If both Get-Recipient and the scope details agree (as they do based on the list in Figure 2), you know that the scope is working as expected.

Creating a Group Adaptive Scope

The process to create a group adaptive scope is like creating a group adaptive scope with the only difference being that a smaller set of properties is available to build the query. Groups don’t generally have assigned departments, cities, countries, and so on. In effect, although you can filter by display name or description, the custom properties will probably be the basis for most filters used in group adaptive scopes. Figure 3 shows the set of properties available.

Although you can’t see the filter criteria in Figure 3, I chose to look for groups with “France” set in CustomAttribute15. The equivalent Get-Recipient command is:

Get-Recipient -RecipientTypeDetails GroupMailbox -Filter {CustomAttribute15 -eq "France"} | Format-Table DIsplayName

DisplayName
-----------
Project Aloha (Sophia-Antipolis)
Nice Cote d'Azur
Tango (Caen)
French Food Lovers

Once again, we can check the groups the adaptive scope finds by using scope details.

Using Adaptive Scopes in Retention Policies

Using adaptive scopes to select target locations for retention policies is simple. Remember, you can’t upgrade a retention policy from static to adaptive, so you must create new policies to use adaptive scopes.

When you create a new retention policy, you can choose static or adaptive. The next step is to select at least one adaptive scope to use. You can combine multiple scopes in one retention policy, such as in the example where you want the same retention policy to apply to employees in multiple European countries.

In Figure 5, the retention policy has two adaptive scopes. When you select adaptive scopes, Microsoft 365 calculates what the valid types of location are. In this case, both are user adaptive scopes, so the target locations are those valid for this scope type.

There’s no difference in selecting retention settings (for example, delete all items after five years). All that’s changed is the dynamic nature of location targeting for the retention policy. When you save the new policy, Microsoft 365 publishes its details to the workloads, which take charge of enforcing the retention settings.

Scoping for the Enterprise

Adaptive scopes solve a problem for organizations where locations subject to retention change often. Enterprises with large and distributed employee populations will probably benefit most, and as those are the organizations most likely to have E5 licenses, they’ll be able to enjoy this new benefit with extra cost. For smaller organizations or those still unconvinced that the features included in E5 are worth the money, adaptive scopes are unlikely to be the point that makes them upgrade because other ways (manual or using PowerShell) exist to update retention policies.

I’d like to see more flexibility in the query builder and applying queries to generate adaptive scope results is too slow (as is the GUI in general). Admittingly, this is a V1.0 release, and hopefully Microsoft will smoothen the way adaptive scopes work over time.