How to Report High-Priority App Permissions That Hackers Might Exploit

Detecting Malicious Apps that Steal Graph Permissions

I recently read an article by Lina Lau discussing How to detect Token Theft in Azure, explaining how attackers use apps to steal access tokens to compromise Entra ID user accounts. The article discusses how phishing emails containing a link requiring the recipient to grant access to a malicious application through the OAuth authorization flow ends up with the malicious app gaining permissions to gather information from the tenant (what Microsoft refers to as illicit consent grants). The bad app can be registered in your tenant or enabled for use in any Microsoft 365 tenant.

Because threats like this exist, it’s important that Microsoft 365 tenants check the set of active apps in the tenant and the permissions assigned to each app. Ideally, an audit of apps and permissions should happen monthly with the intention of removing apps that are no longer required or removing permissions that apps no longer need. You can use Microsoft Defender for Cloud Apps to review apps and permissions, or you can create your own tool for this purpose. The important thing is to review apps regularly.

Azure Automation Helps

The problem is that we all have too much work to do, which is why a scheduled PowerShell script executed regularly by Azure Automation can help. I’ve covered the topic of using Azure Automation in several articles recently, and it seems like this would be a good scenario to explore with managed identities and the Microsoft Graph PowerShell SDK.

Of course, we need to write a script to do the work, but the task isn’t that difficult. A malicious app added to the tenant has an Entra ID identity (the application object) and a service principal, used to hold the set of permissions assigned to the app.

If an app is designed to work across all tenants, the same set of permissions applies to each tenant. ISVs use this capability to allow their code to run with minimal need for configuration in tenants that install ISV solutions. For example, if you use the OAuth version of the Thunderbird email client for POP3 and IMAP4 access to Exchange Online, Thunderbird registers its app in Entra ID. The service principal for the app has a SignInAudience property of AzureADMultipleOrgs, meaning that the same app definition works for all tenants (the setting for apps registered for a single tenant is AzureADMyOrg).

Script Steps

Our script must do the following:

• Search Entra ID to find the set of service principals belonging to registered apps. For this exercise, I included the service principals used by managed identities. I don’t think that attackers have created and exploited managed identities, but given that these identities can be used with Azure Automation to perform highly-permissioned tasks, it seems worthwhile to include them in the mix.
• For each service principal, find the app role assignments (permissions). For the Graph APIs, these are application permissions rather than delegate permissions. Application permissions allow apps to run as background services without a signed-in user being present. It’s possible that attackers might attempt to exploit delegate permissions. If so, the script written by Vasil Michev described in this article includes delegate permissions in its output.
• For each app role assignment, resolve its identifier (a GUID) to a human-friendly form. For instance, translate b633e1c5-b582-4048-a93e-9f11b44c7e96 to “Mail.Send.” You could leave this step out if you’re fluent in all the GUIDs used for Graph permissions, Exchange Online permissions, Azure API permissions, and so on. Most humans are not.
• Output the details found.

At the end of this process, we have data for all the permissions assigned to service principals for the targeted apps. This is interesting information, if only to discover what permissions are consumed by apps. A simple command like the one below to group the discovered permissions reveals how many times the script finds individual app permissions:

$Report | Group Permission | Sort-Object Name | Format-Table Name, Count

In my tenant, the Group.Read.All permission is the most used. This isn’t a surprise because the permission is needed by apps like the Groups and Teams Activity Report to fetch a list of Microsoft 365 groups or Teams.

Finding the Important Permissions Held by Entra ID Apps

Not every permission is dangerous, and some won’t be of much use to attackers. We need to focus on permissions that attackers can use to expropriate or interfere with data. To filter out the apps that deserve administrator review, we can define an array of high-priority permissions and check the app data against the array. For instance, here’s a set of permissions that I think attackers could use to perform actions like fetching information about users in the directory, erasing messages from mailboxes, and sending emails.

# Define the set of high-priority permissions we're interested in
[array]$HighPriorityPermissions = "User.Read.All", "User.ReadWrite.All", "Mail.ReadWrite", "Files.ReadWrite.All", "Calendars.ReadWrite", "Mail.Send", "User.Export.All", "Directory.Read.All", "Exchange.ManageAsApp", "Directory.ReadWrite.All", "Sites.ReadWrite.All"

It’s easy to add extra permissions to the list. For instance, some would argue that Groups.Read.All should be included because if an attacker can use an app with this permission, they might be able to find a group containing administrator names.

To filter the set of apps for review, we loop through the list of apps and permissions to find those that match the high-priority list:

ForEach ($R in $Report) {
  If ($R.Permission -in $HighPriorityPermissions) {
     $ProblemApps.Add($R) }
}

The data for an app looks like this:

DisplayName        : PS-Graph
ServicePrincipalId : 666d4d19-9391-4f13-9ed8-0de304dd286a
Publisher          : Office 365 for IT Pros
Homepage           :
OwnerOrgId         : a662313f-14fc-43a2-9a7a-d2e27f4f3478
OwnerOrgName       : Office 365 for IT Pros
AppRoleId          : 741f803b-c850-494e-b5df-cde7c675a1ca
AppRoleCreation    : 23/08/2019 19:46:50
Id                 : GU1tZpGTE0-e2A3jBN0oao-KiJnCMgdJhkZeg9xZI6c
Resource           : Microsoft Graph
ResourceId         : 14a3c489-ed6c-4005-96d1-be9c5770f7a3
Permission         : User.ReadWrite.All
SPType             : Application
CreatedDate        : 22/08/2016 00:05
RecentApp          : False

Figure 1 shows the set of apps in my tenant for administrator review as viewed through the Out-GridView cmdlet:

Sending the Apps Report to Teams

To bring the report to the attention of administrators, the script posts it to a Teams channel. It could equally be sent to a distribution list via email. Figure 2 shows a sample message posted to Teams. The message lists apps in creation date order (newest apps first) on the basis that if a permissions problem exists, it’s more likely to be in a recently created app.

To make sure that the information about app permissions is brought to the attention of administrators regularly, we can schedule the script to run in Azure Automation. This eliminates the need for administrators to remember to run the script manually and increases the chance that the content of the report will receive some attention. Once weekly might be too often to review app permissions, but it’s a good idea to review them monthly.

Implementation Details

I chose to use Azure Automation with a managed identity to schedule the runbook (script) in my tenant. Scheduling the runbook makes sure that the job is run weekly. If the administrator recipients do their job and check the email delivered to their inboxes, a fair chance exists of catching a bad consent.

The script that I used is available from GitHub. Make sure that you adjust the variables in the script to reflect your environment. For instance, I use Azure Key Vault to store information about the account used to post to Teams, including identifiers for the target team and channel. These values will be different in your tenant. In addition, you might also want to adjust the set of permissions the script scans for to add or remove permissions based on your view of their relative sensitivity.

The Administrative Challenge of Entra ID Apps

A few years ago, the number of app registrations in an average Microsoft 365 tenant could be counted on using the fingers of one hand. Today, the spread of OAuth 2.0 authorization for apps, the growing influence of the Graph APIs, and the availability of the Microsoft Graph PowerShell SDK (beware of over-permissioned interactive access) all contribute to a swelling number of apps found in tenants.