Find Inactive Copilot Users to Remove and Reassign Expensive Licenses
Last January, I wrote about how to find user accounts with underused Microsoft 365 Copilot licenses (or unused Copilot licenses) with the intention of removing those licenses and reassigning them to more deserving users. The script used to analyze Copilot usage and reassign licenses works, but it suffers from two issues.
The first is that the computation to decide when a Copilot license is underused is a blunt instrument. Given that the intention of the script is to illustrate a point rather than to be a full solution, that aspect of its calculations is understandable. It’s easy for an organization to come up with their own algorithm and integrate it into the script.
The second issue is that the usage data used by the script comes from the Microsoft 365 usage reports data store. Incomplete input data invariably leads to bad outcomes, and the usage data is very sparse. It is designed to be the basis for Copilot usage reports in the Microsoft 365 admin rather than to deliver deep analysis. The data for a user simply tells us when they were last active with Copilot in several different apps, such as Outlook, Teams, and OneDrive. No data is available to inform whether someone heavily uses Copilot in any of the apps, and some apps (like Loop) are missing. However, the data suffices to generate a usage report in the Microsoft 365 admin center (Figure 1).
Collectively, the two issues mean that although the script can propose a set of inactive users for license reassignment, the basis of the decision is likely to result in bad choices. For instance, a user who gets incredible value from using Copilot in a single application like Word or Excel and doesn’t use Copilot anywhere else is probably going to be tagged as inactive.
Solving the Data Problem to Find Unused Copilot Licenses
The solution is to use better data to analyze the activity levels of Copilot users. Better data means that more developed computations can be used to detect inactivity and lead to better outcomes. Let’s see what steps can be taken to improve matters.
The fundamental issue is the source of better data. The allInteractionHistory Graph API is certainly a possibility, but the API is still quite raw and slow. The compliance records created to track Copilot interactions for individual users certainly represent a rich source of data, but the data is stored in user mailboxes. Accessing each mailbox to extract and analyze compliance records won’t be a fast process. Fortunately, a third option exists, and that’s the audit records captured in the unified audit log for Copilot interactions. The audit log is a single source of data that’s reasonably easy to access with PowerShell using the Search-UnifiedAuditLog cmdlet from the Exchange Online management module or the Graph AuditLogQuery API.
Using the Graph
I decided to go down the Graph route for two reasons. First, it minimizes the number of product modules that are involved. This might sound like a trite reason, but the Exchange Online management module and the Microsoft Graph PowerShell SDK have had issues loading assemblies and components used by the two products. Resolving the incompatibilities between the different PowerShell modules used for Microsoft 365 automation is a task that Microsoft needs to take on. For now, it seems like the engineering groups that manage the different modules take their own path without worrying about clashes with other modules.
Using the Search-UnifiedAuditLog cmdlet is quicker, but it also comes with a limitation of 50,000 records that can be fetched in a single session. My second reason for using the Graph API is that, although slower because searches run asynchronously as background jobs, the audit logs can process higher numbers of records. That could be an important point in large tenants where thousands of Copilot licenses are in active (or inactive) use.
Besides, I also have some code that can be repurposed from the article about analyzing Teams usage to detect inactive teams, and I always like when I can reuse some PowerShell.
Scripting the Solution to Find Unused Copilot Licenses
I started off with the script created for the previous article. It would be nice to say that all I needed to do was to paste the code from the script to find inactive teams, but such an easy solution is seldom found. You can download the complete script from GitHub to see the changes made to create the final version.
On a permissions level, the script (app or user) needs to have consent for the AuditLog.Read.All permission to run audit jobs and fetch audit records. If running with delegated permissions, the signed-in user must hold a compliance role that allows access to audit logs.
The script starts like the previous version by finding the set of users with Microsoft 365 Copilot licenses (SKU identifier 639dec6b-bb19-468b-871c-c5c441c4b0cb). It then fetches the Copilot usage data, making sure that real usernames are in the data rather than obfuscated values. A usage record looks like this:
Name Value
---- -----
excelCopilotLastActivityDate 2025-02-28
lastActivityDate 2025-03-02
displayName Tony Redmond
powerPointCopilotLastActivity… 2025-02-28
outlookCopilotLastActivityDate 2025-01-17
copilotActivityUserDetailsByP… {System.Collections.Hashtable}
oneNoteCopilotLastActivityDate
reportRefreshDate 2025-03-02
userPrincipalName Kim.Akers@office365itpros.com
copilotChatLastActivityDate 2025-03-02
wordCopilotLastActivityDate 2025-03-01
loopCopilotLastActivityDate 2025-01-29
microsoftTeamsCopilotLastActi… 2025-01-01
Next, the script submits an audit job to find Copilot interactions across the tenant for the 30 days prior to the usage report refresh date to ensure that the audit and usage data cover the same period. The script uses Graph requests instead of Graph SDK cmdlets in an attempt to work around an authentication issue with the cmdlets (which might be fixed when you read this text). Unhappily, the same issue affects requests against the Graph API.
After submitting the audit job, the script enters a loop to wait for the job to complete. Typically, an audit job takes about 10 minutes, but it could take longer if there is high demand on the service or many records to fetch. Because of its asynchronous nature, anything to do with audit records is a good candidate to run in the background with Azure Automation.
Once the audit job completes, the script loads the audit events found by the job into an array and processes the items in the array to extract details of the apps used for Copilot interactions. This data is then processed to generate a list of user interactions with Copilot, which looks like this:
User : Kim.Akers@office365itpros.com UserId : aff4cd58-1bb8-4899-94de-795f656b4a18 ChatInteractions : 29 ExcelInteractions : 8 LoopInteractions : 5 OutlookInteractions : 190 PowerPointInteractions : 3 WordInteractions : 287 OtherInteractions : 5 TotalInteractions : 527
Putting the usage data that was previously used for the assessment together with details of how individual people use Copilot creates a much more comprehensive view of how people use AI assistance. Deciding how to assess inactivity is up to the individual organization. For the purpose of this exercise, the script has a function to calculate a user score based on the formula:
Total count of days since using a Copilot App (Score) divided by the number of Copilot apps used less the total interactions divided by 10.
function Get-UserScore {
param (
[int]$Score,
[int]$ScoreApps,
[int]$TotalInteractions
)
if ($ScoreApps -gt 0) {
[double]$UserScore = (($Score / $ScoreApps) - ($TotalInteractions / 10))
} else {
[double]$UserScore = 0
}
return $UserScore
}
Usage data is available for seven Copilot apps. The account used six apps in the last 30 days and the total number of days since using all the days is 109 (for instance, the last used date for Copilot in Outlook is 17 January 2025, the date of the usage data is 2 March 2025, so the score for Outlook is 44; adding the score for each app together gets the total number of days). The user’s total interactions are 527, mostly in Word. The user score is:
(109/6) – (527/10) = -34.33
The script sets an activity score threshold of 30 to determine unused licenses. Changing the threshold is a matter of updating a variable. A negative score is good because it means that the user is active across many of the Copilot apps during the reporting period.
Don’t get hung up on the formula. It’s there to illustrate a concept rather than making any claim to be the definitive method to measure the use of Copilot. You can adjust the parameters as you like or construct a completely different approach.
After checking all the accounts with Copilot licenses against the activity score threshold, the script outputs a list of those considered not to have used the license at all or only sparingly.
The folllowing 19 users are underusing their assigned Microsoft 365 Copilot license User Number active apps Overall Score ---- ------------------ ------------- Alain Charnier 0 0.00 Paul Robichaux (Office 365 for IT Pros) 0 0.00 Michelle Dubois 0 0.00 Kim Akers (She/Her) 0 0.00 John C. Adams 0 0.00 Joanne Crispa 0 0.00 Jamie Smith 0 0.00 René Artois 0 0.00 James Ryan 0 0.00 Eric Hammond 0 0.00 Eoin Redmond (France) 0 0.00 Brian Weakliam (Operations) 0 0.00 Ben Owens (DCPG) 0 0.00 Ben James (BusDev) 0 0.00 Andy Ruth (Project Director) 0 0.00 James Abrahams 0 0.00 Hans Geering (Project Management) 2 54.00 Lotte Vetler (Paris) 2 62.50 Sean Landy 1 73.00
You can then allow the script to go ahead and remove the Copilot licenses from user accounts (the code handles both direct-assigned and those assigned by group-based licensing). The only thing left to do is to find good homes for the newly-liberated licenses.
Make Better Decisions
One of the strengths of PowerShell is that it makes it easy to mix and match information drawn from different sources to make better decisions. In this case, combining audit and usage data for Microsoft 365 Copilot activity allows license administrators to understand who’s really using the expensive $360/user/year licenses.
The Link to url for GitHub isnt working correctly
It works for me…