A PowerShell script to Remove Debris from User and Shared Mailboxes
From time to time, the need exists to remove items from user mailboxes. Perhaps someone has shared information in an email that they shouldn’t have, or maybe some pernicious malware-ridden message has penetrated Exchange Online Protection and reached inboxes. You could ask users to remove the offending messages manually, but it’s usually more efficient and effective if administrators step in and remove the problem.
In the on-premises world, the Search-Mailbox cmdlet does the job. Microsoft’s enthusiasm to embrace compliance features that work across multiple workloads sometimes has unfortunate consequences, and their decision to deprecate Search-Mailbox in July 2020 removed a valuable tool. At the time of writing, Search-Mailbox still works, but many are put off using it because of its unsupported nature.
Microsoft’s recommended replacement for Search-Mailbox is to use purge actions with compliance searches to remove items from Exchange Online mailboxes. This method uses a compliance search to find matching items followed by the execution of a purge action to remove the items. It works, but processing is slow and seems to be getting slower as time goes by. Last week, I spent some time working on the example script I wrote in 2019 to show how to loop through compliance search results to purge items, and the performance was dreadful. Investigating a script to do the job without using either Search-Mailbox or compliance searches seemed like a good thing, if only to learn how to solve the problem using Graph API requests.
Charting Out the Script
The basic outline of the script is:
- Collect the parameters for the search query.
- Find mailboxes to search.
- Run the search query against the mailboxes. Because we’re using Graph API requests, we need a registered Azure AD app with consent to use the Mail.ReadWrite application permission. Read access is necessary to retrieve message data. Write access is needed to delete messages from mailboxes.
- Apply the action to the messages found by the query. I wanted options to report the messages and delete the messages.
Now that we know what steps must be taken, let’s examine some of the detail.
First Attempt
The original version of the script was very simple. I used the Graph message resource type to retrieve messages from a mailbox with a filter to find the desired messages. Applying a filter finds messages with an exact match. For instance, this request finds all messages with a subject of “Project Ankara To Do Items”:
https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/messages?$select=id,receivedDateTime,subject,from&$filter=subject eq 'Project Ankara To Do Items'
The select request succeeded but only returns a limited set of (because of the exact match), which are unlike those retrieved by Search-Mailbox or a compliance search.
Before getting any deeper, let me emphasize that the Graph Explorer is an essential tool when it comes to working out the details of Graph API requests. You can take a URI calculated by the script and input it into the Graph Explorer to see what happens. Once a request works in the Graph Explorer, you know it will work in the script.
Filter Versus Search
I then moved on to the Microsoft Search API, which is probably where I should have started. The Search API uses Keyword Query Language (KQL) queries to scan mailboxes to find matching messages based on their metadata (properties) or content (message body and attachments). Although clients differ in their implementation, this API is what Outlook clients use for searching.
The initial challenge is to compose a suitable KQL query. If you’re used to composing search criteria for compliance searches (or for Search-Mailbox) then you know that KQL supports a set of specific properties to search email items. The script accepts a bunch of parameters including the message subject, keywords, date range, and sender address that it processes to create the query.
# Prepare search filter
$SearchFilter = "subject:$MessageSubject"
If ($SenderAddress) {
$SearchFilter = $SearchFilter + " AND from:$SenderAddress" }
If ($StartDate) {
$StartDateFilter = (Get-Date $StartDate).toString('yyyy-MM-dd') }
If (($StartDateFilter) -and (!($EndDate))) { # if we have a start date but no end date, set to today's date
$EndDate = Get-Date }
If ($EndDate) {
$EndDateFilter = (Get-Date $EndDate).toString('yyyy-MM-dd') }
If (($StartDateFilter) -and ($EndDateFilter)) {
$SearchFilter = $SearchFilter + " AND received>=$StartDateFilter AND received<=$EndDateFilter" }
If ($SearchQuery) {
$SearchFilter = $SearchFilter + " AND '" + $SearchQuery + "'" }
Here’s what an assembled KQL query looks like. This query looks for messages with “Update to Office 365 Book” anywhere in the message subject sent by Kim Akers between 1 January 2022 and 1 August 2022:
subject:Update to Office 365 Book AND from:Kim.Akers@Office365itpros.com AND received>=2022-01-01 AND received<=2022-08-01
Another example searches for the daily Viva Briefing emails
subject:Your daily briefing AND from:viva-noreply@microsoft.com AND received>=2021-01-01 AND received<=2022-08-02 AND 'Better with Microsoft Viva'
To perform the search, we compose a Graph API request like the following. You can see how the KQL query is passed and that we tell the Graph to return a set of properties for each message.
https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/mailfolders//messages?$search="subject:Update to Office 365 Book AND from:Kim.Akers@Office365itpros.com AND received>=2022-01-01 AND received<=2022-08-01"&$select=id,parentfolderid,receivedDateTime,subject,from
Searching Target Folders
This brings me to target folders. My script allows the user to search a single folder or all folders in the mailbox. If a search processes all folders, the results are like those generated by a search in Outlook desktop with just one issue. The search doesn’t scan items in the Deleted Items folder. This comes down to the implementation of search in different clients. Outlook desktop doesn’t search the Deleted Items folder unless you explicitly search that folder, but OWA does. Both clients use the Microsoft Search API feature of presenting the three most relevant items found by the search first, whereas I want a simple list. The Search-Mailbox cmdlet finds items in all user-visible folders plus the folders in the Recoverable Items structure.
The solution is simple: do one search for the Deleted Items folder and another for all other folders and combine the results. Maybe there’s a better way to search all user-visible folders, but I couldn’t find one and my workaround is effective.
At present, the Graph API can’t access archive mailboxes. If this functionality is needed, you’d need to write it using Exchange Web Services.
Folder Names
The information returned for a message found by a search looks like this:
@odata.etag : W/"FgAAAA==" id : AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNABGAAAAAAB_7ILpFNx8TrktaK8VYWerBwBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA3tTkMTDKYRI6zB9VW59QNAASRem3ZAAA= receivedDateTime : 2022-01-08T00:21:02Z subject : Office 365 Roadmap Updated: 2022-01-08 parentFolderId : AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNAAuAAAAAAB_7ILpFNx8TrktaK8VYWerAQBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA=
The folder where the message is located is stored in the parentFolderId. The value makes perfect sense to a computer but isn’t much good to a human. I needed a way to resolve these values to something that made sense, like Inbox. Deleted Items or Sent Items. To interpret the identifiers, I use the List mailFolders API to fetch the set of folders within a mailbox. The data returned includes the folder identifier and display name, so I could build a hash table keyed on the folder identifier to use as a lookup.
Processing Child Folders
Many mailboxes have a simple one-level folder structure, so it’s enough to fetch the list of folders from the mailbox root. Other mailboxes have nested folders, like those shown in Figure 1.
The information returned by the List mailFolders API includes a count of child folders, so it’s easy to know which folders include folders. To build out the hash table to include folder identifiers and display names of nested folders, the script iterates down four levels to find and process any child folders it can find. It’s strange that the List mailFolders API doesn’t have a qualifier to instruct the Graph to fetch all folders in a mailbox, but that’s the way things work.
Report or Delete
Hopefully, when the script runs, it finds some messages in the target mailboxes (Figure 2).
A parameter determines if the script then reports the messages or deletes (and reports) the messages. To delete a message, the script builds a URI containing the message identifier and then makes a Delete request to remove the message. Here’s what a URI and the API request look like:
https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/messages/AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNABGAAAAAAB_7ILpFNx8TrktaK8VYWerBwBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA3tTkMTDKYRI6zB9VW59QNAASRem3ZAAA= $Status = Invoke-RestMethod $Uri -Method 'Delete' -Headers $Headers
Deleted messages go into the Deletions sub-folder of Recoverable Items. Users can recover items from this folder. If you want them to be unable to recover items, you could move the messages to a hidden folder (like the Purges sub-folder in Recoverable Items) instead.
All Set for Deleting (After Testing)
You can download the latest version of the script from GitHub. Please test it thoroughly before you do any real damage. Don’t fall into the trap that so many have in the past by running a utility that removes emails without knowing what you’re doing. In particular, pay attention to the keywords used in the KQL queries as these can find items that you don’t expect.
I usually run the script in report mode to see what messages are found (Figure 3) before I let it delete anything. Although users can recover deleted items, it’s still better if administrators don’t remove items that they don’t need to. Feel free to live on the edge and delete messages without checking if that’s what you like to do.
If you find bugs, please file an issue in GitHub. Even better, fix the bug and let tell me what you’ve done. I appreciate any help you can give!
The Microsoft 365 Kill Chain and Attack Path Management
An effective cybersecurity strategy requires a clear and comprehensive understanding of how attacks unfold. Read this whitepaper to get the expert insight you need to defend your organization!
Hi, Tony,
It would be amazing if you could update the script (or give some guidance) with additional parameter “Archive”, so that the script could search through folders and subfolders of an in-place archive.
I have tried to achieve this but am rather unsuccessful – if it’s even possible? I find many posts mentioning that Graph API does not support access to in-place archive?
Thank you,
G
https://ivanderevianko.com/2020/02/get-messages-from-archived-mailbox-aka-in-place-archive-using-graph-api says that it’s possible for the Graph to access an archive mailbox, but the examples don’t work for me. The technique explored in 2020 might not work today. I know that part of the plan to deprecate EWS from Exchange Online is to expand coverage for archive mailboxes, so we shall probably have to wait.
Thank you so much, Tony – I also apologise for the delay in replying.
I have one more question, and I can’t figure out the solution. The script has some limitation I cannot go around. The report will only give me 520 items max, while the deletion will only do 260 max (at a time).
While the deletion is not a problem, as I can script it to run in a loop until it’s done, we have a client who wants to get all emails in a CSV that will be deleted. Any idea how I can achieve that or is this hard limit for Graph API?
Thanks,
G
I don’t think there is a hard limit on the number of messages that can be fetched from the Graph, but have never tried to get large numbers for deletion. I’m afraid I cannot be much help as I can’t see the data you’re working with… We write PowerShell to explain principles, not as complete solutions. It’s up to you to amend and adapt the code to fit your needs.
Hi Tony, I really love your script but Im using SkyKick and all PS Modules are pre-enabled, but the script still want to first conenct to Exchange Online Management.. the mandatory appid, tenant id and so on has been set to “false”, but still the script wont do the work – but it started and I can set up all the values “all folders, all mailboxes, start date, etc.” I also tried with parameter instead of the whole script but still dont work. I really appreciate some help on this here!
Thank you!
Have a great day.
I don’t understand what you mean by “using Skykick.”
The script connects to Exchange Online and then uses Graph API requests to interact with mailbox contents. A registered app is used to authenticate the Graph requests. Does such an app exist and does it have consent to use the required Mail.ReadWrite permission?
Hello Tony, thank you very much because your script looks like exactly what I was looking for… I’ll try! And I also think I will subscribe to “Office 365 for IT Pros”, it looks really cool! See you soon!
Cool. We always appreciate the support of people who subscribe to the Office 365 for IT Pros eBook. Our subscribers help us liberate time and energy to update the book monthly. https://gum.co/O365IT/
Hi Tony, everything is working and the script is great!
I modified the script to search (and remove) mails received from a given address with any subject; is it possible to do the same for a given domain or use a wildcard character for the sender’s name?
It’s PowerShell… so you can check the results of a search and decide what to do with each item. If you mean, can you search using a wildcard character? I don’t know… because I haven’t tested.
below is my search query. The search runs however its not finding anything. I am testing this out in my lab, and i have sent a couple of emails with “hellow” in the subject and “hellow world” in the body, but the search is not finding the data
-Folder all -SearchQuery “hellow world” -MessageSubject “hello” -DeleteItems n -Mailbox srathore@ptctest.com -StartDate 1-jan-2023 -EndDate 1-jan-2023 -AppId c431769c-ad7a-47ef-8b98-602ec0ab80a4 -TenantId 03f71422-2e3d-4237-8c6f-29088405379d -AppSecret 45345345345345345345 -SenderAddress srathore@ptctest.com
Can you find the messages with a content search using the same criteria? Also, I would make sure that your search end date is greater than the start date.
okay interesting. if i change the query to search the mailbox the email was sent to then i get back results. Before i was searching the senders mailbox and i was getting back 0 results, but if i search the recipient’s mailbox it works
.\graph-cleanupmailbox.ps1 -Folder all -MessageSubject “hello” -DeleteItems n -Mailbox jhofmann@ptctest.com -StartDate 24-jan-2023 -EndDate 25-jan-2023 -AppId c431769c-ad7a-47ef-8b98-602ec0ab80a4 -TenantId 03f71422-2e3d-4237-8c6f-29088405379d -AppSecret sdfsdfsdfsdfsdfsdfsdfsfdsd -SenderAddress srathore@ptctest.com -SearchQuery “hello world”
Hi Tony . What is the switch to permanently delete? i see when i use the -DeleteItems “y” the items go into the recoverable items folder.
There isn’t a switch to permanently delete in this script. I didn’t implement one.
When i run the script i get error “Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden” It looks like the script is not loading the function. How should i run the script so the function loads ?
The function loads at the start of the script. The error you’re getting is a permissions problem. Have you assigned the correct application permissions to the app and consented to their use for the organization?
That was it. I previously configured the app for “delegated permissions”. After removing the delegated permissions and add the app permissions it works. Thank you
sir i have taken this script but as new to graph api, dont know how to run this script.
Able to download the graph api powershell module and ran it. but gives me secret key correlation error with app id.
can you guide me how to give the correct app id, secret key to run this script?
You need to register a new application in Azure AD and generate a secret for the app: https://learn.microsoft.com/en-us/azure/healthcare-apis/register-application
Sir – thanks for your reply. now i have registered app under my azure and taken the token, tenant and secret key. But getting error as Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden.
Seems to be permission issue. What API permissions to be granted for this call from Powershell through Graph.?
already given for Mailread, MailReadbasic and MailWrite. What else to be given?
Has administrator consent been granted for application (not delegated) permissions? If you have assigned delegate permissions, you can get to your own data. To access other data, you need application permission.
Hi Tony,
It very useful article. I have question. Is it possible to using Graph API to Purge Group Mail Box? My Group mail box is full and Retention policy for O365 Group will be delete all file in sharepoint. I want to purge group mail box only. I tried with script and it work but very slow process. It will be good if Graph API support Purge on Group Mail Box.
Yes. Use the Groups API – https://learn.microsoft.com/en-us/graph/api/group-list-conversations?view=graph-rest-1.0&tabs=http
Here’s an example using the Microsoft Graph PowerShell SDK:
$GroupId = “72ee570e-3dd8-41d2-bc84-7c9eb8024dd4”
$uri = “https://graph.microsoft.com/v1.0/groups/” + $GroupId + “/conversations”
[array]$Data = Invoke-MgGraphRequest -Uri $Uri
$Data = $Data.Value
The $Data array now holds the first 10 conversations in the Inbox folder of the group mailbox. You can delete the messages with the delete conversations API https://learn.microsoft.com/en-us/graph/api/group-delete-conversation?view=graph-rest-1.0&tabs=http
I have given Admin consent from Application permissions setting. But still only the concerned logged in user query executed successfully and being an global admin in office 365, i am not able to access search mails query for other mailboxes through Graph explorer. Also the search query with date range does gives the error ID malformed.
If i run the script with global admin credentials from powershell, i get this error when searching for the same global admin mailbox
Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden
1. The fact that you can search your mailbox but not others implies that you’re using delegated permissions instead of application permissions. Please check the permissions assigned to the app in Azure AD. Graph Explorer uses application permissions when it can.
2. Running with global admin credentials means nothing. Permissions are granted to the app and not the user.
3. 403 errors mean that the app doesn’t have the right permissions.
4. I suspect that the malformed query is because the script isn’t able to fetch some data. The code is PowerShell, you have it and can change it. I can’t see your data and therefore can’t debug it for you.
Hello Tony, is there please analogy to update/create Retention policy tags via graph API? I am unlucky trying to find.
Thanks Z
There’s an API to create and management retention labels https://docs.microsoft.com/en-us/graph/api/resources/security-recordsmanagement-overview?view=graph-rest-beta but nothing yet to apply the labels to items.
Thanks for the detailed reply. I could able to troubleshoot the issue and given app permissions and able to delete mails for other users as an archive process. Two clarifications
1. while exporting the results to CSV, it gives error as
Export-Csv : Cannot bind argument to parameter ‘InputObject’ because it is null
2. how to run the script in report mode without deleting the content? what is the parameter to be used?
Did you look at the script comments? If you don’t set the delete parameter, the script runs in report mode.
As to Export-CSV, the error tells you what the problem is. There is nothing to export to a CSV file so the script did not generate any data.
Finding target mailboxes…
1 mailboxes found.
Processing Message 1 (Delete)
1 messages were found and processed
Information about the messages is available in d:\DeletionsList.csv
Export-Csv : Cannot bind argument to parameter ‘InputObject’ because it is null.
At .\Scripts\mailsearchdelete.ps1:349 char:14
+ $Deletions | Export-CSV -NoTypeInformation D:\DeletionsList.csv
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : InvalidData: (:) [Export-Csv], ParameterBindingValidationException
+ FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ExportCsvCo
mmand
able to find messages but still export option give error.
It works now after just copying the csv line alone from your script to mine.. seems to be some spaces between.
Also able to get report only mode.. thanks..
Why is the Message subject given as mandatory? i just need a date range to delete to reduce the ost file size. Subject seems to be a bigger search criteria to use with.
Look, it’s PowerShell. It’s up to you what you do with the code. I write code to illustrate a principal (in this case how to access mailbox items with the Graph). I do not create fully-supported solutions that are guaranteed to work out of the box. Go ahead and amend the code in whatever way you like, but remember that as soon as you download a script from the internet, you take full responsibility for the maintenance of that code.
Export $DeletionsList, not $Deletions
You mean to assign retention labels to items? Or to create retention labels and policies?
I don’t have such a script. You should use an Exchange Online retention policy to clean out the archive.
Thanks Tony,
Could you provide me the script that deletes all mails from the archive mailbox.1.5TB of archive mailbox exceeded.
Thanks for sharing
Great Post, Chief. While going through the code, I noticed that you have hard-coded the ObjectID in line 288 instead of the variable.
Thank you.
Thanks. I’ve updated GitHub. Always good to have extra eyes on code…