Removing Debris from User and Shared Mailboxes

From time to time, the need exists to remove items from user mailboxes. Perhaps someone has shared information in an email that they shouldn’t have, or maybe some pernicious malware-ridden message has penetrated Exchange Online Protection and reached inboxes. You could ask users to remove the offending messages manually, but it’s usually more efficient and effective if administrators step in and remove the problem.

In the on-premises world, the Search-Mailbox cmdlet does the job. Microsoft’s enthusiasm to embrace compliance features that work across multiple workloads sometimes has unfortunate consequences, and their decision to deprecate Search-Mailbox in July 2020 removed a valuable tool. At the time of writing, Search-Mailbox still works, but many are put off using it because of its unsupported nature.

Microsoft’s recommended replacement for Search-Mailbox is to use purge actions with compliance searches to remove items from Exchange Online mailboxes. This method uses a compliance search to find matching items followed by the execution of a purge action to remove the items. It works, but processing is slow and seems to be getting slower as time goes by. Last week, I spent some time working on the example script I wrote in 2019 to show how to loop through compliance search results to purge items, and the performance was dreadful. Investigating a script to do the job without using either Search-Mailbox or compliance searches seemed like a good thing, if only to learn how to solve the problem using Graph API requests.

Charting Out the Script

The basic outline of the script is:

  • Collect the parameters for the search query.
  • Find mailboxes to search.
  • Run the search query against the mailboxes. Because we’re using Graph API requests, we need a registered Azure AD with consent to use the Mail.ReadWrite application permission. Read access is necessary to retrieve message data. Write access is needed to delete messages from mailboxes.
  • Apply the action to the messages found by the query. I wanted options to report the messages and delete the messages.

Now that we know what steps must be taken, let’s examine some of the detail.

First Attempt

The original version of the script was very simple. I used the Graph message resource type to retrieve messages from a mailbox with a filter to find the desired messages. Applying a filter finds messages with an exact match. For instance, this request finds all messages with a subject of “Project Ankara To Do Items”:

https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/messages?$select=id,receivedDateTime,subject,from&$filter=subject eq 'Project Ankara To Do Items'

The select request succeeded but only returns a limited set of (because of the exact match), which are unlike those retrieved by Search-Mailbox or a compliance search.

Before getting any deeper, let me emphasize that the Graph Explorer is an essential tool when it comes to working out the details of Graph API requests. You can take a URI calculated by the script and input it into the Graph Explorer to see what happens. Once a request works in the Graph Explorer, you know it will work in the script.

I then moved on to the Microsoft Search API, which is probably where I should have started. The Search API uses Keyword Query Language (KQL) queries to scan mailboxes to find matching messages based on their metadata (properties) or content (message body and attachments). Although clients differ in their implementation, this API is what Outlook clients use for searching.

The initial challenge is to compose a suitable KQL query. If you’re used to composing search criteria for compliance searches (or for Search-Mailbox) then you know that KQL supports a set of specific properties to search email items. The script accepts a bunch of parameters including the message subject, keywords, date range, and sender address that it processes to create the query.

# Prepare search filter
$SearchFilter = "subject:$MessageSubject" 

If ($SenderAddress) {
  $SearchFilter = $SearchFilter + " AND from:$SenderAddress" }

If ($StartDate) {
   $StartDateFilter = (Get-Date $StartDate).toString('yyyy-MM-dd') }
 If (($StartDateFilter) -and (!($EndDate))) { # if we have a start date but no end date, set to today's date
   $EndDate = Get-Date }
 If ($EndDate) {
   $EndDateFilter = (Get-Date $EndDate).toString('yyyy-MM-dd') }

If (($StartDateFilter) -and ($EndDateFilter)) {
  $SearchFilter = $SearchFilter + " AND received>=$StartDateFilter AND received<=$EndDateFilter" }

If ($SearchQuery) {
  $SearchFilter = $SearchFilter + " AND '" + $SearchQuery + "'" }

Here’s what an assembled KQL query looks like. This query looks for messages with “Update to Office 365 Book” anywhere in the message subject sent by Kim Akers between 1 January 2022 and 1 August 2022:

subject:Update to Office 365 Book AND from:Kim.Akers@Office365itpros.com AND received>=2022-01-01 AND received<=2022-08-01

Another example searches for the daily Viva Briefing emails

subject:Your daily briefing AND from:viva-noreply@microsoft.com AND received>=2021-01-01 AND received<=2022-08-02 AND 'Better with Microsoft Viva'

To perform the search, we compose a Graph API request like the following. You can see how the KQL query is passed and that we tell the Graph to return a set of properties for each message.

https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/mailfolders//messages?$search="subject:Update to Office 365 Book AND from:Kim.Akers@Office365itpros.com AND received>=2022-01-01 AND received<=2022-08-01"&$select=id,parentfolderid,receivedDateTime,subject,from

Searching Target Folders

This brings me to target folders. My script allows the user to search a single folder or all folders in the mailbox. If a search processes all folders, the results are like those generated by a search in Outlook desktop with just one issue. The search doesn’t scan items in the Deleted Items folder. This comes down to the implementation of search in different clients. Outlook desktop doesn’t search the Deleted Items folder unless you explicitly search that folder, but OWA does. Both clients use the Microsoft Search API feature of presenting the three most relevant items found by the search first, whereas I want a simple list. The Search-Mailbox cmdlet finds items in all user-visible folders plus the folders in the Recoverable Items structure.

The solution is simple: do one search for the Deleted Items folder and another for all other folders and combine the results. Maybe there’s a better way to search all user-visible folders, but I couldn’t find one and my workaround is effective.

At present, the Graph API can’t access archive mailboxes. If this functionality is needed, you’d need to write it using Exchange Web Services.

Folder Names

The information returned for a message found by a search looks like this:

@odata.etag      : W/"FgAAAA=="
id               : AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNABGAAAAAAB_7ILpFNx8TrktaK8VYWerBwBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA3tTkMTDKYRI6zB9VW59QNAASRem3ZAAA=
receivedDateTime : 2022-01-08T00:21:02Z
subject          : Office 365 Roadmap Updated: 2022-01-08
parentFolderId   : AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNAAuAAAAAAB_7ILpFNx8TrktaK8VYWerAQBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA=

The folder where the message is located is stored in the parentFolderId. The value makes perfect sense to a computer but isn’t much good to a human. I needed a way to resolve these values to something that made sense, like Inbox. Deleted Items or Sent Items. To interpret the identifiers, I use the List mailFolders API to fetch the set of folders within a mailbox. The data returned includes the folder identifier and display name, so I could build a hash table keyed on the folder identifier to use as a lookup.

Processing Child Folders

Many mailboxes have a simple one-level folder structure, so it’s enough to fetch the list of folders from the mailbox root. Other mailboxes have nested folders, like those shown in Figure 1.

Nested folders in Outlook
Figure 1: Nested mailbox folders as displayed in Outlook desktop

The information returned by the List mailFolders API includes a count of child folders, so it’s easy to know which folders include folders. To build out the hash table to include folder identifiers and display names of nested folders, the script iterates down four levels to find and process any child folders it can find. It’s strange that the List mailFolders API doesn’t have a qualifier to instruct the Graph to fetch all folders in a mailbox, but that’s the way things work.

Report or Delete

Hopefully, when the script runs, it finds some messages in the target mailboxes (Figure 2).

The mailbox clean-up script scans mailboxes to find messages to delete
Figure 2: The mailbox clean-up script scans mailboxes to find messages to delete

A parameter determines if the script then reports the messages or deletes (and reports) the messages. To delete a message, the script builds a URI containing the message identifier and then makes a Delete request to remove the message. Here’s what a URI and the API request look like:

https://graph.microsoft.com/v1.0/users/aff4cd58-1bb8-4899-94de-795f656b4a18/messages/AAMkADAzNzBmMzU0LTI3NTItNDQzNy04NzhkLWNmMGU1MzEwYThkNABGAAAAAAB_7ILpFNx8TrktaK8VYWerBwBe9CuwLc2fTK7W46L1SAp9AAAA2lHKAAA3tTkMTDKYRI6zB9VW59QNAASRem3ZAAA=
$Status = Invoke-RestMethod $Uri -Method 'Delete' -Headers $Headers

Deleted messages go into the Deletions sub-folder of Recoverable Items. Users can recover items from this folder. If you want them to be unable to recover items, you could move the messages to a hidden folder (like the Purges sub-folder in Recoverable Items) instead.

All Set for Deleting (After Testing)

You can download the latest version of the script from GitHub. Please test it thoroughly before you do any real damage. Don’t fall into the trap that so many have in the past by running a utility that removes emails without knowing what you’re doing. In particular, pay attention to the keywords used in the KQL queries as these can find items that you don’t expect.

I usually run the script in report mode to see what messages are found (Figure 3) before I let it delete anything. Although users can recover deleted items, it’s still better if administrators don’t remove items that they don’t need to. Feel free to live on the edge and delete messages without checking if that’s what you like to do.

Reviewing messages found by the mailbox clean-up script
Figure 3: Reviewing messages found by the mailbox clean-up script

If you find bugs, please file an issue in GitHub. Even better, fix the bug and let tell me what you’ve done. I appreciate any help you can give!

The Microsoft 365 Kill Chain and Attack Path Management

An effective cybersecurity strategy requires a clear and comprehensive understanding of how attacks unfold. Read this whitepaper to get the expert insight you need to defend your organization!

Learn More

About the Author

Tony Redmond

Tony Redmond has written thousands of articles about Microsoft technology since 1996. He is the lead author for the Office 365 for IT Pros eBook, the only book covering Office 365 that is updated monthly to keep pace with change in the cloud. Apart from contributing to Practical365.com, Tony also writes at Office365itpros.com to support the development of the eBook. He has been a Microsoft MVP since 2004.

Comments

  1. skip

    below is my search query. The search runs however its not finding anything. I am testing this out in my lab, and i have sent a couple of emails with “hellow” in the subject and “hellow world” in the body, but the search is not finding the data

    -Folder all -SearchQuery “hellow world” -MessageSubject “hello” -DeleteItems n -Mailbox srathore@ptctest.com -StartDate 1-jan-2023 -EndDate 1-jan-2023 -AppId c431769c-ad7a-47ef-8b98-602ec0ab80a4 -TenantId 03f71422-2e3d-4237-8c6f-29088405379d -AppSecret 45345345345345345345 -SenderAddress srathore@ptctest.com

    1. Avatar photo
      Tony Redmond

      Can you find the messages with a content search using the same criteria? Also, I would make sure that your search end date is greater than the start date.

      1. skip

        okay interesting. if i change the query to search the mailbox the email was sent to then i get back results. Before i was searching the senders mailbox and i was getting back 0 results, but if i search the recipient’s mailbox it works

        .\graph-cleanupmailbox.ps1 -Folder all -MessageSubject “hello” -DeleteItems n -Mailbox jhofmann@ptctest.com -StartDate 24-jan-2023 -EndDate 25-jan-2023 -AppId c431769c-ad7a-47ef-8b98-602ec0ab80a4 -TenantId 03f71422-2e3d-4237-8c6f-29088405379d -AppSecret sdfsdfsdfsdfsdfsdfsdfsfdsd -SenderAddress srathore@ptctest.com -SearchQuery “hello world”

      2. skip

        Hi Tony . What is the switch to permanently delete? i see when i use the -DeleteItems “y” the items go into the recoverable items folder.

        1. Avatar photo
          Tony Redmond

          There isn’t a switch to permanently delete in this script. I didn’t implement one.

  2. skip

    When i run the script i get error “Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden” It looks like the script is not loading the function. How should i run the script so the function loads ?

    1. Avatar photo
      Tony Redmond

      The function loads at the start of the script. The error you’re getting is a permissions problem. Have you assigned the correct application permissions to the app and consented to their use for the organization?

      1. Skip

        That was it. I previously configured the app for “delegated permissions”. After removing the delegated permissions and add the app permissions it works. Thank you

  3. sundaresan

    sir i have taken this script but as new to graph api, dont know how to run this script.
    Able to download the graph api powershell module and ran it. but gives me secret key correlation error with app id.
    can you guide me how to give the correct app id, secret key to run this script?

      1. Sundaresan

        Sir – thanks for your reply. now i have registered app under my azure and taken the token, tenant and secret key. But getting error as Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden.
        Seems to be permission issue. What API permissions to be granted for this call from Powershell through Graph.?
        already given for Mailread, MailReadbasic and MailWrite. What else to be given?

        1. Avatar photo
          Tony Redmond

          Has administrator consent been granted for application (not delegated) permissions? If you have assigned delegate permissions, you can get to your own data. To access other data, you need application permission.

  4. Supakit

    Hi Tony,

    It very useful article. I have question. Is it possible to using Graph API to Purge Group Mail Box? My Group mail box is full and Retention policy for O365 Group will be delete all file in sharepoint. I want to purge group mail box only. I tried with script and it work but very slow process. It will be good if Graph API support Purge on Group Mail Box.

    1. Avatar photo
      Tony Redmond

      Yes. Use the Groups API – https://learn.microsoft.com/en-us/graph/api/group-list-conversations?view=graph-rest-1.0&tabs=http

      Here’s an example using the Microsoft Graph PowerShell SDK:

      $GroupId = “72ee570e-3dd8-41d2-bc84-7c9eb8024dd4”
      $uri = “https://graph.microsoft.com/v1.0/groups/” + $GroupId + “/conversations”
      [array]$Data = Invoke-MgGraphRequest -Uri $Uri
      $Data = $Data.Value

      The $Data array now holds the first 10 conversations in the Inbox folder of the group mailbox. You can delete the messages with the delete conversations API https://learn.microsoft.com/en-us/graph/api/group-delete-conversation?view=graph-rest-1.0&tabs=http

    2. sundaresan

      I have given Admin consent from Application permissions setting. But still only the concerned logged in user query executed successfully and being an global admin in office 365, i am not able to access search mails query for other mailboxes through Graph explorer. Also the search query with date range does gives the error ID malformed.
      If i run the script with global admin credentials from powershell, i get this error when searching for the same global admin mailbox
      Get-GraphData : System.Net.WebException: The remote server returned an error: (403) Forbidden

      1. Avatar photo
        Tony Redmond

        1. The fact that you can search your mailbox but not others implies that you’re using delegated permissions instead of application permissions. Please check the permissions assigned to the app in Azure AD. Graph Explorer uses application permissions when it can.
        2. Running with global admin credentials means nothing. Permissions are granted to the app and not the user.
        3. 403 errors mean that the app doesn’t have the right permissions.
        4. I suspect that the malformed query is because the script isn’t able to fetch some data. The code is PowerShell, you have it and can change it. I can’t see your data and therefore can’t debug it for you.

  5. Zbyněk

    Hello Tony, is there please analogy to update/create Retention policy tags via graph API? I am unlucky trying to find.

    Thanks Z

    1. sundaresan

      Thanks for the detailed reply. I could able to troubleshoot the issue and given app permissions and able to delete mails for other users as an archive process. Two clarifications
      1. while exporting the results to CSV, it gives error as
      Export-Csv : Cannot bind argument to parameter ‘InputObject’ because it is null
      2. how to run the script in report mode without deleting the content? what is the parameter to be used?

      1. Avatar photo
        Tony Redmond

        Did you look at the script comments? If you don’t set the delete parameter, the script runs in report mode.

        As to Export-CSV, the error tells you what the problem is. There is nothing to export to a CSV file so the script did not generate any data.

        1. sundaresan

          Finding target mailboxes…
          1 mailboxes found.
          Processing Message 1 (Delete)

          1 messages were found and processed

          Information about the messages is available in d:\DeletionsList.csv

          Export-Csv : Cannot bind argument to parameter ‘InputObject’ because it is null.
          At .\Scripts\mailsearchdelete.ps1:349 char:14
          + $Deletions | Export-CSV -NoTypeInformation D:\DeletionsList.csv
          + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
          + CategoryInfo : InvalidData: (:) [Export-Csv], ParameterBindingValidationException
          + FullyQualifiedErrorId : ParameterArgumentValidationErrorNullNotAllowed,Microsoft.PowerShell.Commands.ExportCsvCo
          mmand

          able to find messages but still export option give error.

          1. sundaresan

            It works now after just copying the csv line alone from your script to mine.. seems to be some spaces between.
            Also able to get report only mode.. thanks..

            Why is the Message subject given as mandatory? i just need a date range to delete to reduce the ost file size. Subject seems to be a bigger search criteria to use with.

          2. Avatar photo
            Tony Redmond

            Look, it’s PowerShell. It’s up to you what you do with the code. I write code to illustrate a principal (in this case how to access mailbox items with the Graph). I do not create fully-supported solutions that are guaranteed to work out of the box. Go ahead and amend the code in whatever way you like, but remember that as soon as you download a script from the internet, you take full responsibility for the maintenance of that code.

          3. Avatar photo
            Tony Redmond

            Export $DeletionsList, not $Deletions

    2. Avatar photo
      Tony Redmond

      You mean to assign retention labels to items? Or to create retention labels and policies?

  6. Avatar photo
    Tony Redmond

    I don’t have such a script. You should use an Exchange Online retention policy to clean out the archive.

  7. Kiran P

    Thanks Tony,
    Could you provide me the script that deletes all mails from the archive mailbox.1.5TB of archive mailbox exceeded.

  8. bob

    Thanks for sharing

  9. Dinesh

    Great Post, Chief. While going through the code, I noticed that you have hard-coded the ObjectID in line 288 instead of the variable.
    Thank you.

    1. Avatar photo
      Tony Redmond

      Thanks. I’ve updated GitHub. Always good to have extra eyes on code…

Leave a Reply