Bottomless Archives Terminate on November 1

Microsoft announced “auto-expanding, highly scalable archiving” for Exchange Online on June 3, 2015. The original blog post held out promises of a “truly bottomless archive.” The new feature allowed users with the necessary licenses (Office 365 E3 and above or Exchange Online Plan 2, or even an Exchange Online archiving license) to stuff as much information as they wanted into their archive mailbox.

User mailboxes start off with 100 GB archives while shared mailboxes start with 50 GB. As information is added to the archives, Exchange Online automatically monitors the storage and if the archive gets within 10% of its limit, it adds an extra 50 GB “chunk” of storage to the chain of physical shards (storage locations) which formed the logical archive mailbox. The expansion process happens automatically without any need for the user to intervene.

The advent of the bottomless archive was a nice story which supported Microsoft’s Office 365 Import Service to help customers move PST files to Exchange Online. Auto-expanding archives also delivered a real advantage over the capabilities available to competitors like Google, and it didn’t hurt Microsoft to increase the amount of customer data in Office 365. More data means extra stickiness. In other words, it’s harder for a tenant to move off Office 365.

Roll on to 2016, and Microsoft started to enable expandable archives in customer tenants. As you might expect, the new technology met some deployment challenges and full implementation across the service didn’t happen until 2017.

Operational Issues

After getting past the deployment issues, Microsoft discovered other operational problems, like the way some migration projects took the opportunity to move vast amounts of data from legacy archiving systems to Exchange Online, often using shared mailboxes as the archive target. This led to the current guidance that an expandable archive is purely for personal use and cannot grow at more than 1 GB/day. Microsoft saysA user’s archive mailbox is intended for just that user. Using journaling, transport rules, or auto-forwarding rules to copy messages to an archive mailbox is not permitted. Microsoft reserves the right to deny unlimited archiving in instances where a user’s archive mailbox is used to store archive data for other users or in other cases of the inappropriate use.”

Very large archives caused other concerns, such as the amount of time required to move mailboxes with large archives between mailbox databases or servers (Microsoft commonly rebalances mailbox load across Exchange Online servers). From a customer perspective, once you enable expandable archives, you can’t move these mailboxes back to on-premises servers because Exchange Server doesn’t support the feature.

Reining in Bottomless Archives

Concerned by the use of auto-expanding archives, Microsoft attempted to rein in their growth by imposing a 1 TB maximum in November 2019. The limitation was artificial in that it did not exist in software and no mechanism was available to control growth past the new limit. In addition, Microsoft’s communication around the introduction of the new limit was terrible and customers didn’t react well.

In January 2020, Microsoft retreated from the 1 TB limit and updated the Exchange Online service description to remove any mention of a maximum archive size. At the time, I said that we would hear about this topic again “after Microsoft has worked through the ins and outs of the decision and created a proper communications and implementation plan. Tenants will be told, administrators will be given the tools to manage large archive mailboxes, and the limit will be enforced.

November 1 Introduces a New 1.5 TB Limit

So far, MC288051 (published September 28) is the communications plan. Microsoft says that they’re going to remove the word “unlimited” from the Exchange Online service description and create a non-configurable 1.5 TB limit for archive mailboxes. The change happens on November 1, 2021, and when the new limit is active, users will be unable to add more information to archive mailboxes if it pushes the mailbox past the 1.5 TB limit. In technical terms, Exchange Online will allow you to move data into the archive up to the limit but will not add any more 50 GB chunks to expand the archive mailbox.

I suspect that the new limit will not affect many users. Curiously, in MC288051, Microsoft mentions that customers “have previously worked with Microsoft Support to provide exceptions for existing archives exceeding 1.5TB, those specific archives will not be affected by this change.” That sounds like the limit was effective previously, perhaps after customers with very large archives encountered problems in their operation which required intervention from Microsoft support.

Reporting Archive Mailboxes

In any case, the new limit will arrive on November 1, 2021. It’s time to check the set of archive mailboxes in your tenant just in case some are approaching 1.5 TB. It’s easy to check the status of the most likely culprits by using the mailboxes view in the new EAC (Figure 1).

Examining a mailbox's archive statistics using the new Exchange Online admin center
Figure 1: Examining a mailbox’s archive statistics using the new Exchange Online admin center

Hunting down individual mailboxes in a GUI is boring and this is exactly the kind of situation when PowerShell shines. Here’s some code using the Exchange Online management module to find all archive-enabled shared and user mailboxes in a tenant and generate a report. Not all archive-enabled mailboxes have expanding archives.

#
# Find archive-enabled mailboxes and report their status
[array]$Mbx = Get-ExoMailbox -RecipientTypeDetails SharedMailbox, UserMailbox -Filter {ArchiveStatus -ne $Null} -ResultSize Unlimited -Properties ArchiveQuota, ArchiveStatus, AutoExpandingArchiveEnabled, RecipientTypeDetails
If ($Mbx -eq 0) { Write-Host "No mailboxes found with archives" ; break}

$Report = [System.Collections.Generic.List[Object]]::new()
ForEach ($M in $Mbx) {
   Write-Host "Processing mailbox" $M.DisplayName
   $ExpandingArchive = "No"
   If ($M.AutoExpandingArchiveEnabled -eq $True) { $ExpandingArchive = "Yes" }
   $Stats = Get-ExoMailboxStatistics -Archive -Identity $M.UserPrincipalName
   [string]$ArchiveSize = $Stats.TotalItemSize.Value
   [string]$DeletedArchiveItems = $Stats.TotalDeletedItemSize.Value 
	
   $ReportLine = [PSCustomObject][Ordered]@{  
       Mailbox             = $M.DisplayName
       UPN                 = $M.UserPrincipalName
       Type                = $M.RecipientTypeDetails
       ArchiveQuota        = $M.ArchiveQuota.Split("(")[0] 
       Expanding           = $ExpandingArchive
       ArchiveStatus       = $M.ArchiveStatus
       ArchiveSize         = $ArchiveSize.Split("(")[0] 
       ArchiveItems        = $Stats.ItemCount
       DeletedArchiveItems = $DeletedArchiveItems.Split("(")[0] 
       DeletedItems        = $Stats.DeletedItemCount    
    }
    $Report.Add($ReportLine) 
} #End ForEach

Write-Host ("{0} mailboxes processed" -f $Mbx.count)
$Report | Out-GridView 
$Report | Export-CSV -NoTypeInformation c:\temp\ArchiveMailboxes.csv

The output of the report is a CSV file. We also output the data on-screen using the Out-GridView cmdlet (Figure 2).

Report about the use of Exchange Online archive mailboxes
Figure 2: Report about the use of Exchange Online archive mailboxes

Update: The script described in this post reports the growth rate of current archives and the number of days before archives will reach the new 1.5 TB limit.

Limited Effect of New Limit

It’s hard to say how much difference this change will make in a practical sense. Most Exchange Online tenants will never generate a 1.5 TB archive. It takes real effort to accumulate a monster archive. Using the 1 GB/day limit, the 1,536 GB will take 4.2 years to fill, or 7.68 years at a more reasonable 200 working days per year. Those who need very large archives are already aware of the requirement and are probably enterprise tenants with ongoing communication with local Microsoft representatives. This is just another thing to add to their list of ups and downs with Microsoft 365.

About the Author

Tony Redmond

Tony Redmond has written thousands of articles about Microsoft technology since 1996. He is the lead author for the Office 365 for IT Pros eBook, the only book covering Office 365 that is updated monthly to keep pace with change in the cloud. Apart from contributing to Practical365.com, Tony also writes at Office365itpros.com to support the development of the eBook. He has been a Microsoft MVP since 2004.

Comments

  1. Jakub

    Hey,

    Do you have an idea what will happens with the e-mail data with below scenario?

    – Auto-expanding is enabled for entire organization
    – We have an active mailbox with E3 license with more than 100 GB of data
    – We decided to convert mailbox to the shared mailbox and remove the E3 license

    Will the archive gone due to the lack of E3/Mailbox archive license?

    1. Avatar photo
      Tony Redmond

      You need a license (Exchange Online Archiving) for the shared mailbox to maintain its archive. I don’t think Microsoft will remove the archive, but they are entitled too if you don’t license it.

  2. Peter Smith

    Great article and summary Tony.

    The 1 GB per day limit is a real problem.

    Strange that in OWA this is called an “INPLACE-Archive” and on Outlook it is listed as “ONLINE ARCHIVE”.

    Pity you cannot access this Online Archive via Android or iPhone Outlook mail clients.

    1. Avatar photo
      Tony Redmond

      The names chosen by OWA and Outlook aren’t important. I wouldn’t worry about it. And the reason why you can’t access the archive mailboxes via mobile clients is that the archive is designed for retention of information that needs very intermittent access. Most times on the mobile, you’re using it as a triage device to work through your incoming email. If you need access to the archive via a mobile, run OWA in a browser session and access it there.

      1. Jim Fracassa

        I must disagree. The naming conventions are critical. I for one find them all terribly confusing. And when they change the names just because they feel like. UGH.

        The Exchange Plans each have “(formerly named Exchange x)” following their name. What is now Standard used to be called Premium. And why do they have Microsoft 365 Business Basic” Microsoft 365 Business Standard” and then the next level up is called “Office 365 E3” – I mean c’mon

        1. Avatar photo
          Tony Redmond

          Microsoft Marketing needs to keep itself busy. A nice rebranding exercise does the job…

  3. Charles Carmichael

    I’d love to see an article from someone that is not technical but looking at this from the legal perspective. How is this legal?

    MS makes a contractual obligation through marketing and the service description for each workload and service plan.

    When signing up for these services, we all make business decisions based upon those service descriptions.

    How can they keep changing the rules anytime they want? Shouldn’t existing users be grandfathered in?

  4. Rob

    Hi Tony
    Do you know if the 1.5tb limit also applies to the recoverable storage limit of the archive?
    I can see legal firms easily hitting this with legal hold requirements and 3rd party archiving solutions that delete the email once filed to the 3rd party solution but legal hold retaining the item for the legal hold retention period.

    1. Avatar photo
      Tony Redmond

      Quota limits apply to the regular folders, not the Recoverable Items structure.

      1. Matt

        The Microsoft documentation is now being updated – it seems this limit will apply to the Recoverable Items structure as well: https://docs.microsoft.com/en-us/microsoft-365/compliance/increase-the-recoverable-quota-for-mailboxes-on-hold?view=o365-worldwide

        Having this as a hard limit with no option to increase (even for extra money) seems a bit odd. Granted, most orgs won’t come near this, but for any regulated company with long retention requirements and/or those who have adopted the E5 eDiscovery data connectors (which also use the user mailbox for storage), having this quota imposed is probably going to have a chilling effect.

        Oh well, I’ll just pop something on UserVoice…oh…hang on :\

        1. Avatar photo
          Tony Redmond

          The text in the documentation is not as clear as I would like: “Additional archive storage space up to 1.5 TB will be provisioned when necessary.” This could be read to mean the size of regular folders or it could be the entire mailbox (i.e., including Recoverable Items).

          https://docs.microsoft.com/en-us/microsoft-365/compliance/autoexpanding-archiving?view=o365-worldwide says:
          “The archiving feature in Microsoft 365 (called auto-expanding archiving) provides up to 1.5 TB of additional storage in archive mailboxes. When the storage quota in the archive mailbox is reached, Microsoft 365 automatically (and incrementally) increases the size of the archive until the archive mailbox reaches 1.5 TB.” That’s not any clearer.

          I’ve reached out to Microsoft to seek clarification…

          1. Avatar photo
            Tony Redmond

            Microsoft says that the archive size is measured at the total size of the mailbox, including the Recoverable Items folders. They’ll update the documentation to make this clearer.

          2. Avatar photo
            Tony Redmond

            I wrote this to help people understand how close their archives are to the new limit and the growth rate for their current archive:

            How to Find Exchange Online Archive Mailboxes Close to the New 1.5 TB Limit

            A 1.5 TB limit applies to Exchange Online archive mailboxes from November 1, 2021. In this article, we use PowerShell to report how close expandable archives are to the new limit. In reality, not many archive mailboxes will approach the new limit, but it’s nice to know things like the daily growth rate for an archive and how many days it will take for an archive to reach 1.5 TB. All whimsical stuff calculated with PowerShell!

            https://office365itpros.com/2021/10/04/how-to-find-large-exchange-online-archive-mailboxes/

  5. Bastiaan Kortenbout

    Hello Tony. Great article. Thanks for info!! I’ve got a client that is also using a shared mailbox with exchange plan 2 license. They use the archive of the shared mailbox for archiving project related mails. The size is around 200GB. Since an Outlook desktop app update the mailbox search does not work anymore. Outlook Online search through mailbox archive is still possible. Is searching an unlimited mailbox archive officially supported from Outlook desktop app?

Leave a Reply