• Topics
    • Office 365
    • Teams
    • SharePoint
    • Exchange 2019
    • Exchange 2016
    • Exchange 2013
    • Hybrid
    • Certificates
    • PowerShell
    • Migration
    • Security
    • Azure
  • Blog
  • The Practical 365 Podcast
  • Books
  • Community
  • About
  • Subscribe
    • Facebook
    • Twitter
    • RSS
    • YouTube

Practical 365

You are here: Home / Exchange Server / Resilient File System (ReFS) and Microsoft Exchange Server

Resilient File System (ReFS) and Microsoft Exchange Server

October 12, 2015 by Paul Cunningham 37 Comments

Microsoft's Resilient File System (ReFS) was introduced with Windows Server 2012. ReFS is not a direct replacement for NTFS, and is missing some underlying NTFS features, but is designed to be (as the name suggests) a more resilient file system for extremely large amounts of data.

ReFS maintains high degree of compatibility with NTFS while providing enhanced data verification and auto-correction techniques as well as an integrated end-to-end resiliency to corruptions especially when used in conjunction with the storage spaces feature.

One example that springs to mind for extremely large amounts of data is Exchange databases, which have a theoretical maximum of 16TB per database file. In reality of course we tend to run much smaller database sizes. Microsoft reportedly uses 2TB as their ideal maximum database size in Exchange Online. In customer environments I've worked in the trend is towards more, smaller databases such as 200GB maximum for single server scenarios, and 500GB maximum for multi-server high availability (database availability group) deployments. Even falling well short of the 16TB maximum a 2TB or even 500GB database file is still “big”.

Support for ReFS with Exchange Server

From Exchange Server 2013 and upwards (which includes Exchange Server 2016 today) Microsoft supports the use of ReFS for Exchange servers, and in fact they now recommend it as the preferred file system for Exchange Server 2016, within the following guidelines.

For Exchange Server 2013:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features must be disabled for the database (.edb) files or the entire volume that hosts database files.
  • Hotfix KB2853418 must be installed.
  • For Windows 2012, the following hotfixes must be installed:
    • KB2822241
    • KB2884597
    • KB2894875

This means that you should continue to use NTFS for your operating system and Exchange Server 2013 installation volume, but you can consider using ReFS for the volumes hosting Exchange databases, log files, and index files.

For Exchange Server 2016:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features are recommended to be disabled.
  • For Windows 2012, the following hotfixes must be installed:
    • KB2822241
    • KB2884597
    • KB2894875

This means that you should continue to use NTFS for your operating system and Exchange Server 2016 installation volume, and it is recommended ReFS for the volumes hosting Exchange databases, log files, and index files. Note also that at this time there is no guidance directing you to disable ReFS data integrity features or install additional hotfixes when using ReFS with Exchange Server 2016. Updated the guidance above with some additional information that is expected to be added to TechNet soon.

Creating an ReFS Formatted Volume

In Windows Server 2012 or R2 during the New Volume Wizard when you get to the step for configuring File System Settings change the file system from NTFS to ReFS.

exchange-server-refs

However, using the New Volume Wizard does not give you the option to disable data integrity at the volume level. To set it at the volume level itself use PowerShell when configuring new volumes. On my demo server disk 3 has been added to the server and is ready to initialize and format with ReFS.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
PS C:\> Get-Disk
 
Number Friendly Name                            OperationalStatus                    Total Size Partition Style
------ -------------                            -----------------                    ---------- ---------------
2      Microsoft Virtual Disk                   Online                                   100 GB GPT
1      Microsoft Virtual Disk                   Online                                   100 GB GPT
3      Microsoft Virtual Disk                   Online                                   100 GB RAW
0      Virtual HD ATA Device                    Online                                    95 GB MBR
 
 
PS C:\> Get-Disk 3 | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -UseMaximumSize | Format-Volume -FileSystem REFS -AllocationUnitSize 65536 -NewFileSystemLabel Volume3 -SetIntegrityStreams $false
 
Confirm
Are you sure you want to perform this action?
Warning, all data on the volume will be lost!
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): y
 
DriveLetter       FileSystemLabel  FileSystem       DriveType        HealthStatus        SizeRemaining             Size
-----------       ---------------  ----------       ---------        ------------        -------------             ----
                  Volume3          ReFS             Fixed            Healthy                  99.31 GB         99.81 GB

Additional Considerations

When you are deploying an Exchange 2016 database availability group and using Autoreseed the disk reclaimer needs to know which file system to use when formatting spare disks.

1
C:\> Set-DatabaseAvailabilityGroup NameOfYourDAG -FileSystem ReFS

For Exchange Server 2013 DAGs, manually format the spare volumes with ReFS.

Summary

If you are running Exchange Server 2013 on Windows Server 2012/R2 today and you are using NTFS volumes then the effort to provision new ReFS volumes and migrate your data likely exceeds the benefits of ReFS over NTFS for that scenario. If you're deploying a new Exchange Server 2013 server today you can consider using ReFS within the support guidelines mentioned in the article above.

For Exchange Server 2016 deployments ReFS is now the recommended file system, so you should plan any new deployments to use ReFS volumes for Exchange databases, log files, and content indexes.

References:

  • Exchange 2013 storage configuration options
  • Exchange 2016 Preferred Architecture
  • Exchange Storage for Insiders: It's ESE (Ignite video)

Thanks to Ross Smith IV from Microsoft for helping to clarify some of the missing and fragmented information on this topic.

Paul Cunningham

Paul is a Microsoft MVP for Office Apps and Services and a Pluralsight author. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server.

Exchange Server Exchange 2013, Exchange 2016, ReFS, Storage

Comments

  1. Patrik says

    February 8, 2019 at 11:50 pm

    I have a few Exchange 2016 server on Windows 2016 with ReFS disks mounted as mountpoints on an NTFS (4K blocksize) formatted disk. We have had some Microsoft guys here saying this is an issue and we should format the mountpoint disk with ReFS. Haven’t been able to find any information regarding this other than them saying it. They couldn’t point me to any article regarding it either. Does anyone know if this is true?

    Reply
  2. Jino says

    April 4, 2018 at 8:25 pm

    we are using REFS file system for Exchange server 2016 , is there any recommended hotfix or update for windows server 2012 R2.

    we are continuously facing issue with content index state failed.

    Verified FileIntegrity is disabled on all volume ( GPT)

    Reply
  3. Aymen MAMI says

    March 7, 2018 at 7:46 am

    You need to mention that ReFS is not supported on RAID or SAN architecture. We only talking JBOD or S2D here.

    Reply
    • Craig Tolley says

      March 26, 2018 at 8:35 pm

      I came here to make this exact same comment!

      Reply
      • Paul Cunningham says

        March 26, 2018 at 9:23 pm

        I can update the article if someone has a source for that. I don’t see it on the Exchange storage config/options page on TechNet.

        Reply
        • JUanM says

          November 14, 2018 at 1:36 pm

          Have you got Craig / Aymen about this point?

          Regards

          Reply
        • Tom Murphy says

          January 17, 2019 at 2:31 pm

          https://docs.microsoft.com/en-us/windows-server/storage/refs/refs-overview#supported-deployments

          Basic disks include local non-removable direct-attached via BusTypes SATA, SAS, NVME, or RAID.

          Reply
  4. Muhammad Asif says

    November 14, 2017 at 5:40 am

    If I try to create the mailbox database on ReFS formated drive then Exchange 2016 shows given below error. There is no issue if I try to create the mailbox databse on NTFS formated drive. Both drives have the same perssmissions. Is anyone else face same issue?

    Both Adinistrator and System have full permissions.

    There was a problem creating directory “D:\Program Files\Microsoft\Exchange Server\V15\Mailbox\REFSDB01” on server “TL-Main-EX01.thevlab.com”. In order to ensure that the Exchange data is kept secure, please create the directory, if it does not already exist, and configure directory permissions so that “SYSTEM” and the local “Administrators” group have “Full Control” for the folder, subfolders, and files. It is recomended that permission inheritance be broken so that only “SYSTEM” and “Administrators” have access to the directory.

    Reply
  5. LJ says

    January 27, 2017 at 9:03 pm

    Hi Paul,
    2 questions.
    1. If we change the set up of one of our Exchange 2013 servers in our DAG to use Refs on our DB and log drive/s and the other servers in the DAG remained using NTFS, will this cause any problems.
    I am 90% sure it wouldn’t really matter, but I don’t know for sure.

    2. The server that we want to make the changes to, which will involve formatting the disk. Whats the best way to do this.
    My thoughts are to put the server into maintenance mode, remove the databases and logs.
    Then format the drives to use ReFS, then reseed the DBs from another Exchange server, then bring out of maintenance mode.
    You opinion would be appreciated.

    Thank you

    Reply
  6. Dan Turnbull says

    October 13, 2016 at 9:34 pm

    When running these updates they fail “The update is not applicable to your computer”.

    Have these updates been superseded?

    Reply
    • Paul Cunningham says

      October 14, 2016 at 1:14 am

      Which operating system are you running?

      Reply
      • Dan Turnbull says

        October 18, 2016 at 9:43 pm

        Hi,

        I’m running Server 2012 R2…

        Thanks

        Reply
      • Rcik says

        September 11, 2019 at 10:11 am

        Hello Paul,

        I´m running my Exchange 2016 on Windows server 2016. I´ve noticed that the info and hotfixes are intented for windows server 2012.

        Do I need any particular KB for Windows 2016 for may DAG 2016 to work ok with Refs? or any particular warning before I move into it?

        Thanks in advanced!

        Reply
  7. Bobby West says

    October 1, 2016 at 5:47 am

    Hi There,

    I was wondering if I could get your thoughts on the use of REFS when doing a deployment on shared storage which has RAID and some other healing smarts built in to the file system. A couple examples would be Nutanix or even just more of a traditional Hyper-V deployment with shared storage.

    I have done some research and found a couple sites that did benchmarks which found ~7-10% decrease in IOPS from going with REFS instead of NTFS.

    Examples:
    http://www.joshodgers.com/2016/07/10/storage-performance-refs-vs-ntfs/
    https://eightwone.com/2015/05/16/exchange-file-system/

    They didnt argue that REFS didn’t have its place for the preferred architecture and JBOD w/ Autoreseed but reading between the lines it seemed they felt it was unnecessary when going with a virtualized deployment that leverages shared storage that has some protection mechanisms built in. Not sure if there is any other compelling reason to use REFS like upcoming features that would leverage it…

    Just curios on your thoughts?

    Reply
  8. Toni says

    June 10, 2016 at 6:21 pm

    same here.. anyone has news about ReFS? I cant find something about DPM in System Center 2016 is supporting ReFS or not. Maybe Azure DPM?

    Reply
    • Craig [MSFT Exchange PFE] says

      June 19, 2016 at 1:55 am

      “Support for protecting file server using Resilient File System (ReFS).”

      What’s new in System Center 2012 SP1 – DPM
      https://technet.microsoft.com/en-us/library/dn296605(v=sc.12).aspx

      Reply
  9. Mike says

    June 9, 2016 at 11:17 pm

    Does anyone know when Azure DPM will support refs for backup on exchange 2016?
    Thanks

    Reply
  10. Hamada says

    April 21, 2016 at 3:00 am

    Why did not you assign a drive letter to the ReFS volume?

    Reply
    • Mikael says

      May 28, 2016 at 12:04 am

      Why use drive letters when you have mountpoints?

      Reply
    • Joey Winterrowd says

      November 4, 2016 at 11:36 pm

      Hamada,

      If you want to assign a drive letter, use the following:

      Get-Disk 3 | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -DriveLetter F -UseMaximumSize | Format-Volume -FileSystem REFS -NewFileSystemLabel DB-01 -SetIntegrityStreams $false

      Reply
  11. Muhammad says

    April 4, 2016 at 8:06 pm

    Guyz Just sharing for your kind information!!!

    Visit the below URL regarding the ReFS
    https://technet.microsoft.com/en-us/library/ms.exch.setupreadiness.win2k12urefsupdatenotinstalled(v=exchg.160).aspx

    Reply
  12. Sebastien says

    March 15, 2016 at 7:20 pm

    Very nice article! Crystal clear, straigth to the point! Thanks

    Reply
  13. David says

    March 10, 2016 at 8:14 am

    What commands could I run to determine if SetIntegrityStreams has been set to true or false after it has been formatted? I can not seem to find it anywhere.
    Thanks

    Reply
    • Paul Cunningham says

      March 10, 2016 at 8:39 pm

      If E: is the volume, you can run

      PS C:> Get-Item E:* | Get-FileIntegrity

      I think that will tell you about whether files/folders have integrity streams disabled, which I guess implies that the file system has it disabled, but I’m not sure of another way to be 100% sure as I’ve never looked into it. I just make sure I format them correctly at the start 😉

      Reply
  14. Dave Nuss says

    March 9, 2016 at 9:02 am

    I know you cannot speak to backup solutions but Microsoft’s own DPM does not yet support Exchange 2016 but an MS employee said it will be supported in just a few weeks (finally). The wrinkle is it will only support NTFS, ReFS is still a bit in the future. What is the trade off by going NTFS? I cannot gather what I am really giving up if I go ahead and deploy with NTFS. I have 77 mailboxes and right now the exchange db is 100 GB.

    Reply
    • Dave Nuss says

      March 9, 2016 at 9:06 am

      I did want to do HA and deploy a three server DAG group as describe in your ebook I purchased.

      Reply
    • Paul Cunningham says

      March 9, 2016 at 11:40 am

      ReFS is more resilient. It’s not something I know a ton about, but I’m sure the benefits are well explained on TechNet.

      That said, if you’ve got an integration that doesn’t support ReFS, like a backup product, then naturally you would not use it. NTFS is still supported, ReFS is just recommended where possible.

      Reply
  15. Soren Rasmussen says

    January 18, 2016 at 10:46 pm

    Hi Paul

    Thanks for another good and helpful article.

    Do you happen to know if Exchange 2016 Databases and Logs are supported on Windows Dynamic Disks? Or recommended?

    Thanks in advance!

    Reply
    • Paul Cunningham says

      January 19, 2016 at 9:17 am

      Supported, but not best practice.

      Reply
  16. Peter Forster says

    October 12, 2015 at 6:39 pm

    Hi Paul,

    do you have informations if backup systems do have any issues with ReFS?
    Microsoft DPM is supporting ReFS – but no specific word on Exchange and ReFS.

    Reply
    • Paul Cunningham says

      October 12, 2015 at 6:58 pm

      No, you should ask your backup vendor. ReFS itself is not new so hopefully the major vendors are supporting it by now.

      Reply
      • Anu says

        November 14, 2015 at 12:59 am

        Thanks Paul for the excellent article, based on your customer interactions will it make sense for customers upgrading from Exchange 2010/2013 to upgrade their file system to ReFS? Is it worth the pain?

        Reply
        • Paul Cunningham says

          November 14, 2015 at 7:09 am

          You don’t “upgrade” a file system to ReFS, you choose ReFS when you’re creating the volume to begin with. So there’s no pain, it’s just a slightly different step when you’re building the server before you put Exchange on it.

          Reply
  17. filip says

    October 12, 2015 at 2:54 pm

    Should the Refs volume still be 64k allocation unit size formatted?

    Reply
    • Paul Cunningham says

      October 12, 2015 at 4:59 pm

      64K is the only choice with ReFS.

      Reply
      • Benoit Boudeville says

        October 19, 2017 at 4:47 am

        not with Win2016 🙂 be careful default is 4KB so your instructions can be misleading

        Reply
        • Paul Cunningham says

          October 19, 2017 at 10:15 am

          Indeed. And the storage/sizing calculator spits out scripts that specify the allocation unit size as well now. I’ve updated my example. Thanks for the heads up.

          Reply

Leave a Reply Cancel reply

You have to agree to the comment policy.

Recent Articles

  • Build your own custom SharePoint document library bulk provisioning system using the PowerPlatform – Part 1
  • Build your own custom SharePoint document library bulk provisioning system using the PowerPlatform – Part 2
  • The Practical 365 Podcast Ep 2: Azure AD, Message Center Updates, and Roadmap
  • The Practical 365 Podcast: Episode 01 – Exchange Updates and Roadmap
  • Introducing the Office 365 ATP Recommended Configuration Analyzer (ORCA)
Practical 365

Related Posts

Exchange Server 2016 Migration – Public Folder Migration
Migrating legacy public folders from Exchange 2010 to the new Exchange 2016 server during an
Exchange Server 2016 Migration – Migrating Mailboxes
Planning and performing the move of mailboxes during an Exchange Server 2016 migration project.
Managing Exchange Server Mailbox Quotas
How to plan and configure mailbox storage quotas on Exchange Server databases and mailboxes.

Training Courses

  • Configuring and Managing Office 365 Security
  • Office 365 Admin Playbook
  • Exchange 2016 Exam 70-345
  • Managing Exchange Mailboxes and Distribution Groups in PowerShell
  • More Training Courses...

Recommended Resources

  • Office 365 Security Resources
  • Office 365 Books
  • Exchange Server Books
  • Exchange Server Migrations
  • Exchange Analyzer
  • Digicert SSL Certificates

About This Site

Practical 365 is a leading site for Office 365 and Exchange Server news, tips and tutorials. Read more...

Find out more about advertising with us.

Contact us


Subscribe to our newsletter
  • Facebook
  • Twitter
  • RSS
  • YouTube

Copyright © 2019 Quadrotech Solutions AG · Disclosure · Privacy Policy
Alpenstrasse 15, 6304 Zug, Switzerland