Microsoft’s Resilient File System (ReFS) was introduced with Windows Server 2012. ReFS is not a direct replacement for NTFS, and is missing some underlying NTFS features, but is designed to be (as the name suggests) a more resilient file system for extremely large amounts of data.

ReFS maintains high degree of compatibility with NTFS while providing enhanced data verification and auto-correction techniques as well as an integrated end-to-end resiliency to corruptions especially when used in conjunction with the storage spaces feature.

One example that springs to mind for extremely large amounts of data is Exchange databases, which have a theoretical maximum of 16TB per database file. In reality of course we tend to run much smaller database sizes. Microsoft reportedly uses 2TB as their ideal maximum database size in Exchange Online. In customer environments I’ve worked in the trend is towards more, smaller databases such as 200GB maximum for single server scenarios, and 500GB maximum for multi-server high availability (database availability group) deployments. Even falling well short of the 16TB maximum a 2TB or even 500GB database file is still “big”.

Support for ReFS with Exchange Server

From Exchange Server 2013 and upwards (which includes Exchange Server 2016 today) Microsoft supports the use of ReFS for Exchange servers, and in fact they now recommend it as the preferred file system for Exchange Server 2016, within the following guidelines.

For Exchange Server 2013:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features must be disabled for the database (.edb) files or the entire volume that hosts database files.
  • Hotfix KB2853418 must be installed.
  • For Windows 2012, the following hotfixes must be installed:

This means that you should continue to use NTFS for your operating system and Exchange Server 2013 installation volume, but you can consider using ReFS for the volumes hosting Exchange databases, log files, and index files.

For Exchange Server 2016:

  • ReFS is supported for volumes containing Exchange database files, log files, and content index files.
  • ReFS is not supported for volumes containing Exchange binaries (the program files).
  • ReFS is not supported for volumes containing the system partition.
  • ReFS data integrity features are recommended to be disabled.
  • For Windows 2012, the following hotfixes must be installed:

This means that you should continue to use NTFS for your operating system and Exchange Server 2016 installation volume, and it is recommended ReFS for the volumes hosting Exchange databases, log files, and index files. Note also that at this time there is no guidance directing you to disable ReFS data integrity features or install additional hotfixes when using ReFS with Exchange Server 2016. Updated the guidance above with some additional information that is expected to be added to TechNet soon.

Creating an ReFS Formatted Volume

In Windows Server 2012 or R2 during the New Volume Wizard when you get to the step for configuring File System Settings change the file system from NTFS to ReFS.

exchange-server-refs

However, using the New Volume Wizard does not give you the option to disable data integrity at the volume level. To set it at the volume level itself use PowerShell when configuring new volumes. On my demo server disk 3 has been added to the server and is ready to initialize and format with ReFS.

PS C:\> Get-Disk

Number Friendly Name                            OperationalStatus                    Total Size Partition Style
------ -------------                            -----------------                    ---------- ---------------
2      Microsoft Virtual Disk                   Online                                   100 GB GPT
1      Microsoft Virtual Disk                   Online                                   100 GB GPT
3      Microsoft Virtual Disk                   Online                                   100 GB RAW
0      Virtual HD ATA Device                    Online                                    95 GB MBR


PS C:\> Get-Disk 3 | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -UseMaximumSize | Format-Volume -FileSystem REFS -AllocationUnitSize 65536 -NewFileSystemLabel Volume3 -SetIntegrityStreams $false

Confirm
Are you sure you want to perform this action?
Warning, all data on the volume will be lost!
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [S] Suspend  [?] Help (default is "Y"): y

DriveLetter       FileSystemLabel  FileSystem       DriveType        HealthStatus        SizeRemaining             Size
-----------       ---------------  ----------       ---------        ------------        -------------             ----
                  Volume3          ReFS             Fixed            Healthy                  99.31 GB         99.81 GB

Additional Considerations

When you are deploying an Exchange 2016 database availability group and using Autoreseed the disk reclaimer needs to know which file system to use when formatting spare disks.

C:\> Set-DatabaseAvailabilityGroup NameOfYourDAG -FileSystem ReFS

For Exchange Server 2013 DAGs, manually format the spare volumes with ReFS.

Summary

If you are running Exchange Server 2013 on Windows Server 2012/R2 today and you are using NTFS volumes then the effort to provision new ReFS volumes and migrate your data likely exceeds the benefits of ReFS over NTFS for that scenario. If you’re deploying a new Exchange Server 2013 server today you can consider using ReFS within the support guidelines mentioned in the article above.

For Exchange Server 2016 deployments ReFS is now the recommended file system, so you should plan any new deployments to use ReFS volumes for Exchange databases, log files, and content indexes.

References:

Thanks to Ross Smith IV from Microsoft for helping to clarify some of the missing and fragmented information on this topic.

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Patrik

    I have a few Exchange 2016 server on Windows 2016 with ReFS disks mounted as mountpoints on an NTFS (4K blocksize) formatted disk. We have had some Microsoft guys here saying this is an issue and we should format the mountpoint disk with ReFS. Haven’t been able to find any information regarding this other than them saying it. They couldn’t point me to any article regarding it either. Does anyone know if this is true?

  2. Jino

    we are using REFS file system for Exchange server 2016 , is there any recommended hotfix or update for windows server 2012 R2.

    we are continuously facing issue with content index state failed.

    Verified FileIntegrity is disabled on all volume ( GPT)

  3. Aymen MAMI

    You need to mention that ReFS is not supported on RAID or SAN architecture. We only talking JBOD or S2D here.

    1. Craig Tolley

      I came here to make this exact same comment!

      1. Avatar photo
        Paul Cunningham

        I can update the article if someone has a source for that. I don’t see it on the Exchange storage config/options page on TechNet.

        1. JUanM

          Have you got Craig / Aymen about this point?

          Regards

  4. Muhammad Asif

    If I try to create the mailbox database on ReFS formated drive then Exchange 2016 shows given below error. There is no issue if I try to create the mailbox databse on NTFS formated drive. Both drives have the same perssmissions. Is anyone else face same issue?

    Both Adinistrator and System have full permissions.

    There was a problem creating directory “D:\Program Files\Microsoft\Exchange Server\V15\Mailbox\REFSDB01” on server “TL-Main-EX01.thevlab.com”. In order to ensure that the Exchange data is kept secure, please create the directory, if it does not already exist, and configure directory permissions so that “SYSTEM” and the local “Administrators” group have “Full Control” for the folder, subfolders, and files. It is recomended that permission inheritance be broken so that only “SYSTEM” and “Administrators” have access to the directory.

  5. LJ

    Hi Paul,
    2 questions.
    1. If we change the set up of one of our Exchange 2013 servers in our DAG to use Refs on our DB and log drive/s and the other servers in the DAG remained using NTFS, will this cause any problems.
    I am 90% sure it wouldn’t really matter, but I don’t know for sure.

    2. The server that we want to make the changes to, which will involve formatting the disk. Whats the best way to do this.
    My thoughts are to put the server into maintenance mode, remove the databases and logs.
    Then format the drives to use ReFS, then reseed the DBs from another Exchange server, then bring out of maintenance mode.
    You opinion would be appreciated.

    Thank you

  6. Dan Turnbull

    When running these updates they fail “The update is not applicable to your computer”.

    Have these updates been superseded?

      1. Dan Turnbull

        Hi,

        I’m running Server 2012 R2…

        Thanks

      2. Rcik

        Hello Paul,

        I´m running my Exchange 2016 on Windows server 2016. I´ve noticed that the info and hotfixes are intented for windows server 2012.

        Do I need any particular KB for Windows 2016 for may DAG 2016 to work ok with Refs? or any particular warning before I move into it?

        Thanks in advanced!

  7. Bobby West

    Hi There,

    I was wondering if I could get your thoughts on the use of REFS when doing a deployment on shared storage which has RAID and some other healing smarts built in to the file system. A couple examples would be Nutanix or even just more of a traditional Hyper-V deployment with shared storage.

    I have done some research and found a couple sites that did benchmarks which found ~7-10% decrease in IOPS from going with REFS instead of NTFS.

    Examples:
    http://www.joshodgers.com/2016/07/10/storage-performance-refs-vs-ntfs/
    https://eightwone.com/2015/05/16/exchange-file-system/

    They didnt argue that REFS didn’t have its place for the preferred architecture and JBOD w/ Autoreseed but reading between the lines it seemed they felt it was unnecessary when going with a virtualized deployment that leverages shared storage that has some protection mechanisms built in. Not sure if there is any other compelling reason to use REFS like upcoming features that would leverage it…

    Just curios on your thoughts?

  8. Toni

    same here.. anyone has news about ReFS? I cant find something about DPM in System Center 2016 is supporting ReFS or not. Maybe Azure DPM?

  9. Mike

    Does anyone know when Azure DPM will support refs for backup on exchange 2016?
    Thanks

  10. Hamada

    Why did not you assign a drive letter to the ReFS volume?

    1. Mikael

      Why use drive letters when you have mountpoints?

    2. Joey Winterrowd

      Hamada,

      If you want to assign a drive letter, use the following:

      Get-Disk 3 | Initialize-Disk -PartitionStyle GPT -PassThru | New-Partition -DriveLetter F -UseMaximumSize | Format-Volume -FileSystem REFS -NewFileSystemLabel DB-01 -SetIntegrityStreams $false

  11. Sebastien

    Very nice article! Crystal clear, straigth to the point! Thanks

  12. David

    What commands could I run to determine if SetIntegrityStreams has been set to true or false after it has been formatted? I can not seem to find it anywhere.
    Thanks

    1. Avatar photo
      Paul Cunningham

      If E: is the volume, you can run

      PS C:> Get-Item E:* | Get-FileIntegrity

      I think that will tell you about whether files/folders have integrity streams disabled, which I guess implies that the file system has it disabled, but I’m not sure of another way to be 100% sure as I’ve never looked into it. I just make sure I format them correctly at the start 😉

  13. Dave Nuss

    I know you cannot speak to backup solutions but Microsoft’s own DPM does not yet support Exchange 2016 but an MS employee said it will be supported in just a few weeks (finally). The wrinkle is it will only support NTFS, ReFS is still a bit in the future. What is the trade off by going NTFS? I cannot gather what I am really giving up if I go ahead and deploy with NTFS. I have 77 mailboxes and right now the exchange db is 100 GB.

    1. Dave Nuss

      I did want to do HA and deploy a three server DAG group as describe in your ebook I purchased.

    2. Avatar photo
      Paul Cunningham

      ReFS is more resilient. It’s not something I know a ton about, but I’m sure the benefits are well explained on TechNet.

      That said, if you’ve got an integration that doesn’t support ReFS, like a backup product, then naturally you would not use it. NTFS is still supported, ReFS is just recommended where possible.

  14. Soren Rasmussen

    Hi Paul

    Thanks for another good and helpful article.

    Do you happen to know if Exchange 2016 Databases and Logs are supported on Windows Dynamic Disks? Or recommended?

    Thanks in advance!

  15. Peter Forster

    Hi Paul,

    do you have informations if backup systems do have any issues with ReFS?
    Microsoft DPM is supporting ReFS – but no specific word on Exchange and ReFS.

    1. Avatar photo
      Paul Cunningham

      No, you should ask your backup vendor. ReFS itself is not new so hopefully the major vendors are supporting it by now.

      1. Anu

        Thanks Paul for the excellent article, based on your customer interactions will it make sense for customers upgrading from Exchange 2010/2013 to upgrade their file system to ReFS? Is it worth the pain?

        1. Avatar photo
          Paul Cunningham

          You don’t “upgrade” a file system to ReFS, you choose ReFS when you’re creating the volume to begin with. So there’s no pain, it’s just a slightly different step when you’re building the server before you put Exchange on it.

  16. filip

    Should the Refs volume still be 64k allocation unit size formatted?

      1. Benoit Boudeville

        not with Win2016 🙂 be careful default is 4KB so your instructions can be misleading

        1. Avatar photo
          Paul Cunningham

          Indeed. And the storage/sizing calculator spits out scripts that specify the allocation unit size as well now. I’ve updated my example. Thanks for the heads up.

Leave a Reply