Home » Exchange Server » Installing Cumulative Updates on Exchange Server 2016

Installing Cumulative Updates on Exchange Server 2016

This article will demonstrate the step by step process for installing cumulative updates for Exchange Server 2016.

The steps for installing cumulative updates on Exchange 2016 are:

  1. Prepare by downloading update files, checking backups, and reviewing known issues
  2. Update mailbox servers in the internet-facing sites
  3. Update mailbox servers in any remaining internal sites (if any)
  4. Update Edge Transport servers (if any)
  5. Perform health checks and rebalancing of servers

Preparation

Before you install any cumulative updates on your Exchange 2016 servers, you should first:

  • Download the cumulative update from Microsoft. Do not download from any third party websites. You can download the latest cumulative update and upgrade an Exchange 2016 to the latest version in one update. You do not need to install all of the cumulative updates released between your current version and the latest version.
  • Verify that you have confirmed, working backups of your Active Directory.
  • Verify that you have confirmed, working backups of your Exchange servers and databases.
  • Verify that you have documented any customizations to your Exchange server that will need to be re-applied, such as custom OWA login pages, web.config changes, registry changes, or third party add-ons. Generally speaking you do not need to re-apply standard Exchange configurations that are set via the Exchange Admin Center or Exchange management shell (e.g. changing default message size limits).
  • Verify that your Exchange SSL certificates have not expired.

Known Issues

Comprehensive lists of known issues with cumulative update installation generally do not exist, however to improve your awareness of issues experienced by other customers, you should read the comments on the Exchange team blog entry for the relevant cumulative update, and check the TechNet forums for other reported issues.

You should also be aware of the following issues:

Order of Installation of Exchange 2016 Updates

Cumulative updates for Exchange 2016 should be installed in the internet-facing site first, before installing in other sites in the organization.

  • Mailbox servers are updated first
  • Edge Transport servers can be updated last

For load-balanced servers and Exchange 2016 DAG members, there will be a period of time during which all servers are not at the same version. This is expected, and supported, but you should plan to continue upgrading servers so that they are all updated within a reasonable period of time. You can balance that recommendation with the need for caution, e.g. waiting for issues to arise on the first upgraded server before deploying to the other servers. As a rule of thumb, aim for “days or weeks” rather than “months” between server upgrades, depending on the size of your environment.

Deploying Exchange 2016 Cumulative Updates

The process for installation is as follows:

  1. Perform the Active Directory schema changes and updates. This is performed once for the entire Active Directory environment. You do not need to repeat this for each server being upgraded.
  2. Upgrade servers. For each server in turn:
    • Place the server into maintenance mode.
    • Install the update.
    • Perform testing.
    • Take the server out of maintenance mode.
  3. Perform post-installation tasks:
    • Rebalance database availability groups.
    • Restore customizations.
    • Perform a health check of the environment.

Active Directory Schema Changes and Updates

Most cumulative updates will include Active Directory schema changes, as well as other updates such as changed to RBAC roles. In some cases, the existence of changes will depend on which previous CU you're upgrading from. So as a general rule you should plan for AD schema changes and updates to occur.

The AD preparation tasks can be run in advance of your server upgrades, or they can be allowed to run automatically as part of the first server upgrade process. In either case, Enterprise Admins and Schema Admins rights will be required. And if you're running the update from an Exchange server, the RSAT-ADDS feature must be installed.

Before applying the schema update follow the steps provided by Michael B Smith to retrieve the existing Exchange schema version, so that you can compare it before and after the AD preparation steps have been completed to verify that the schema update was applied.

  1. Run setup.exe /PrepareSchema /IAcceptExchangeServerLicenseTerms (requires Enterprise Admins and Schema Admins permissions, and must be performed in the same AD Site as the Schema Master on a server with the RSAT-ADDS-Tools feature installed – the Schema Master itself would meet these requirements)
  2. Run setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms
  3. Run setup.exe /PrepareDomain /IAcceptExchangeServerLicenseTerms in each domain in your forest that contains Exchange servers or mailboxes. If you have a single domain, the previous step has already done this for you.

When the Active Directory changes have been applied, on each server run the upgrade.

Upgrading Exchange 2016 Servers

For Exchange 2016 Mailbox and Edge Transport servers, whether they are standalone, load-balanced, or part of a DAG, use the following procedure.

Set the HubTransport component to “Draining”, and redirect any messages currently in the queue to another server. If you're running a single Exchange server, you can skip the redirect command.

If the server is a DAG member, run the following commands. If your server is not a DAG member, skip to the command for setting ServerWideOffline.

Disable database copy auto-activation. This command will also move any active database copies to other DAG members, assuming there are other healthy DAG members available. This is not instantaneous, it can take several minutes for the moves to occur. We'll check on it shortly anyway.

Make a note of the database copy auto-activation policy on the server, so you can set it back to this value at the end of maintenance.

If the policy is not already set to “Blocked”, run the following command to set it.

Check for any database copies that are still mounted on the server. This command should return no results. If any database copies are still active on the server, and there are other DAG members that host copies of the database, perform a manual switchover.

Place the server into maintenance mode.

For servers that are in a load-balanced pool:

  • Verify that the load balancer health checks have taken the server out of the pool or marked it as offline/inactive.
  • If the load balancer does not automatically do this, manually mark the server as offline/inactive.

For servers that are in a DNS round robin group, remove the DNS record for this server's IP address.

Before you run Exchange setup to install the cumulative update:

  • Perform a restart of the server to clear any pending reboot status that will stop Exchange setup from running.
  • Verify that the PowerShell execution policy is set to Unrestricted as per KB981474.

After the restart, launch an elevated CMD prompt, and run the following command from the folder where the Exchange setup files are located:

After the cumulative update has installed, restart the server. When the server has been restarted, perform a basic health check of the server:

  • Review event logs for new or excessive errors and warnings
  • Check that auto-start services on the server have started

You can now remove the server from maintenance mode. Note, if the server is not a DAG member, then only the first and last commands are necessary. If the server is a DAG member, use the database copy auto-activation policy value that was set on the server prior to being placed into maintenance mode (the default is “Unrestricted”).

Post-Installation Tasks

After deploying an Exchange 2016 cumulative update there are some post-installation tasks that you should perform.

Rebalance Database Availability Groups

Throughout the update process the database copies in your DAG will have been moved between DAG members, possibly multiple times. If you want to return your active database copies to their most preferred DAG member (aka “rebalancing the DAG”), use the PowerShell script supplied by Microsoft.

Restore Customizations

After you have completed updating your servers you will need to re-apply any customizations that you had documented during the preparation steps above.

Perform a Health Check of Servers

Here are some suggestions for health checking your Exchange 2013 servers after applying updates.

  • Check the cluster nodes are all up – verify that you have not left any DAG members suspended in the cluster by running the Get-ClusterNode cmdlet on one of the DAG members.
  • Test service health – use the Test-ServiceHealth cmdlet to verify that all required services are running on each server.
  • Test MAPI connectivity to every database – use the Test-MAPIConnectivity cmdlet to verify that all databases are mounted and accessible.
  • Check the database copy status for DAGs – use the Get-MailboxDatabaseCopyStatus cmdlet to verify that all database copies, copy/replay queues, and content indexes are healthy.
  • Test replication health for DAGs – use the Test-ReplicationHealth cmdlet on each DAG member to verify replication health is good.
  • Check the database activation policy for each Mailbox server – verify that each Mailbox server that is in a DAG has the correct database activation policy for your environment.
  • Check server component status – use Get-ServerComponent to verify that you have not left any servers in maintenance mode.
  • Run Exchange Analyzer to check for best practices compliance.

The Test-ExchangeServerHealth.ps1 script can perform some of the steps above for you. You should also consider running one or more tests from ExRCA.com to verify client connectivity and inbound mail flow are working.

Paul is a Microsoft MVP for Office Servers and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul is a co-author of Office 365 for IT Pros and several other books, and is also a Pluralsight author.
Category: Exchange Server

40 comments

  1. Manfred says:

    Hi Paul,

    Thank you very much for the information and your article!

    I’m currently plan to migrate from Ex2010 to 2016 and have installed Ex2016 RTM with 2x ExNodes (DNS-RoundRobin) and 1x DAG, mainly with your instructions. Again, THX for this great Job! 🙂

    At the past, I’m using for my Ex2010 enviroment PS-Script ‘StartDagServerMaintenance.ps1’ for maintenance start and ‘StopDagServerMaintenance.ps1’ for maintenance stop.

    At some other posts I can found for Ex2013 Server Update to use also post scripts ‘UpdateCas.ps1’ and ‘UpdateConfigFiles.ps1’ scripts.

    … “After each installation of a cumulative update for Exchange 2013, remember to execute both the UpdateCas.ps1 and UpdateConfigFiles.ps1 Windows PowerShell scripts.”

    Here some questions:

    1. Should the script ‘StartDagServerMaintenance.ps1’ no longer be used?
    2. If yes, in which order and combination with your post information?
    3. Should the both ‘UpdateCas’ scripts are also executed after the CU update?
    4. How is the order at a Exchange coexistence (Ex2010/Ex2013/Ex2016) to expand the Scheme/Forrest/Domain and updating each ExNode?

    Excuse my bad English and greetings.

    • 1. Correct, the 2010 maintenance scripts should not be used for 2013/2016.
      2. n/a
      3. I’ve never found it necessary for 2013, and only once have needed to run UpdateCas.ps1 for a 2010 server, many, many years ago.
      4. Schema/AD update is performed once only. Update the highest version of Exchange first.

      • Manfred says:

        Hi Paul,

        THX for your reply.

        Was not really clear for me, but I have now understood. Updated yesterday Exchange 2016 CU1 with no problem, thanks to your guidance.

  2. Elizabeth Barnes says:

    HI Paul, thank you so much for your blog. I’m a new Exchange Server Administrator and your tutorials are invaluable to me. I have 2 questions:

    1) If I run the schema update with the setup.exe /PrepareSchema or the setup.exe /PrepareAD switch and is not required by the CU, will I break anything in by doing this?

    2) At the beginning of the Article you said “Verify that you have documented any customizations to your Exchange server that will need to be re-applied”. If I inherited the Exchange environment, how will I know what has been done? or how can I check this in more detail?

    Thank you in advance.

    • 1) No

      2) Look for things that make your Exchange installation different from a “vanilla” installation, such as customized OWA login pages, integration with third party products that are installed on the Exchange server, that sort of thing.

  3. User2001 says:

    Hi,

    To be able to start setup.exe /prepareschema ou prepareAD from E2016 source files, you do require to use Windows 2012+ server OS. So if it is done from your Schema master AD server, it has to have that OS (Which might not be the case)

    Regards.

  4. Eric says:

    Greetings Paul ,

    I have come across a weird issue while upgrading My exchange 2013 Mail Box server from CU 5 to CU 11 and would dearly need your help. We have 4 2013 servers running in our environment 2 CAS and 2 MBX servers. Have successfully upgraded the first 3 servers to CU 11 without any issues and this is the last server in the organization. Find below the Details of the issue.

    1. The server is a Mailbox server participating in a DAG. The other mailbox server is already upgraded to CU 11 without any issues.

    2. Have tried installing CU 11 upgrade through GUI, intial screen appears and disappears immediately. Exchange setup logs do not have any conclusive logs.

    3. Tried to install through powershell (setup /m…..), the setup failed but with some conclusive error messages. please find below a snippet of the log files.

    [04/20/2016 06:11:31.0475] [0] The following roles have been unpacked: BridgeheadRole ClientAccessRole MailboxRole UnifiedMessagingRole AdminToolsRole

    [04/20/2016 06:11:31.0475] [0] The following datacenter roles are unpacked:

    [04/20/2016 06:11:31.0490] [0] The following roles are installed: BridgeheadRole ClientAccessRole MailboxRole UnifiedMessagingRole AdminToolsRole

    [04/20/2016 06:11:31.0537] [0] [ERROR] Exception has been thrown by the target of an invocation.

    [04/20/2016 06:11:31.0553] [0] [ERROR] Requested value ‘15.0.913.22’ was not found.

    [04/20/2016 06:11:31.0553] [0] CurrentResult SetupLauncherHelper.loadassembly:444: 1

    [04/20/2016 06:11:31.0553] [0] The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the :ExchangeSetupLogs folder.

    [04/20/2016 06:11:31.0553] [0] CurrentResult main.run:235: 1

    [04/20/2016 06:11:31.0553] [0] CurrentResult setupbase.maincore:396: 1

    [04/20/2016 06:11:31.0568] [0] End of Setup

    The things which i derived from the setup logs

    1. It is clear that the steup is failing when the setup tried to find the adminversion ID of the CU5 installer15.0.913.22.

    2. Now, when i see the exchange setup logs of the servers where the installation was successful, it says 15.0.913.22 was present.

    3. Would dearly want to know whether you have faced this issue , or a solution would be nice. even now the setup fails with the same error ” [ERROR] Requested value ‘15.0.913.22’ was not found.”

    Thanks in Advance.
    eric

  5. Darrell Q says:

    Hello Paul,
    We run Exchange 2010, but are moving to 2 new Edge servers in a different DMZ. First off I setup 2 Exchange 2016 (Edge Servers) in the new DMZ, but am not sure if I have to PrepareAD internally to successfully introduce them. Because they are the latest version (and internally its still all 2010) is it necessary to prep AD and the schema first?

  6. Piet v.d. Hout says:

    Can the exchange 2016 scripts: StartDagServerMaintenance.ps1 and StopDagServerMaintenance.ps1 be used for this ?
    If not, why not ?

      • Piet v.d. Hout says:

        Thanks Paul, do you think that the Microsoft exchange team will change the scripts accordingly. I guess these scripts are there for a reason namely to do maintenance on a Dag member server.

        • No, I don’t anticipate them putting any development time into those scripts. They were originally introduced in Exchange 2010, and were recommended for use with 2010. But that had a different maintenance process than 2013/2016. The scripts were changed, but like I said they don’t include all the recommended steps for performing maintenance (same is true for both 2013 and 2016).

  7. skykitchen says:

    Hi Paul. Great Arcticle. I have a problem with the unattenend upgrade – First it shows CU1 but it is definetly CU2 – When i start the GUI installation i see CU2… very strange…

    But the setup hang @ Restoring Services COMPLEDTED and nothing happen anymore. Do you know what can be the Problem ?

    Thanks for help

  8. Allen Stalker says:

    Hey Paul, running into an issue during setup and followed your instructions to a “T.”
    Here is the error that’s getting thrown. It seems to be a permissions issue but I can’t figure out why. I am running from an elevated cmd.exe and I have all the appropriate permissions on the Exchange server.

    Write-ExchangeSetupLog -Info (“An exception ocurred while configuring
    Search Foundation PowerShell Snapin. Exception: ” + $_.Exception.Message);
    }

    }
    ” was run: “System.Exception: Failure cleaning up SearchFoundation Data
    folder. – C:Program FilesMicrosoftExchange
    ServerV15BinSearchCeresHostControllerData – Exception calling “Delete”
    with “2” argument(s): “Access to the path
    ‘Microsoft.ClientResourceView.FlowService.dll’ is denied.”
    at
    Microsoft.Exchange.Configuration.Tasks.Task.WriteError(Exception exception,
    ErrorCategory category, Object target, Boolean reThrow, String helpUrl)
    at
    Microsoft.Exchange.Configuration.Tasks.Task.WriteError(Exception exception,
    ErrorCategory category, Object target)
    at
    Microsoft.Exchange.Management.Deployment.WriteExchangeSetupLog.InternalProcessRe
    cord()

    at Microsoft.Exchange.Configuration.Tasks.Task.b__b()
    at
    Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String
    funcName, Action func, Boolean terminatePipelineIfFailed)”.

    Any help would be greatly appreciated.

  9. Jeff says:

    Hi Paul,

    Great job with your documentation of the install process, thank you!

    I’ve installed 2016 RTM a while ago and accidentally installed .Net 4.6. I don’t recall if it was before or after installing Exchange. So if you’ve already installed .Net 4.6, can you still install CU2? I see that it’s recommended to install Exchange 1st then .Net but that’s not the my current situation.

    If not, how would you recommend proceeding? Thank you.

  10. Alan says:

    Hi,
    very useful info. Do you recommend performing this on live servers? I usually do regular Widows updates during scheduled downtime (i.e the wekend), will this procedure cause issues to end users or does it still need downtime? (for a system with multiple CAS servers and a DAG with 3+ copies of each database).

  11. Alice Goodman says:

    Hi Paul – Great information as always. I have a question about removing the server from maintenance mode. What is the difference between Set-ServerComponentState EX2016SRV1 –Component ServerWideOffline –State Active –Requester Maintenance and the subsequent same command but for HubTransport? Doesthe ServerWideOffline also take care of the HubTransport?

  12. Mircea Sandu says:

    Hello all,

    I had two Exchange 2016 servers in my infrastructure and I started to upgrade them to Exchange 2016 CU3.
    One of the servers has been updated successfully, but the second one ran into a problem at ‘Client Access Front End service’ stage.
    You can see the CMD output below.

    F:>Setup /m:upgrade /IAcceptExchangeServerLicenseTerms

    Welcome to Microsoft Exchange Server 2016 Cumulative Update 3 Unattended Setup

    Copying Files…
    File copy complete. Setup will now collect additional information needed for installation.

    Languages
    Management tools
    Mailbox role: Transport service
    Mailbox role: Client Access service
    Mailbox role: Unified Messaging service
    Mailbox role: Mailbox service
    Mailbox role: Front End Transport service
    Mailbox role: Client Access Front End service

    Performing Microsoft Exchange Server Prerequisite Check

    Configuring Prerequisites COMPLETED
    Prerequisite Analysis COMPLETED

    Configuring Microsoft Exchange Server

    Language Files COMPLETED
    Restoring Services COMPLETED
    Language Configuration COMPLETED
    Exchange Management Tools COMPLETED
    Mailbox role: Transport service COMPLETED
    Mailbox role: Client Access service COMPLETED
    Mailbox role: Unified Messaging service COMPLETED
    Mailbox role: Mailbox service COMPLETED
    Mailbox role: Front End Transport service COMPLETED
    Mailbox role: Client Access Front End service FAILED

    The following error was generated when “$error.Clear();
    .
    “$RoleInstallPathScriptsUpdate-AppPoolManagedFrameworkVersion.ps1″ -AppPoolName:”MSExchangeServicesAppPool”
    -Version:”v4.0″;
    get-WebServicesVirtualDirectory -server $RoleFqdnOrName | set-WebServicesVirtualDirectory
    -windowsAuthentication:$true -WSSecurityAuthentication:$true -OAuthAuthentication:$true
    ” was run:
    “System.Runtime.InteropServices.COMException (0x800700B7): Filename: \?C:Program FilesMicrosoftExchange
    ServerV15FrontEndHttpProxyEWSweb.config
    Line number: 8
    Error: Cannot add duplicate collection entry of type
    ‘add’ with unique key attribute ‘key’ set to ‘HttpProxy.ProtocolType’

    at
    Microsoft.Web.Administration.Interop.IAppHostAdminManager.GetAdminSection(String bstrSectionName, String bstrPath)
    at
    Microsoft.Web.Administration.Configuration.GetSectionInternal(ConfigurationSection section, String sectionPath, String
    locationPath)
    at
    Microsoft.Exchange.Management.SystemConfigurationTasks.ExchangeServiceVDirHelper.EwsAutodiscMWA.EnableOrDisableCafeEndpo
    int(Configuration
    configuration, String endpointName, Boolean enableEndpoint)
    at
    Microsoft.Exchange.Management.SystemConfigurationTasks.ExchangeServiceVDirHelper.EwsAutodiscMWA.OnSetManageWCFEndpoints(
    Task
    task, EndpointProtocol protocol, Boolean enableWSSecurity, ExchangeVirtualDirectory adVirtualDirectory)
    at
    Microsoft.Exchange.Management.SystemConfigurationTasks.SetWebServicesVirtualDirectory.InternalProcessRecord()
    at
    Microsoft.Exchange.Configuration.Tasks.Task.b__b()
    at
    Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean
    terminatePipelineIfFailed)
    at Microsoft.Exchange.Configuration.Tasks.Task.ProcessTaskStage(TaskStage taskStage,
    Action initFunc, Action mainFunc, Action completeFunc)
    at Microsoft.Exchange.Configuration.Tasks.Task.ProcessRecord()

    at System.Management.Automation.CommandProcessor.ProcessRecord()”.

    The Exchange Server setup operation didn’t complete. More details can be found in ExchangeSetup.log located in the
    :ExchangeSetupLogs folder.

    F:>

    At this stage I cannot do anything with the server, and /owa is not working anymore. It seems that I have a corrupt installation.
    Do you have any ideas how can I solve the error and ran the installation task again?

    Regards,
    Mircea

  13. mhody says:

    Hi Paul,

    Im experiencing a weird issue. Its really not giving me a more descriptive note with regards to the problem. It only says “Performance counter names and help text failed to unload. Lodctr exited with error code ‘2001’.” ive been stuck in language installing ever since. i tried doing the lodctr /r and other things from google but with no luck.

  14. Jeff says:

    Hi Paul,

    When referencing “If the server is a DAG member, run the following commands. If your server is not a DAG member, skip to the command for setting ServerWideOffline.”, I do not see the command for setting ServerWideOffline. Could you provide that for me?

    Thank you.

  15. ThatGuy says:

    Paul, I am updating from EXCH 2016 RTM (15.1.225.42) to EXCH 2016 RU6. We are in O365 now and only keep this server around to allow for the receive connector to send email anonymously (UPSs, Storage devices). Is there anything that we need to be concerned with in 0365?

    BTW, great article. I’m never let down when reading your posted content.

  16. Samy says:

    Hello Paul,

    Thanks for the wonderful article. I’m facing an issue when trying to run the upgrade setup from Exchange Powershell I received the following error message and setup fails

    Please help

    Performing Microsoft Exchange Server Prerequisite Check

    Configuring Prerequisites COMPLETED
    Prerequisite Analysis FAILED

    Setup can’t continue with the upgrade because the powershell (3564) has open files. Close the process, and then restart
    Setup.

    • Exchange setup isn’t run from PowerShell. Use a CMD prompt. You will need to close down any PowerShell sessions that have Exchange files locked as well (e.g. close all Exchange Management Shell windows).

Leave a Reply

Your email address will not be published. Required fields are marked *