When an Exchange Server 2010 Mailbox server that is a member of a Database Availability Group has failed, part of the recovery process is to remove it from DAG membership.

In some scenarios this process may result in an error “A quorum of cluster nodes was not present to form a cluster“.  The full error text is below.

[PS] C:\>Remove-DatabaseAvailabilityGroupServer -Identity DAG -MailboxServer EX2

Confirm
Are you sure you want to perform this action?
Removing Mailbox server "EX2" from database availability group "dag".
[Y] Yes  [A] Yes to All  [N] No  [L] No to All  [?] Help (default is "Y"): y
WARNING: The operation wasn't successful because an error was encountered. You may find more details in log file
"C:ExchangeSetupLogsDagTasksdagtask_2010-11-25_03-48-09.814_remove-databaseavailabiltygroupserver.log".
There was a problem changing the quorum model for database availability group dag. Error: An Active Manager operation f
ailed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"SetClusterQuorumResource() f
ailed with 0x1725. Error: A quorum of cluster nodes was not present to form a cluster"' failed..
    + CategoryInfo          : InvalidArgument: (:) [Remove-DatabaseAvailabilityGroupServer], DagTaskProblemChangingQuo
   rumException
    + FullyQualifiedErrorId : 80D96894,Microsoft.Exchange.Management.SystemConfigurationTasks.RemoveDatabaseAvailabili
   tyGroupServer

To resolve this issue use the -ConfigurationOnly switch instead to remove the failed Mailbox server from the Exchange 2010 DAG.  In this example server EX2 is being removed.

[PS] C:\>Remove-DatabaseAvailabilityGroupServer -Identity DAG -MailboxServer EX2 -ConfigurationOnly

Next, evict the failed node from the Windows Failover Cluster.

Note: you need to import the Failover Cluster module into your Exchange Management Shell session to perform this task.

[PS] C:\>Import-Module FailoverClusters
[PS] C:\>Get-ClusterNode EX2 | Remove-ClusterNode -Force

The failed server has now been removed from the Exchange 2010 DAG and the Windows Failover Cluster.

[adrotate banner=”49″]

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Jong Ku Yu

    Get-Cluster DAG | Remove-Cluster -Force -CleanupAD
    You must remove the cluster through the command.

  2. Oneil Matlock

    Great article! We did a force quorum on our two Node DAG and created a split brain scenario. We’ve removed secondary node from cluster manager, disabled its NIC, but get error when trying to run the remove commands to remove it from the DAG, but errors saying that there is a still a database copy present.

    Any thoughts about how to resolve this?

  3. AMF

    Error:

    [PS] C:\Windows\system32>Set-DatabaseAvailabilityGroup -Identity DAG -WitnessDirectory c:\DAGWitness -WitnessServer Newwitnessserver.domain.local

    There was a problem changing the quorum model for database availability group EXGDAG. Error: An error occurred while
    attempting a cluster operation. Error: Cluster API failed: “DeleteClusterResource() failed with 0x139f. Error: The
    group or resource is not in the correct state to perform the requested operation”
    + CategoryInfo : InvalidArgument: (:) [Set-DatabaseAvailabilityGroup], DagTaskProblemChangingQuorumExcept
    ion
    + FullyQualifiedErrorId : [Server=EX01,RequestId=0303ae8a-6df1-47f9-8fd2-1390763b2eb2,TimeStamp=12/19/2017 4:4
    4:02 AM] [FailureCategory=Cmdlet-DagTaskProblemChangingQuorumException] 56C85867,Microsoft.Exchange.Management.Sys
    temConfigurationTasks.SetDatabaseAvailabilityGroup
    + PSComputerName : EX01.domain.local

        1. AMF

          Hi

          I checked my quorum resources state, it has already quorum resources as File Share Witness, the above blog talking about when there is no quorum resources present in the DAG.

      1. AMF

        The above blog speaks about Quorum resources, In my exchange already the Quorum resources showing as File Share Witness

        1. Avatar photo
          Paul Cunningham

          If that article hasn’t helped you then I suggest you open a support case with Microsoft so that they can look at your environment directly and provide a solution.

          1. AMF

            Hi Paul,

            Thanks for your time.

          2. AMF

            Hi Paul,

            The issue has been resolved by manually deleted the File share Witness from FCM.

            Steps Done:

            1. Set No witness (Now witness will be “None”)
            2.Check there is any witness shows in FCM. If there is witness under cluster resource, Right click and remove.
            3.Set-DatabaseAvailabilityGroup -Identity DAG -WitnessServer Witnessserver.domain.local -WitnessDirectory C:\DAGWitness.

            Now My DAG shows

            WitnessServer : Witnessserver.domain.local
            WitnessDirectory : C:\DAGWitness
            AlternateWitnessServer :
            AlternateWitnessDirectory :
            WitnessShareInUse : Primary (This was showing as Invalidconfiguration)
            DxStoreWitnessServers :

          3. AMF

            No downtime is required.

  4. AMF

    Set-Databaseavailabilitygroup -Identity DAG -Witnessserver NEWWITNESSSERVER.domain.local -Witnesssdirectory C:\DAGWITNESS

    In witness server I have added exchange trusted subsystem to local administrator Group. before run the above cmdlet.

  5. AMF

    Hi,

    I am getting “WitnessShareInuse : InvalidConfiguration” when I get the DAG status. What could be the reason and I tried “Set-DatabaseAvailabilitygroup -Identity -WitnessServer -WitnessDirectory ” to reconfigure the witnessserver. and unable to change.

    But there is a difference in witness type between GUI fail over cluster and cmdlet witness type.

    In GUI witness type is showing as “file share witness”, In cmdlet quorum type is showing as “nodeandfilesharemajority”

    1. Avatar photo
      Paul Cunningham

      What does “unable to change” mean? Do you get an error message?

      1. AMF

        WARNING: The witness server and directory currently in use by database availability group ‘DAG’ doesn’t match the
        configured primary or alternate witness server. This may be due to Active Directory replication latency. If this
        condition persists, please use the Set-DatabaseAvailabilityGroup cmdlet to correct the configuration.
        WitnessServer : Witnessserver.domain.local
        WitnessDirectory : C:\DAGWitness
        AlternateWitnessServer :
        AlternateWitnessDirectory :
        WitnessShareInUse : InvalidConfiguration
        DxStoreWitnessServers :

        What does “unable to change” mean? (Unable to change the witness server to another Server)

        1. AMF

          Site : 2 Sites
          Servers : PRIMARY 3 and DR 1
          DAG Members Total : 4
          Witness : Placed in DR (Is this correct?)
          DAC : Enabled

        2. Avatar photo
          Paul Cunningham

          Why are you unable to change it? What command are you running? What error message do you get?

  6. Najeeb

    Get-ClusterNode PRD01

    Name State
    ================================
    PRD01 Down

  7. Najeeb

    I am getting below error while trying to remove a corrupt node from cluster.
    The node is crashed and not able to make it up.
    PS C:\> Get-ClusterNode PRD01|Remove-ClusterNode -Force
    Remove-ClusterNode : An error occurred evicting node ‘PRD01’.
    The cluster request is not valid for this object
    At line:1 char:45
    + Get-ClusterNode PRD01|Remove-ClusterNode <<<< -Force
    + CategoryInfo : NotSpecified: (:) [Remove-ClusterNode], ClusterCmdletException
    + FullyQualifiedErrorId : Remove-ClusterNode,Microsoft.FailoverClusters.PowerShell.RemoveClusterNodeCommand

    1. Avatar photo
      Paul Cunningham

      No idea. I assume “Get-ClusterNode PRD01” returns a good result?

      If you’re stuck on that step I suggest you open a support case with Microsoft for further assistance.

  8. Nicola

    Hi, i have followed the procedure in the previus one article but my cluster dowsn’t go up. if i try to use this procedure i have this error:
    [PS] C:\Windows\system32>Get-ClusterNode EXCH-MBX03 | Remove-ClusterNode -Force
    Get-ClusterNode : Failed to retrieve the node ‘EXCH-MBX03’ from the cluster ‘DAG01’.
    An error occurred opening node ‘EXCH-MBX03’.
    The cluster node was not found
    At line:1 char:16
    + Get-ClusterNode <<<< EXCH-MBX03 | Remove-ClusterNode -Force
    + CategoryInfo : NotSpecified: (:) [Get-ClusterNode], ClusterCmdletException
    + FullyQualifiedErrorId : Get-ClusterNode,Microsoft.FailoverClusters.PowerShell.GetNodeCommand

    How can i do?
    Thanks

    1. Avatar photo
      Paul Cunningham

      It’s saying that cluster node was not found. Are you sure it’s the right name?

  9. Demeter Szabolcs

    Grate! Worked

  10. Trond Jenssen

    Thanks!

    Helped me a lot. Had to remove a dead Exchange 2013 server from DAG, and remove-clusternode was the solution

  11. Ron Steurer

    This definitely helped me out in crisis when a server was added to a DAG but the IP still resolved to two servers so there was a conflict. Thanks Paul!!

  12. Zargham Haider

    I am unable to remove a Member from DAG. it shows below error:

    An Active Manager operation failed with a transient error. Please retry the operation. Error: A transient error occurred during discovery of the database availability group topology. Error: The fully qualified domain name for node ‘EX2’ could not be found.

    will you please help me.

    Thanks

    1. Avatar photo
      Paul Cunningham

      “The fully qualified domain name for node ‘EX2’ could not be found”

      Is your server named EX2?

  13. Alexey

    Great post! Thank you so much! Same situation with Exchnage 2013. It helps me a lot!

  14. Nelson Ndigi

    This truely sorted me out. Thanks man!

  15. Imran

    Thanks so much Paul , your always helpful for me .

    God blesses you,

    imran safi
    from afghanistan

  16. Rune Goksør

    I ran these commands on a server that was still in the environment but halfway uninstalled from before. I was unable to uninstall Exchange 2013 from this “orphan” server after running the commands in this article because the server seemed to think that it was still a part of the DAG. MS helped me out with running this command first on one of the servers that is left in the DAG:

    cluster node ORPHAN-MBX /forcecleanup

    After that I could uninstall Exchange from this “orphan” server, and thus clean up AD properly.

  17. Erik Santos

    Paul.. great help!
    Thanks

  18. Baffajo Beita

    Always on point! your articles always work!

  19. Mohammad

    Hi Paul and Thanks Again
    In my cluster and DAG everything was OK
    situation is :
    Two Ex 2010 SP1 in the DAG (mailbox01 and Mailbox02) and Two SP3 Added to DAG (mailbox1 and Mailbox2)
    Now I want to remove the previous ones

    but this error show up again and again while removing mailbox01 with no database copy on it

    anyway i used first cmdlet
    [PS] C:\>Remove-DatabaseAvailabilityGroupServer -Identity DAG -MailboxServer Mailbox0X2 -ConfigurationOnly

    and it is removed from DAG but second ones

    [PS] C:Windowssystem32>Get-ClusterNetwork Maibox01 | Remove-ClusterNode -Force
    Get-ClusterNetwork : Failed to retrieve the network ‘Maibox01’ from the cluster ‘DAG’.
    An error occurred opening network ‘Maibox01’.
    The cluster network was not found
    At line:1 char:19

    do not work, give me following error and the server is not removed from windows failover cluster

    1. Mohammad

      oh sorry
      i had no patience 🙂
      just needed some time maybe to clear the configurations from the cluster and then it worked
      i think you should mention that some time is needed between running those commands 😉

  20. Derek

    You have no idea how this helped. I couldn’t remove the member server from DAG because Windows Cluster Service would not start. Thanks!

  21. BB

    I have an two exchange 2010 sp2 multi-role servers. In EMC it shows the replication is up to date with no queue built up. However, the DR server shows in the event viewer.

    “passive/active copy is not UP according to clustering”

    In cluster failover management the node shows as offline and so does the NIC. Both are online and the replication seems to be fine. The primary dag server shows all is well. Can I use the above to remove and readd the recovery exchange server? Do I need to remove all DB copies first or can I just execute the commands above and it will take care of it?

  22. wasim

    Hi

    I am trying to import Import-Module FailoverClusters but msem shell is not getting this command please help

    Thanks

    1. Avatar photo
      Paul Cunningham

      what is “msem”?

      The command is a PowerShell command and should work on a Windows 2008 R2 server that is an Exchange 2010 DAG member.

      1. wasim

        Hi
        msem is microsoft excange management shell, Actually i was Using Pssession from local machine
        Thank you so much it works from Dag member.

  23. Kwang

    Great post! It works like a champ…!

Leave a Reply