In this video I demonstrate the end user experience during different failure scenarios in a highly available Exchange Server 2010 environment.

The demonstration involves:

  • Two connected mailbox users sending email back and forth
  • An Exchange 2010 CAS Array consisting of two Client Access servers
  • A Database Availability Group consisting of two Mailbox servers

If you can’t see the video click here to view it on Youtube.

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. ZAHOOR

    Dear Sir,
    I am facing Problem when i try to join the remote site Exchagne server to DAG
    Summary: 1 item(s). 0 succeeded, 1 failed.
    Elapsed time: 00:26:06

    SAJDDREX01
    Failed

    Error:
    A database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server. [Server: SAJDHQEX01.nbk.com.sa]

    A database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
    Click here for help… http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.3.123.3&t=exchgf1&e=ms.exch.err.ExC9C315

    Warning:
    Network name ‘SAEXDAG1’ is not online. Please check that the IP address configuration for the database availability group is correct.

    Warning:
    The operation wasn’t successful because an error was encountered. You may find more details in log file “C:ExchangeSetupLogsDagTasksdagtask_2013-04-01_09-38-20.844_add-databaseavailabiltygroupserver.log”.

    Exchange Management Shell command attempted:
    Add-DatabaseAvailabilityGroupServer -MailboxServer ‘SAJDDREX01’ -Identity ‘SAEXDAG1’

    Elapsed Time: 00:26:07

    This is driving me crazy.
    any help will be appreciated.

    1. ZAHOOR

      here is the latest error:
      Summary: 1 item(s). 0 succeeded, 1 failed.
      Elapsed time: 00:25:45

      SAJDDREX01
      Failed

      Error:
      A database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server. [Server: SAJDHQEX01.x.x.com]

      A database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
      Click here for help… http://technet.microsoft.com/en-US/library/ms.exch.err.default(EXCHG.141).aspx?v=14.3.123.3&t=exchgf1&e=ms.exch.err.ExC9C315

      Warning:
      The operation wasn’t successful because an error was encountered. You may find more details in log file “C:ExchangeSetupLogsDagTasksdagtask_2013-04-01_10-09-30.737_add-databaseavailabiltygroupserver.log”.

      Exchange Management Shell command attempted:
      Add-DatabaseAvailabilityGroupServer -MailboxServer ‘SAJDDREX01’ -Identity ‘SAEXDAG1’

      Elapsed Time: 00:25:45

      below is the last part of the log file:
      ==========================
      [2013-04-01T10:09:35] Updated Progress ‘Adding server ‘SAJDDREX01′ to the cluster.’ 8%.
      [2013-04-01T10:09:35] Working
      [2013-04-01T10:35:16] The following log entry comes from a different process that’s running on machine ‘SAJDHQEX01.nbk.com.sa’. BEGIN
      [2013-04-01T10:35:16] [2013-04-01T10:09:35] Opening a local AmCluster handle.
      [2013-04-01T10:09:35] Updated Progress ‘Adding server ‘sajddrex01’ to database availability group ‘SAEXDAG1′.’ 2%.
      [2013-04-01T10:09:35] Working
      [2013-04-01T10:09:35] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SAJDDREX01, dwStatus = 0x0 )
      [2013-04-01T10:35:16] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseContinue, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 12, szObjectName = SAJDDREX01, dwStatus = 0x800713bb )
      [2013-04-01T10:35:16] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 12, szObjectName = SAJDDREX01, dwStatus = 0x800713bb )
      [2013-04-01T10:35:16] Found a matching exception: Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException: A database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
      [2013-04-01T10:35:16] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SAJDDREX01, dwStatus = 0x0 )
      [2013-04-01T10:35:16] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )

      [2013-04-01T10:35:16] The preceding log entry comes from a different process running on computer ‘SAJDHQEX01.nbk.com.sa’. END
      [2013-04-01T10:35:16] The operation wasn’t successful because an error was encountered. You may find more details in log file “C:ExchangeSetupLogsDagTasksdagtask_2013-04-01_10-09-30.737_add-databaseavailabiltygroupserver.log”.
      [2013-04-01T10:35:16] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A database availability group administrative operation failed. Error: The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server. —> Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException: A database availability group administrative operation failed. Error: Windows Failover Clustering timed out while trying to validate server ‘SAJDDREX01’. If this is in a disjoint DNS namespace, the DNS suffixes for all servers in the database availability group must be present on every server.
      at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
      at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
      — End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.DagTaskValidateNodeTimedOutException) —
      at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
      at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
      at Microsoft.Exchange.Cluster.ReplayService.ReplayRpcServer.c__DisplayClass34.b__33()
      at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
      — End of stack trace on server (SAJDHQEX01.x.x.com) —
      at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
      at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperationDbName(AmServerName serverName, String databaseName, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
      at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunRpcOperation(AmServerName serverName, Nullable`1 dbGuid, Int32 timeoutMs, IHaRpcExceptionWrapper rpcExceptionWrapperInstance, InternalRpcOperation rpcOperation)
      at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
      at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
      [2013-04-01T10:35:16] Updated Progress ‘Done!’ 100%.
      [2013-04-01T10:35:16] COMPLETED
      add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().

        1. ZAHOOR

          Well it was working fiine lately, We switched to link between the sites to other provider and it worked fine. The FQDN name have been manually updated wiht X.X to hide the domain name.
          thanks.

          Now Quick Question sir,

          I have exchange 2010 SP3 on both sites main and DR wih CAS,MBand HT roles in one box only on each site.
          Is it possible to have High Availability with DR site bewteen the two Exchange 2010 SP3 boxes.
          and How?
          thanks in advance. and appreciate your replys.

          Another question is that the DATA base copy on the DAG meber is showing status Resynchronising .

          checkiing the Health iget the below error in one item

          [PS] C:\>Test-ReplicationHealth -Identity sajddrex01

          Server Check Result Error
          —— —– —— —–
          SAJDDREX01 ClusterService Passed
          SAJDDREX01 ReplayService Passed
          SAJDDREX01 ActiveManager Passed
          SAJDDREX01 TasksRpcListener Passed
          SAJDDREX01 TcpListener Passed
          SAJDDREX01 ServerLocatorService Passed
          SAJDDREX01 DagMembersUp Passed
          SAJDDREX01 ClusterNetwork Passed
          SAJDDREX01 QuorumGroup Passed
          SAJDDREX01 FileShareQuorum Passed
          SAJDDREX01 DBCopySuspended Passed
          SAJDDREX01 DBCopyFailed Passed
          SAJDDREX01 DBInitializing *FAILED* Failures:…
          SAJDDREX01 DBDisconnected Passed
          SAJDDREX01 DBLogCopyKeepingUp Passed
          SAJDDREX01 DBLogReplayKeepingUp Passed

          ZAHOOR

Leave a Reply