When activating a database copy in an Exchange Server 2010 Database Availability Group it may fail with an error message that catalog index files are in a failed state.

exchange-2010-cannot-activate-database

——————————————————–
Microsoft Exchange Error
——————————————————–
Cannot activate database copy ‘Activate Database Copy…’.

Activate Database Copy…
Failed
Error:
An Active Manager operation failed. Error The database action failed. Error: An error occurred while trying to validate the specified database copy for possible activation. Error: Database copy ‘MB-HO-03’ on server ‘HO-EX2010-MB2.exchangeserverpro.net’ has content index catalog files in the following state: ‘Failed’.. [Database: MB-HO-03, Server: HO-EX2010-MB1.exchangeserverpro.net]

An Active Manager operation failed. Error An error occurred while trying to validate the specified database copy for possible activation. Error: Database copy ‘MB-HO-03’ on server ‘HO-EX2010-MB2.exchangeserverpro.net’ has content index catalog files in the following state: ‘Failed’..

When you view the copy status of the mailbox database the content index is in a failed state.

[PS] C:>Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

Name              : MB-HO-01HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-04HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-03HO-EX2010-MB1
ContentIndexState : Failed

Name              : MB-HO-02HO-EX2010-MB1
ContentIndexState : Healthy

There are two likely causes of this issue.

  1. The content index has failed due some error or fault occurring
  2. Indexing has been disabled for that database (a common configuration for databases hosting journal mailboxes)

Fixing Failed Content Indexes

To resolve the issue update the content index on the server on which it has failed.

[PS] C:\>Update-MailboxDatabaseCopy "MB-HO-03HO-EX2010-MB1" -CatalogOnly

The content index should now be in a healthy state.

[PS] C:>Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

Name              : MB-HO-01HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-04HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-03HO-EX2010-MB1
ContentIndexState : Healthy

Name              : MB-HO-02HO-EX2010-MB1
ContentIndexState : Healthy

After the content index is healthy you can attempt to activate the database copy again.

Performing Database Switchovers When Indexing is Disabled

If indexing has been intentionally disabled you can still perform a database switchover using the Exchange Management Shell, by running Move-ActiveMailboxDatabase with the -SkipClientExperienceChecks switch.

[PS] C:>Move-ActiveMailboxDatabase MB-HO-03 -ActivateOnServer HO-EX2010-MB2 -SkipClientExperienceChecks

Assuming no other issues exist with the database the move should complete successfully.

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Lang Khach

    Dear Paul

    I have a mailbox database named: Mailbox Database 1174790013- HO009
    Previously, because of full disk, this mailbox database failed, Content Index State reported failed
    I’ve done enough to get it to mount again as unsuccessful.

    Excuse me. I can execute orders

    Update-MailboxDatabaseCopy “MB-HO-03HO-EX2010-MB1” -CatalogOnly

    so that it can go back to normal?

    Thanks

  2. Ali Zain

    Hi Paul,
    We are facing issues in Out Off office auto replay on random user in our organization, On some users its working fine but on some users its behavior quite abnormal some times its not implementing and on the mean time if its works its not reverting. Kindly advice.

  3. sadegh

    Hello All,
    Update-MailboxDatabaseCopy “10GB-mailboxes” -CatalogOnly command in Exchange 2016 I got Error :

    Database ’10GB-mailboxes\*’ has only one copy. This task is supported only for databases that have more than one copy.

  4. Sarfraz Aslam

    Hi Paul,

    Your articles always helpful for me. Thank you.

    I need your assistance here, while executing the command ‘Update-MailboxDatabaseCopy “DB111Server” -CatalogOnly

    I am getting following error
    “A source-side operation failed. Error An error occurred while performing the seed operation. Error: An error occurred while updating the search catalog files from server “ActiveServer” to PassiveServer”. Error: can’t pause the search catalog. Error: Microsoft.Exchange.Search.Common.FteCatalogNotFoundException: SearchCatalog.PauseIndexing failed, error 0x80043629 —> System.ComponentModel.Win32Exception: Unknown error (0x80043629)
    —-End of Inner exception stack trace —-
    at Microsoft.Exchange.Cluster.Replay.SearchCatalogProxy.pauseIndexing()
    at Microsoft.Exchange.Cluster.Replay.CiFilesSeedeInstance.(SeedThreadProcInternal)b___3(Object , EventArgs )
    at Microsoft.Exchange.Cluster.Replay.CiFileSeederInstance.REtryCiOperation (Eventhandler evt) .. [Database: DB111, Server: PassiveServer
    +CategoryInfo : InvalidOperation: (:) [Update-MailboxDatabaseCopy], CiSeederGenericException
    +FullyQualifiedErrorId : FFC07141,Microsoft.Exchange.Management.SystemConfigurationTasks.UpdateDatabaseCopy

    Quick reply will be highly appreciated.

    Thanks and Regards
    Sarfraz

  5. Paul Huxham

    Thanks Paul; this was a huge help on a Monday morning!

  6. Andrew Moss

    Paul,

    Thank you very much. This worked well for me. I saw other websites with solutions, but your website is the one the I trust.

    Andrew

  7. Chretien

    Hey Paul,

    What to do when content index of active and passive mailbox are in crawing state?

    It’s very urgent please?

    Thanks to read you

    Merci

  8. Justus Grunow

    Seriously, what would I do without you? …Buy support from Microsoft, probably. THANKS!

  9. Alan Morris

    Happy New Year to you Paul.

    Excellent article. Worked like a charm.

    I have one other problem you may be able to help with.

    I have moved all of the mailboxes from the offending database but it it still showing that it has a 50Gb .edb file. Is it just a matter of deleting the database seeing as it is probably not going to be used again?
    Any thoughts or help would be greatly appreciated.

    1. Avatar photo
      Paul Cunningham

      Moving mailboxes out of a database doesn’t shrink the database file itself. It can only ever grow, not shrink.

      If you no longer want the empty database you can remove it via the Exchange console.

  10. Ian

    First, thanks for the article! Your website is a great help.

    Today I discovered that more than half of my (substantial quantity of) database copies showed “failed” for the ContentIndexState. I was able to resolve the issue with the Update-MailboxDatabaseCopy -CatalogOnly command (thanks to talishka for the where ‘Failed’ pipeline command) but I’m a little confused as why this worked. For some of the databases, all copies of the ContentIndex showed failed including the Mounted active database. My understanding is that the update-mailboxdatabasecopy command copies (“seeds”) information from the active database to the non-active copies. If even the active database is failed, how can this information be properly copied and resolve the issue? Shouldn’t I have had to use the ResetSearchIndex.ps1 script to re-index the Active database copy?

    Part two: The large list of “failed” contentindexes was quite alarming (presumably this would have prevented automatic takeover in case of server failure!) and I have no idea how long there have been failed contentindexes (have they been building up over time? or did they all suddenly appear?). How can I determine when the failures occurred (particular event IDs in logs?) and what caused them? Is there any way to prevent them? I see you said above in comments you’re not sure what causes them but are there best practices that might reduce their occurrence?

    1. Avatar photo
      Paul Cunningham

      1) Yes an index reset/rebuild would be necessary if all copies of the database had failed indexes.

      2) Unhealthy content indexes will not prevent failover. They are a factor in the best copy selection process, so a healthy index would be preferred over an unhealthy one, but if they’re all unhealthy one of them should mount (assuming no other issues prevent it).

      Monitor your server/database health to avoid surprises like this.

  11. lz1fb7

    STUN-Wah advice Paul, thanks very much, I used your Update command in EMS and the catalog reseeded perfectly on the sick database, all well again, I appreciate your expertise.

  12. Arnel

    Hi Sir,

    Please help me, All my database copy in a DAG has a failedandsuspend index content status and when I run Update-MailboxDatabaseCopy -Identity “DB 01MBX101” -catalogonly I get this errors:

    WARNING: Seeding of content index catalog for database ‘DB 01’ failed. Please verify that the Microsoft Search
    (Exchange) and the Host Controller service for Exchange services are running and try the operation again. Error: The
    seeding operation failed. Error: An error occurred while performing the seed operation. Error: An error occurred while
    updating the search catalog files from server ‘MBX1’ to ‘MBX2’. Error: A
    transient exception from Exchange Search was encountered. Error: -1..

    Hope you can help me.

    Thanks,
    Arnel

  13. jc

    Hi Sir Paul,

    I really need your help when i try to activate or switch my database copy in the DAG I get the following error:

    The Microsoft Exchange Replication service does not appear to be running on “xxxmb101”. Make sure that the server is operating, and that the services can be queried remotely.

    Please help me.

    Thanks Sir Paul,
    Happy Holidays!

  14. Sean

    It helps me. Thanks.

  15. RobK

    Incredible to see how MS does nothing to address this little problem – the work on the overnext problem … erm… Exchange version seems to be more important than to auto-fix problems that are obvious; be it in the ource or by auto-healing.

    Sad to see that a company which claims to be powerful is so weak – but to be honest, I did not expect more…

    Rob

  16. Admin

    Excellent. reseeding the Catalog only worked like a charm. Thank you!!

  17. Dev

    WORKED LIKE A CHARM…BIG THANKS.

  18. ppp

    Thanks!!!

  19. Jeff

    Hi Paul,

    Have you seen any issues with Exchange 2010 SP3 in EMC >> Server Config >> Mailbox. Under Database Copies > Copy Status not updating correctly with a refresh.

    I end up logging into another Exchange server just to see the current correct copy status.

    Thanks,
    Jeff

  20. Upendra

    Hi Paul,

    Really it is very useful.

    I have one question , Can you explain how does replication works in DAG, if suppose if any log file got corrupted in passive copy in DAG , how identify and how will resolve it.

    Highly appreciated your quick response.

    Thanks,
    Upendra.

    1. Avatar photo
      Paul Cunningham

      There is a lot of great information on TechNet that explains how Exchange database replication works.

  21. Praveen

    Hi Paul

    I have content index state on one DB as crawling for both the active and the passive noe. Any help would be appreciated

  22. Mauro Rita

    Thank you !

    Have a wonderful weekend !

    Greetings from Lisboa !

  23. johnres

    Well, you can use Long Path Tool, it works good for such problems……….

  24. Mohammad

    Hi Paul
    It’s one of the best Website which I used forever.
    Thank you for your short and good guides to resolve problems

    Best Regards

  25. Ciprian Bacanu

    Hi Paul,
    thanks for this post. I wonder what’s the solution when there’s only one server i.e. only one database.
    I noticed index state was failed, I reseed search index but perf mon crawler is telling me there is one mailbox to be seeded. This lasts from the 1st of November. Also, I have error id 4087:
    Failed to move active database ‘Mailbox Database DB1’ from server ‘ExSrv1’. Move comment: Database failover initiated by Responder ‘SearchLocalCopyStatusEscalate’ target resource ‘Mailbox Database DB1’

    Error: An Active Manager operation failed. Error: The database action failed. Error: This operation is not supported on standalone mailbox servers.

    Could you tell me what’s going on? Will crawling process stop? If not, how could I find the mailbox?

    Best regards,
    Ciprian

    1. Ciprian Bacanu

      Hi Paul, Happy New Year! Finally solved issue, please delete this comment.

      Best regards,

      Ciprian

      1. Tarek

        Hi Ciprian – can you tell me what the fix was?

        Thanks in advance.

        1. Ciprian Bacanu

          Tarek, hello,
          Unfortunately I didn’t find out what was the mailbox that caused the trouble so I started out and created a new database where I moved some of mailboxes (I took reverse the size they had). No joy after rendex. I created a new one and moved another ones. At the third one I noticed that after reindex databases were healthy. So, I moved back users mailboxes and delete the temporary databases.
          I don’t have a lot of users and a had also a lot of space available – take care that moving a mailbox to another database does not free the space.
          I managed to find out the maibox looking the migration logs.

          Ciprian

  26. Paul taylor

    I have opened up a powershell command window as administrator, I have then typed get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate and I get the following error

    get-mailboxserver : The term ‘get-mailboxserver’ is not recognized as the name of a cmdlet, function, script file, or
    operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try
    again.
    At line:1 char:1
    + get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate
    + ~~~~~~~~~~~~~~~~~
    + CategoryInfo : ObjectNotFound: (get-mailboxserver:String) [], CommandNotFoundException
    + FullyQualifiedErrorId : CommandNotFoundException

    I am just learning Powershell so why wont it work?

    1. Avatar photo
      Paul Cunningham

      Open the Exchange Management Shell (which is PowerShell with the Exchange snapin automatically loaded for you).

  27. Sule Yaya

    Hi Paul,

    Thanks for always being a source of help & support for all of us.

    Sulex

  28. Jamsheed Kadooran

    HI Paul

    it’s worked, thanks for your gr8 support.

    BR!

  29. Lori

    Thank you!

  30. John Azevedo

    Worked like a charm!

    Thanks!

  31. Keith Risinger

    Worked like a charm. Thank you Paul for this and other helpful information you provide.

  32. Monty St.

    Thanks for that. Concise and very helpful.

  33. Ian Salgado

    Hi,

    I have a Database which has a Catalog Index = FAILED

    The database IS NOT part of a DAG. It is a stand alone DB which is use for archiving emails (archiving function in Exchange 2010)

    The DB is currently in a DISMOUNT state because of the CATALOG INDEX FAILED & cannot be mount.

    I tried applying a command such as:

    Update-MailboxDatabaseCopy “Mailbox Database 01EX1” -CatalogOnly

    It does not work as the DB is not in a DAG.

    I have also try the following command:

    [PS] D:Program FilesMicrosoftExchange ServerV14Scripts>.Troubleshoot-CI.ps1 -Database “Mailbox Database 01EX1” –Action DetectAndResolve

    However, I did not have any luck with this one either.

    How can i correct this catalog index & hence be able to mount the DB ?

    Thank You in advance

    Ian Salgado

  34. Balsa

    It works! But before typing this commands I needed to restart my Exchange services. Thank you very much!!!

  35. Markus

    Thank you for that article – it helps me in a short way 🙂

  36. Amit

    When ran reset index for mailbox databases, seems this error
    Stop-Service : Service ‘Microsoft Exchange Search Indexer (MSExchangeSearch)’ cannot be stopped due to the following error: Cannot open MSExchangeSearch service on computer ‘.’.
    At C:Program FilesMicrosoftExchange ServerV14ScriptsResetSearchIndex.ps1:105 char:13
    + stop-service <<<< MSExchangeSearch
    + CategoryInfo : CloseError: (System.ServiceProcess.ServiceController:ServiceController) [Stop-Service],
    ServiceCommandException
    + FullyQualifiedErrorId : CouldNotStopService,Microsoft.PowerShell.Commands.StopServiceCommand

    I guess search service not respoding for reset,

    Any idea ?

    Thanks,
    Amit

    1. Wanou

      You need to run Powershell as an Admin Account otherwise you get this error.
      (right click on powershell, choose ‘run as administrator’)
      That’s all.

  37. Rax

    in that case, you need to go to the Exchange script folder where you have Active copy of DB with contect index Failed . you may find it is in default location which is C:Program FilesMicrosoftExchange ServerV14scripts

    after going in to Script directory through EMS, run below command

    .ResetSearchIndex.ps1

    It will ask you to press enter for continue. so say YES.

    Once it done, intial state of Contectindexing will show as crawling. later it will turn in to Healthy state.

    Cheers!

  38. Amit

    Hi,

    Could you have details on contentindex state failed on active node database server, As of we reseed only passive copy with active node. In case its active node database server how it possible to make its contentindex state failed to healthy

    does any one have answer ?

    Thanks,
    Amit

  39. Rax

    if there are more than 1 mailbox server in to Exchange environment then you need to run following commnad to know actually which copy of the databse having content indexing issue.

    get-mailboxserver |Get-MailboxDatabaseCopyStatus | fl name, contentindexstate

    Based on the result, it will give us that which databse on which server is having status fail of content indexing

  40. Amit

    Hey,

    Did you have information on failed index state as error message ” The Time stamp of the Catalog is too old for database”

    Please us know,

    Thanks,
    Amit

  41. Elliston

    Worked for me after changing my database paths and the copy wouldnt become active. Thanks – really great to have helpful people in the commuinity.

  42. Les

    Thanks Paul. Worked great for me.

  43. talishka

    Get-MailboxDatabaseCopyStatus | where {$_.contentindexstate -eq ‘Failed’} | Update-MailboxDatabaseCopy -CatalogOnly

    1. Adam Rollings

      Paul’s command failed for me because “[database]* matched multiple entries” but this sorted it thank you.

  44. Kirld

    Hi Paul,

    This works!! Thanks a lot.. Your posts has been a big help to the exchange community..

  45. Jon

    Hey Paul,

    Thanks for this info, it has worked both times I have had this issue. One question I have that I haven’t been able to find anywhere is what causes this to happen in the first place and how can I avoid it?

    1. Avatar photo
      Paul Cunningham

      Not entirely sure, to be honest. At least I can’t remember off the top of my head right now.

      1. RobK

        Noone is very sure about why that happens. We’ve seen it on nearly empty databases more often than on filled ones (for God’s sake!), and see it happen when the GC is not available in a Failover Cluster.

        Greetings,

        Rob

  46. Mike

    I found two other articles concerning the same problem, with slightly different solutions, which sounded promising…

    http://msexchangeguru.com/2010/12/01/cannot-activate-database-copy-content-index-catalog-files-in-failed-state/

    http://blogs.msdn.com/b/pepeedu/archive/2010/12/14/exchange-2010-database-copy-on-server-has-content-index-catalog-files-in-the-following-state-failed.aspx

    Unfortunately, neither of these could resolve my problem. In the end, I had to just stop the Microsoft Exchange Search Indexer service on both nodes, then delete the Catalog folder, and restart the service. I just hope this doesn’t happen too often.

    Mike

  47. Mike

    This isn’t working for me. The command completes without errors, but the state of the database doesn’t go back to healthy, and the server keeps logging ExchangeStoreDB 123 events (which says to run the Update-MailboxDatabaseCopy command to reseed the catalog). This has affected two databases in the past two days. The first one was a new database, so I fixed it by deleting and recreating it, but that’s not a long-term solution. Any other ideas? I’ll keep searching. Thanks.

  48. Leon

    Hi paul, works like a charm!! thnx for the good solution. +1

    Regards,

    Leon

  49. Jeff

    Hi,

    Thanks for detailing that out; however my databases are stuck in a permanent ‘crawling’ state and cannot run the -catalogonly command.

    Any suggestions would be greatly appreciated.

    1. Avatar photo
      Paul Cunningham

      How permanent? Is it just that your DB’s are big so its taking a while? Does dismount/mount clear it? Or cycling the search indexer service?

  50. Neil

    Thanks Paul, I’ve seen this before but it’s nice when the error and solution match right up.

  51. Chris Lee

    It’s work~ Thanks for your help.

    1. Hemant

      We have restarted Microsoft Exchange Search Indexer service. After restarting service all mailbox content indexes are Healthy.

      1. Lee Hutchinson

        This worked for me!

        1. Thacius

          Only restarted SearchIndexer!!!
          Worked for me! Thanks…

        2. Joe

          Only restart service… and work fine!!! thx!!!

Leave a Reply