Home » Exchange Server » Exchange Server Database Corruption and Dirty Shutdown Scenarios

Exchange Server Database Corruption and Dirty Shutdown Scenarios

This article is an excerpt from the Exchange Server Troubleshooting Companion.

During normal Exchange database operations, it’s certainly not unusual for a database process to terminate unexpectedly. A termination might be due to losing access to backend storage, file system corruption, or server-wide power loss. Many Exchange support problems involve corrupt databases which will not mount, and therefore prevent users from accessing their mailbox data. The following issues can prevent an Exchange database from mounting:

  • The Microsoft Exchange Information Store Service is unable to start
  • Missing Exchange database files
  • Database is in “Dirty Shutdown” state
  • Not enough free disk space on database or log file volume
  • Loss of access to underlying storage
  • While database is in Clean Shutdown state, significant logical corruption exists causing the database to dismount when an attempt is made to read the data

In situations where the Information Store service will not start, the most common issues involve communications between Exchange and Active Directory or some interaction involving a file used by Exchange and anti-virus software. I recommend using event viewer to verify Active Directory communications as well as verifying there are no service dependencies not met (such as the Microsoft Exchange Active Directory Topology Service not being started). With anti-virus software, verify all exclusions have been properly configured. It is common to discover that an anti-virus product either locks the database files or blocks necessary RPC communications. Anti-virus is also a common reason why database files become corrupt as a result of multiple processes attempting to access the files simultaneously. It is also common to see that an anti-virus product quarantines database files or logs, a step that might break the sequence of the log stream, making recovery from backup challenging, and possibly breaking DAG replication.

Dirty Shutdown

The words “Dirty Shutdown” are fairly self-explanatory; a database is down and not in a healthy state. The words can still cause anxiety for Exchange Administrators who recall spending many hours while frantically attempting to get a database to mount and restore functionality. Over ten years of support, I have probably worked on well over a hundred Dirty Shutdown support cases. But to be honest, a decade worth of experience is not required to troubleshoot a Dirty Shutdown event accurately, especially since the proper course of actions can easily fit into a decision tree. The diagram below provides a high-level flow chart for navigating common scenarios where a database will not mount.

exchange-server-dirty-shutdown-troubleshooting

First, let’s define what happens in a Dirty Shutdown. When Exchange dismounts a database, the Information Store ensures that all transactions (dirty database pages) in cache are committed to the database (.edb) file. When this is allowed to happen, the database is said to be in “Clean Shutdown” and requires no transaction logs to mount because all transactions have already been replayed into the database. The diagram below demonstrates how the ESEUTIL utility validates that a database is in Clean Shutdown state.

exchange-server-dirty-shutdown-eseutil-01

However, if something happens to cause the database to terminate abruptly, it is probable to result in a Dirty Shutdown state. Whether a Dirty Shutdown happens and which transaction logs may be required to recover the database is dependent entirely upon which transactions were being processed at the time of the failure. The diagram below displays a database in Dirty Shutdown, requiring log 3 (E0000000003.log) to mount.

exchange-server-dirty-shutdown-eseutil-02

Note: An easy way to get a database into Dirty Shutdown (for testing purposes, only do this in a lab) is to use Task Manager to kill the Store.exe (Exchange 2000-2010) or the Microsoft.Exchange.Store.Service.exe (Exchange 2013-2016) process. Before doing this, I recommend setting the Microsoft Exchange Information Store Service to “Disabled” (but not stopping it). This will prevent the service from automatically restarting itself and attempting to mount the database while you inspect it.

The DB1 database shown in the diagram above requires transaction log E0000000003.log because the transactions contained within it were either partially committed to the .edb file or are simply required to bring the database to a consistent state. It is hard to say exactly why certain logs are required without a deep programmatic understanding of JET Databases, but nonetheless the log is required and must be available before the database can be mounted.

Learn more about solving Exchange database problems in the Exchange Server Troubleshooting Companion.

Paul is a Microsoft MVP for Office Servers and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul is a co-author of Office 365 for IT Pros and several other books, and is also a Pluralsight author.
Category: Exchange Server

12 comments

  1. filip says:

    Ho Andrew,
    Is it still so that if you repair a db, you are in unsupported state according to MS? Are you also doing a write up how to fix a db with the required logs and other possible (dirty) shutdowns fixes?

    • Hi Frank, your bank might be blocking the transaction if it’s outside of your normal buying patterns. You can try choosing PayPal instead and using your CC that way (doesn’t require a PayPal account), or contact your bank to get the block removed.

  2. Joshua Bines says:

    Hey Andrew & Paul,

    Can you create a blank required transaction log such as E0000000003.log and mount the DB? Just wondering how 3-3 (0x3-0x3) converts to E0000000003.log i’m sure the prod farm may not convert this easily.

    Cheers
    Josh

    • Andrew Higginbotham says:

      The first database on a server will have a log sequence starting with E00. The second database will have a sequence starting with E01 & so on. Each sequence will start at that, then progress upwards using hexadecimal. So the first log would be E0000000001.log followed by E0000000002.log all the way to E00FFFFFFFF.log

      To answer your other question, an ESE database will only accept properly formatted transaction log files so what you suggest will not work. There are some tricks you can pull (like if the last log is missing, renaming the preceding log to E00.log, & using the /a parameter) but unfortunately that is not one of them

        • Joshua Bines says:

          Yep, that work flow is right from what I’m seing
          MapiExceptionCallFailed: Unable to mount database. (hr=0x80004005, ec=-515)

          Error Number JET Error Error Description
          Error -501 (0xfffffe0b) JET_errLogFileCorrupt Log File is Corrupt
          Error -514 (0xfffffdfe) JET_errBadLogVersion Log file generated with different Exchange Server or edition
          Error -515 (0xfffffdfd) JET_errInvalidLogSequence Any log file from the sequence is missing
          Error -533 (0xfffffdeb) JET_errCheckpointCorrupt Checkpoint file is deleted or corrupt

          • Andrew Higginbotham says:

            Yep. You can run an “eseutil /ml E00” (with E00 being the log sequence of that particular database) to validate all logs are present and not corrupted. So in this case, it views it as corrupted.

  3. Rob Betts says:

    book purchassed.

    I very rarely purchase technical books, finding that the majority of information is already either out there on the internet or in my head already. However since becoming a subscriber going back to the bootcamp days I have found myself more and more visiting practical365.com
    Its easy to read format makes it a valuable resource to any exchange server adminsistrator at any level – helpdesk to senior messaging. The purchase of the book was a no-brainer.
    Thanks to Paul and the rest of those who contributed, looking forward to what is probably another quality publication.

Leave a Reply

Your email address will not be published. Required fields are marked *