Fellow Exchange MVP Jaap Wesselius has written about reports of content index failures for Exchange servers that have been installed or updated with Exchange Server 2013 CU14 or Exchange Server 2016 CU3. Those cumulative updates were released by Microsoft in September of this year.
Although I have not personally observed the issue in the field, a number of customers are reporting the same issue in a Microsoft TechNet forums thread.
One customer who opened a support ticket with Microsoft has reported back to the forum thread to say that a bug has been acknowledged, and that the solution is to deploy a new Exchange server (either Exchange Server 2013 CU13 or Exchange Server 2016 CU2) and migrate mailboxes to the new server.
If you are experiencing the issue in your own production environment then I recommend you raise a support case with Microsoft to determine whether you are impacted by the same bug (don’t rely on blog posts or forum threads to confirm a production impacting issue).
If you are considering deploying a new server as the resolution to the issue then you can refer to my like for like Exchange migration article for some high level guidance.
Update: the indexing issues are reportedly fixed in the December 2016 releases for Exchange 2013 and 2016
Hello
i also have on Exchange 2013 this issue with latest CU23
emails with attachments are not send
ActiveSync protocol does not accept any attachments
here what i found in IIS logs
/Microsoft-Server-ActiveSync/default.eas Cmd=SendMail&User=domain%5Cusera&DeviceId=androidc1419159386&DeviceType=Android 443 – 82.99.112.136 Android-SM-N975F/9 – 413 0 0 31
urgent help needed
thx
Pingback: torrent
Hi Paul,
I have an Exchange 2013 Server on CU4, what will be the best way to get it up to the latest CU21. the dotnet framework has been updated to 4.7.1
Hi,
I am using exchange 2016, facing newly create users login problem in owa and outlook.
getting error through owa. old users working properly.
(Something went wrong
The account you have signed in with doesn’t have access to email. Please sign out, then sign in to Outlook with another account. {0}
X-ClientId: ZL9U – O3HV – 40MF – MB7EPD3SOA
X-OWA-Error: SDServerErr;Microsoft.Exchange.Clients.Owa2.Server.Core.OwaNotMailEnabledException
X-OWA-Version: 15.1.225.42
X-FEServer: SERVER-NAME
X-BEServer: Servername.domainname.in
Date: 7/10/2018 8:24:25 AM
Fewer details…)
how to resolve.
Exchange 2013 standalone Cu14 and content index failing after fail…..
Thanks Microsoft for those wonderfull Cu’s lately and the extensive testing you do….. NOT.
Updates released today have fixes for the issue.
https://www.practical365.com/december-2016-updates-released-versions-exchange-server/
I tried to create a DAG on two exchange 2016 CU3 on Windows 2012 R2 servers but as soon as I want to install it on the second member I get these errors in the Highavailability eventviewer on the second server:
Cluster batch writer failed while attempting operation ‘GetClusterKey’ (error: An error occurred while attempting a cluster operation. Error: Cluster API failed: “GetClusterKey failed with 0x46. Error: The remote server has been paused or is in the process of being started”).
the temporary fix for this is :
add the public folder to a favorite ( favorites can be (properties -> download favorites ) downloaded localy and the outlook search will work on them.
BUT and this is a big BUT.. no pun intented..
if your PF’s are 2 TB… you better think not to fav the root or you will get a 2TB local cache file and i doubt many workstations have a 2tb free for that..
so pick and choose the PF folder you need search and fav those.
like i said this is the MSFT turnaround untill they figure it out in cu 15
Please let me know when microsoft has fix for the content index issue. I am running single echange 2013 cu14 on windows 2012 r2. I have turned off the content index for now.
We have an Exchange 2013 on which we installed CU14.
We now have the issue, that the ActiveSync protocol does not accept any attachments.
The error seems to lie on the IIS side. I activated all the protocols. When I send an email with an attachment the only log where I see the connection is the main IIS LOg.
This is one example:
2016-11-18 08:44:19 W3SVC1 core 172.16.0.34 POST /Microsoft-Server-ActiveSync/default.eas Cmd=SendMail&User=XXX%5Calp&DeviceId=SEC108XXXC4DB91D&DeviceType=SAMSUNGSMG935F 443 – 213.XXX.108.94 Android-SAMSUNG-SM-G935F/101.60001 – 413 0 0 229 115104 203
In the ActiveSync protocol these connection do not appear at all.
Only sending without attachments are delivered and appear in the ActiveSync Log.
All the Settings about eMail sizes and ContentLengths and so forth are set to 60 MByte.
Hi Paul,
With this issue raised in CU 14, would it be best to stay at CU 13 ? I wasn’t sure if they would end up pulling a CU due to customer issues. We don’t typically like to go to the latest CU so we usually stay at N-1.
For anyone experiencing the Exchange 2016/Windows 2016 issues, Microsoft has acknowledged a bug exists:
https://www.practical365.com/dont-deploy-exchange-server-2016-windows-server-2016-now-due-stability-issues/
I contacted MS support regarding the public folder search issue, spent approx an hour explaining the situation and demonstrating the problem. Search index was verified healthy (I rebuilt them during pre escalation troubleshooting) and all other configuration checked OK.
MS confirmed that there is nothing publicly available regarding this problem and after insisting on a response, I received an email confirming the issue. In a nutshell, their response was to wait for CU15. They did not suggest a workaround / recovery plan. I will send anyone that wants a copy of this confirmation by email. The cost of my case was refunded.
I did come up with a work around that may work for some people, it’s simple but blunt. In my case I need to search public folders, in addition to mailboxes, so I engaged cache mode in outlook, marked the public folder as a favorite and then added the public folder favorite to the local outlook cache. Search is working. Here are the steps:
Switch to Outlook’s Folder List.
Expand Public Folders and find the folder you want indexed.
Right-click the folder and click Add to Favorites….
Set any options or rename the favorite and click the Add button.
Right-click on Public Folders and click Properties for “Public
Folders”….
Click the Advanced… button.
Click the Advanced tab.
Check Use Exchange Cached Mode and Download Public Folder
Favorites.
Click OK to close the Advanced property sheet.
Click OK to close the Properties property sheet.
Select your favorite folders or otherwise synchronize your client to
download all items. This can take quite some time and is not recommended
over remote connections.
Now to script and deploy.
Credit to https://blogs.msdn.microsoft.com/heaths/2006/02/16/fast-searching-in-public-folders/ for the configuration steps. I only used the steps above in my fix. Now to script this.
We had a problem that CU14 exposed. Our external CA OWA certificate was bound the Exchange BackEnd IIS site on Port 444. The Self-Signed Exchange cert must be used here for Exchange to work properly. Not sure how that had happened but it wasn’t an issue until CU14 and all user connectivity went away. Not a fun day…..
Hey Paul,
We’re running Exchange 2013 CU14, and have had clients complaining of 0 results returned when using Outlook to perform mailbox and public folder searches.
Would this be symptomatic of the content indexing errors reported here, or just coincidence?
As I understand it, Exchange doesn’t index PF content as it does mailbox content – unless this behaviour has changed with modern public folders?
I am also having issues with the latest Exchange 2016 CU3 update. Outlook clients are connecting fine, the problem we’re having is with our mobile clients. iOS and Android devices are able to connect and receive email, but they can’t send.
Oddly enough the mobile MS Outlook app on both iOS and Android works
What have you tried so far?
Hey Paul, after hours of research, we went back through our initial Exchange 2016 notes and went through all the settings one by one, what we found is that any custom IIS changes that are made, were overwritten. So the changes we made to allow AD Certificate Authentication were wiped out. I went ahead and re-did the settings and rest IIS and voila my test devices started working.
Wasted a whole Saturday on this problem…LOL, but glad it got resolved.
P.S. ,I love this site, alot of great tools and write ups
It appears that I am not the only one with this issue. I just found this article:
https://social.technet.microsoft.com/Forums/lync/en-US/142ff1ab-242c-4436-a870-068720b5580f/exchange-server-2016-windows-server-2016-datacenter-iis-apppools-constantly-crashing?forum=Exch2016PS
I just migrated from exch2010 to exch2016(cu2) without issue. I then created a DAG on (2) 2016 servers running exch2016(cu3). I moved some mailboxes to new DAG DBs and cannot connect to mailboxes (password prompts, OWA hangs after login). I rebooted the DAG hosts hoping that it was just some old virtual directory configurations that needed to be refreshed. After the reboot connection issues persist and now I see a few other issues on hosts:
1. Cannot connect to either 2016(exch2016 cu3) host via the Exch Mgmt Shell get following error:
“[NullReferenceException]: Object reference not set to an instance of an object.”
2. System logs is flooded with WAS EventID 5011 Warnings for ALL Exchange related app pools:
“A process serving application pool ‘MSExchangeAutodiscoverAppPool’ suffered a fatal communication error with the Windows Process Activation Service. The process id was ‘31780’. The data field contains the error number.”
3. HTTPERR logs constantly report errors for all Exchange App Pools:
“Connection_Abandoned_By_ReqQueue MSExchangeServicesAppPool”
4. The Microsoft Connectivity Analyzer reports:
“A Web exception occurred because an HTTP 503 – ServiceUnavailable response was received from Unknown.”
I have confirmed that the certificate looks good. If I move the mailboxes back to the 2012R2(Exch2016 CU2) host all is good.
Any suggestions?
This directory also filling up with AppCrash_w3wp.exe files:
C:ProgramDataMicrosoftWindowsWERReportQueue
With the following contents:
Version=1
EventType=BEX64
EventTime=131221402635417971
ReportType=2
Consent=1
ReportIdentifier=e090d9f2-9d1e-11e6-836e-00155d005f00
IntegratorReportIdentifier=e73c91da-1e2d-456c-bc22-260e64ff8e9d
NsAppName=w3wp.exe
AppSessionGuid=0000616c-0000-000d-a661-669e2b31d201
TargetAppId=W:0000f519feec486de87ed73cb92d3cac802400000000!000054443c275eddd9bbe75c6d47cf9c24076b5d785c!w3wp.exe
TargetAppVer=2016//07//16:02:27:22!15ee0!w3wp.exe
BootId=4294967295
Response.type=4
Sig[0].Name=Application Name
Sig[0].Value=w3wp.exe
Sig[1].Name=Application Version
Sig[1].Value=10.0.14393.0
Sig[2].Name=Application Timestamp
Sig[2].Value=57899b8a
Sig[3].Name=Fault Module Name
Sig[3].Value=ntdll.dll
Sig[4].Name=Fault Module Version
Sig[4].Value=10.0.14393.351
Sig[5].Name=Fault Module Timestamp
Sig[5].Value=5801a332
Sig[6].Name=Exception Offset
Sig[6].Value=000000000009520f
Sig[7].Name=Exception Code
Sig[7].Value=c0000409
Sig[8].Name=Exception Data
Sig[8].Value=000000000000000a
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=10.0.14393.2.0.0.400.8
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=1033
DynamicSig[22].Name=Additional Information 1
DynamicSig[22].Value=318f
DynamicSig[23].Name=Additional Information 2
DynamicSig[23].Value=318f8eb61e176e217a9ae2192c7b640f
DynamicSig[24].Name=Additional Information 3
DynamicSig[24].Value=23de
DynamicSig[25].Name=Additional Information 4
DynamicSig[25].Value=23ded2732a75ed3d0d1da6bc85112415
UI[2]=c:windowssystem32inetsrvw3wp.exe
UI[5]=Check online for a solution (recommended)
UI[6]=Check for a solution later (recommended)
UI[7]=Close
UI[8]=IIS Worker Process stopped working and was closed
UI[9]=A problem caused the application to stop working correctly. Windows will notify you if a solution is available.
UI[10]=&Close
…
FriendlyEventName=Stopped working
ConsentKey=BEX64
AppName=IIS Worker Process
AppPath=c:windowssystem32inetsrvw3wp.exe
NsPartner=windows
NsGroup=windows8
ApplicationIdentity=FCBB9E96A3E01E440E0E4D9368EBD4AC
MetadataHash=2046253764
I excluded LoadedModule sections as it was quite lengthy.
There are reports floating around of trouble with Exchange 2016 CU3 on WS2016. You could log a Microsoft support case to see if it’s a known issue, or just wait to see if something is publicly confirmed.
I can confirm you (and Microsoft support will do it if you open a case) that if you create a DAG 2xExchange 2016 over Windows 2016 will cause IIS failure and EMS, ECP, ect will fail. It’s still under investigation.
Workaround: do not create DAG with Exchange 2016 over Windows 2016 , use Windows 2012 / R2 until fix release or next CU
I have the same issue, hopefully it’s only a lab environment:
Try disabling stopping and disabling the following services:
MSExchangeHMRecovery
MSExchangeHM
Hope this helps,
Benoit.
Yes, only testing. Unfortunately stopping/disabling those services did not solve all of the issues. It resolved the application and system log errors and warnings, but the exchange management shell will still not connect to the local exchange host. I have a ticket opened with MS. Will update if I get an answer.
I had a long contentindex problem on a fresh 2016 CU3 DAG install a few days ago..
but I was able to resolve it with some usual/old Trick “stop services (stop-service MSExchangeFastSearch stop-service HostControllerService) and delete index”, many Update-MailboxDatabaseCopy xxxx -CatalogOnly -BeginSeed, – CatalogOnly, and some reboots..
It seems to me that the problem was also somewhat related to circular logging…
Anyway, after many this and that, CI was ok and still is today.
Not sure if it is really related.. (Don’t have time to read many forum etc..) but perhaps it can help..
It will not Impact and will not give any error message in Outlook after decommission.
Here are registry values which can be helpful to tell outlook that there is no public folder now.
Key:
HKCUSoftwareMicrosoftExchangeSetup
DWORD Value:
HasPublicFolders
HasPublicFolders = 0
Paul wrote a very helpful article on decommissioning Public folder Database. That can be very helpful in your scenario.
https://www.practical365.com/exchange-2010-remove-public-folder-database/
Thanks Ashish Kumar and Paul
I have just migration from exchange 2010 to exchange 2106 cu3 full completed. But not yet migration public folder. I have a questions. Do i need move public folder from 2010 to 2016. Because the public folder do not use in exchange my organization
Thanks
You can decommission them if you want to. Public folders are not mandatory.
Thanks paul.
When my client check mail using Ms Outlook 2010 or 2013 is good. I have check connected via Ctrl + right click. I see a row still using exchange public version 14. 3.1. I afraid of decommission exchange 2010 to error problem.
Anything explain Pauls
Thankd
We had a problem with timeouts accessing public folders on CU14 and determined we could now get rid of them and did so. Be careful of Public Folders and CU14.
Thanks Gary