During an upgrade of Exchange Server 2013 to a new cumulative update some customers may experience an issue where setup fails, leaving the server in a non-operational state.

The error thrown by setup is:

Mailbox role: Transport service FAILED
The following error was generated when “$error.Clear();
$connectors = Get-ReceiveConnector -Server $RoleFqdnOrName;
foreach($connector in $connectors) { if($connector.MaxLocalHopCount -gt 1) { Set-ReceiveConnector -Identity $connector.Identity -MaxLocalHopCount 5 }};
” was run: “Microsoft.Exchange.Management.SystemConfiguration Tasks.ReceiveConnectorRoleConflictException: The values that you specified for the Bindings and RemoteIPRanges parameters conflict with the settings on Receive connector “EX2013SRV2Test”. Receive connectors assigned to different Transport roles on a single server must listen on unique local IP address & port bindings.

This error is caused by a receive connector conflict. When the error occurs setup stops, but unfortunately it has already removed the previous build of Exchange in readiness to install the newer build. So the server is left in a completely non-operational state.

In this article I’ll explain:

  • Why this error occurs in the first place
  • How to fix it so that you can run setup again to repair your failed server
  • How to check for this problem before you try to run a cumulative update

Why This Error Occurs in the First Place

The reason that this error occurs is that the server has two (or more) receive connectors bound to different transport services that are trying to listen on the same IP address and port number. My fellow Exchange MVP Andrew Higginbotham has a very detailed explanation of this here.

Every Exchange 2013 or Exchange 2016 server is configured with a set of default receive connectors that follow a standard naming convention that includes the server name.

Three default receive connectors are homed on the Front End Transport service:

  • Default Frontend SERVERNAME – this connector listens on TCP port 25 to accept SMTP connections, and acts as the entry point for email into the Exchange organization. It is pre-configured to perform that role, and there are few scenarios in which modifying this connector is appropriate. As a general rule I recommend you do not disable, modify, or remove it. Think of this receive connector as the connector of last resort.
  • Client Frontend SERVERNAME – this connector requires client authentication, accepts secure SMTP connections (with TLS applied), and listens on port TCP 587.
  • Outbound Proxy Frontend SERVERNAME – this connector listens on TCP port 717 and is only used when a send connector is configured to proxy outbound email through the front end servers or services. This is an optional configuration, and not very common.

Two default receive connectors are homed on the Transport service:

  • Default SERVERNAME – this connector accepts connections from other Mailbox and Edge Transport servers using TCP port 2525. Clients do not connect directly to this connector, and there are no reasons for you to ever disable, modify, or remove this connector.
  • Client Proxy SERVERNAME – this connector accepts connections from Front End Transport services on the same server, or on other servers, using TCP port 465. Again, clients do not connect directly to this connector, and you should not disable, modify, or remove it.

You should not modify or remove the default connectors. Any other connectors on the server are likely to have been manually created, and that’s where the problem can occur. The short version is that when creating the receive connector the wrong transport role was chosen.

choosing-service-for-receive-connector

“Hub Transport” is the wrong choice here. Unfortunately it is also the default choice, and a lot of Exchange admins experienced in previous versions of Exchange would recognise “Hub Transport” as the role where a receive connector would have been created. However, there is no “Hub Transport” role in Exchange Server 2013, nor is there a service called “Hub Transport”. You could argue that “Hub Transport” shouldn’t even be an option in that particular dialog.

Fortunately the most recent builds of Exchange Server 2013 (SP1 and above, from what I’m told) contain logic to prevent an administrator from choosing the wrong option and causing the connector conflict.

Of course, that doesn’t help customers who have perhaps deployed the RTM version of Exchange Server 2013 first, and then later tried to upgrade to the latest CU. This is more common than you might think it would be, due to some customers acting with the incorrect assumption that the RTM build needs to be deployed first before any service packs or cumulative updates are applied.

Those customers, if they happen to have created conflicting receive connectors, are going to have a bad day when they upgrade. So let’s look at how to fix it.

How to Fix it so You Can Run Setup Again to Repair Your Failed Server

For organizations with multiple Exchange 2013 servers the fix is simpler, because you can still access the Exchange Admin Center on one of the operational servers. In mail flow -> receive connectors you can select the non-operational server and see the list of connectors on that server. In this example the connector named “Test” is highlighted, and you can see that it is bound to the “HubTransport” role.

exchange-2013-receive-connector-configuration-01

That same connector name was in the error that was thrown by setup earlier, which is how you know which connector(s) to look for.

The values that you specified for the Bindings and RemoteIPRanges parameters conflict with the settings on Receive connector “EX2013SRV2Test”. Receive connectors assigned to different Transport roles on a single server must listen on unique local IP address & port bindings.

To change the role for the connector open the Exchange Management shell and run Set-ReceiveConnector as follows.

[PS] C:\>Set-ReceiveConnector EX2013SRV2Test -TransportRole FrontendTransport

For single server organizations it’s a little tricker. With no operational Exchange 2013 servers the management tools can’t be used. Instead we need to dive into ADSIEdit.msc to fix the connector. Launch ADSIEdit.msc and connect to the “well known Naming Context” of Configuration.

The receive connector objects can be found in Configuration -> Services -> Microsoft Exchange -> Your Organization Name -> Administrative Groups -> Exchange Administrative Group (FYDIBOHF23SPDLT) -> Servers -> Server Name -> Protocols -> SMTP Receive Connectors.

The attribute we’re interested in is the msExchSmtpReceiveRole. In the example below are two receive connectors. The one on the left has the role attribute set to 16384, which is Frontend Transport. The one on the right is set to 32, which is Hub Transport.

exchange-2013-receive-connector-server-roles

Modify the value from 32 to 16384 on your receive connector and then apply the change.

After taking either of the corrective actions above re-run the cumulative update and setup should proceed past the failed step and complete successfully.

How to Check for this Problem Before Running an Update

As I mentioned earlier there is logic in current builds of Exchange Server 2013 that should prevent this issue from occurring in future, but for those of you reading this who are planning to upgrade from a very early version of Exchange 2013 you might be concerned and want to avoid the problem entirely. There are two ways to do this.

The first is to simply open your Exchange Admin Center and look at the receive connectors on your servers. If you see custom connectors that you’ve created and that are bound to the Hub Transport role then you can pro-actively apply the Set-ReceiveConnector fix demonstrated above and you should be fine from there.

If you’re still unsure then you can simply run the same PowerShell commands that setup is executing when it throws the error. I’ve adjusted them slightly here for readability and added the -WhatIf switch so that Set-ReceiveConnector doesn’t make any actual changes to your configuration.

$connectors = Get-ReceiveConnector;
foreach($connector in $connectors)
{
    if($connector.MaxLocalHopCount -gt 1)
    {
        Set-ReceiveConnector -Identity $connector.Identity -MaxLocalHopCount 5 -WhatIf
    }
}

Pasting those lines into the Exchange Management Shell and pressing enter will check all of the receive connectors in your org for the conflict.

check-connector-conflicts

Read the errors carefully though, because it will throw one for your mis-configured connector but also for other default connectors that are not mis-configured. Obviously you should only apply the fix to your custom connector and not modify any of the default connectors on the server.

Summary

As you can see this is an unfortunate error condition that has quite a severe impact on Exchange 2013 servers during an upgrade. In particular, single server organizations are left with no operational servers at all after this error occurs. Hopefully anyone in that situation will be able to quickly find the solutions demonstrated in this article and get their server up and running again.

About the Author

Paul Cunningham

Paul is a former Microsoft MVP for Office Apps and Services. He works as a consultant, writer, and trainer specializing in Office 365 and Exchange Server. Paul no longer writes for Practical365.com.

Comments

  1. Michael

    We have 3 Exchange 2013 servers. Can i install the CU23 on just one server and then the other 2 on a later date?

  2. Ralf

    Thank You. You save may (Sun)-Day.

  3. Hos

    Hi Paul,
    I have same issue recently for install the CU17 on the Exchange 2016. we have 4 Exchange server and do we need to change on the all exchange servers or just one by one when we want to install the CU

  4. Marcel W.

    You Made and Rescue my Day (And my ass 🙂 )

    Greetings from Germany

  5. john

    Great work paul, you save me once again.

  6. Jay Nettleblad

    I have a 2013 RTM with all (both) roles installed on one server (VM).
    There are a couple Receive Connector Hub Transport rules, Default, and Client Proxy installed. I ran the test script and it passed. Can I keep these, or should I change to FrontEnd as the article recommends?

  7. Thomas Guldberg

    Believe it or not I just completed a 2013 RTM installation and went straight to CU21. Single server installation. Your article just saved my day. Thx again :O)

    1. Jay Nettleblad

      I would love to know if you did any special pre-install work or had any issues?

  8. Shelton Lomp

    Hi Mr. Paul ,

    Thanks for the Great Tip, Too Bad mine problem wasn’t resolved with what you gave us.
    Mine Error is at the

    MailBox role: Mailbox Service.

    Error:
    ” was run: “Microsoft.Exchange.Data.DataValidationException: Database is mandatory on UserMailbox.
    at Microsoft.Exchange.Configuration.Tasks.Task.ThrowError(Exception exception, ErrorCategory errorCategory, Object target, String helpUrl)
    at Microsoft.Exchange.Configuration.Tasks.Task.WriteError(Exception exception, ErrorCategory category, Object target, Boolean reThrow)
    at Microsoft.Exchange.Configuration.Tasks.DataAccessTask`1.Validate(TDataObject dataObject)
    at Microsoft.Exchange.Configuration.Tasks.SetTaskBase`1.InternalValidate()
    at Microsoft.Exchange.Configuration.Tasks.SetRecipientObjectTask`3.InternalValidate()
    at Microsoft.Exchange.Management.Common.SetMailEnabledRecipientObjectTask`3.InternalValidate()
    at Microsoft.Exchange.Management.RecipientTasks.SetUserBase`2.InternalValidate()
    at Microsoft.Exchange.Management.RecipientTasks.SetMailboxBase`2.InternalValidate()
    at Microsoft.Exchange.Management.RecipientTasks.SetMailbox.InternalValidate()
    at Microsoft.Exchange.Configuration.Tasks.Task.b__b()
    at Microsoft.Exchange.Configuration.Tasks.Task.InvokeRetryableFunc(String funcName, Action func, Boolean terminatePipelineIfFailed)”.

    This is part of what the Error sais.

    Hope you can help me to with error.

    Thx

  9. Daniel

    Thanks, you saved my life!

  10. Ben George

    This is a fantastic article – it saved my bacon after an abortive upgrade from Exchange 2013 RTM to CU20. Thank you very much, Paul!

  11. Chris O'Dell

    Thank you!!! You saved our upgrade night.

  12. Navneet

    as checked below two connectors are showing with hub transport

    •Default SERVERNAME
    •Client Proxy SERVERNAME

    i think in article these were mentioned, default, so do i need to change this ? only these are two with Hub Transport

  13. Navneet

    Hello Paul,

    I have Hybrid envrionment of Exchange 2013 with Office365. there are few recieve connectors are showing pointing to hub transport on all 4 servers

    So how can i check if those are default connectors or created while going to hybrid environment. I dont want to take a risk.. Please suggest

    and changing from hub transport to front-end transport, will make any difference ? or it is very much fine to change ? any impact

    Looking forward for your response..

    and thank you so much to u. i read this article before upgrade..

    1. Paul Cunningham

      I don’t understand your question. Are you asking me what the names are for the receive connectors that Exchange creates for itself automatically during setup? Those connector names are listed in my article above.

      1. Navneet

        what do you think, is this issue can come from upgrading from cu13 to cu17 ? in sp1

        and changing from hub transport to front-end transport, will make any difference ?

        1. Paul Cunningham

          It could happen in any upgrade scenario, so you should check it before you upgrade.

          1. Navneet

            as i see recieve connectors are in type hyb transport ..

            so can i change all to front-end transport, ? will it impact anything in current envrionment

            we are in hybrid environment, with O365

  14. The Cyberwolfe

    OK, this is like the third time one of your articles has saved my ass. You need to put a “buy me a beer” button on your profile.

    1. Paul Cunningham

      When people buy my ebooks or watch my Pluralsight courses I get money that I can use to buy beer, so yeah 🙂

      1. Brian

        We just got Pluralsight. I will try to find you on there. Thanks.

  15. Rasmus

    You good sir, saved my day.

    THANKS!

  16. A. O.

    Thank you!!
    The ADSIEdit fix for a single server worked like a charm!

    FYI, before I started the CU17 install I did run your script but it did not return any errors, but then it did error out during the install.

    Either way, thanks! You saved me many hours!

  17. James Wallace

    Thank you!! ADSIEdit worked perfectly. Life-Saver article! Saved me from rolling back to backup.

    1. Paul Cunningham

      If you mean rolling back using a VM snapshot, that isn’t supported and would likely have caused more damage to your environment.

  18. Dan Turnbull

    Hi Paul,

    Does this only apply to multi-role servers?

    Thanks

    Dan

    1. Paul Cunningham

      Well, you should only be deploying multi-role servers anyway 🙂

      For Exchange 2013 it applies to multi-role servers. For Exchange 2016 there’s no separation of CAS and MBX anyway.

      It’s probably possible to mess it up with separate CAS/MBX servers as well but since I never deploy Exchange that way I couldn’t go into more specifics.

  19. Brandon

    You saved my ass on July the 4th man, thanks so much.

  20. Jan Fokkelman

    Um, if I change the receive connector to front end, mail stop flowing all together. So the hub transport is doing something, if I change it back (In ADSIEDIT) mails working again?

  21. eric

    This is a great article! I did exactly what you had mentioned for a test environment, trying to update from a RTM build to CU6 with a custom receive connector set to HubTransport instead of FrontendTransport. Great article!

  22. dany

    hi Paul,

    i cant find :
    Configuration -> Services -> Microsoft Exchange -> Your Organization Name -> Administrative Groups -> Exchange Administrative Group (FYDIBOHF23SPDLT) -> Servers -> Server Name -> Protocols -> SMTP Receive Connectors.

  23. Gary

    Great job on the article.. Going to watch for this one in the future.

    Many thanks !

  24. Asmaile Frage

    Mush appreciated Paul, you’re the MAN

  25. Magudeeswaran M

    Hi Paul,

    I think there is a correction needed at this article,

    Wrong One: Configuration -> Services -> Microsoft Exchange -> Your Organization Name -> Administrative Groups -> Exchange Administrative Group (FYDIBOHF23SPDLT) -> Servers -> Server Name -> Protocols -> SMTP Receive Connectors.

    Correct one: Configuration -> Services -> Microsoft Exchange -> Your Organization Name -> Administrative Groups -> Exchange Administrative Group (FYDIBOHF23SPDLT) -> Servers -> Server Name -> Protocols -> SMTP Receive Connectors -> Default Exchange 2013 –> properties

    I have experienced the same issue in my lab and corrected it some time back.

      1. Magudeeswaran M

        hi Paul,

        but I can’t locate the attribute “msExchSmtpReceiveRole” under the properties of “SMTP Receive Connectors” , I can see it only under the properties “Default/Default FrontEnd /ClientProxy/ClientFrontEnd/OutboundProxyFrontEnd” after expanding “SMTP Receive Connectors”

        i’m still confused.

        1. Paul Cunningham

          The servers have several receive connectors that are configured by Exchange setup. They are listed in the article above. Those should never be touched.

  26. Farshid Rahimi

    HI,

    I have problem with my Exchange server 2013 Sp1 ,the stucked emails to the draft folder 3 week after installation. please help me.
    thank you.

    F. R

  27. TJ Demas

    Paul, you are the friggin’ man! Wow. CU13 just takes a dump and leaves you cold and alone in the wilderness to fend for yourself. I’m a single site, and running RTM for over two years! (Hey, it was 99.9% functional…so, stop your laughing!)

    Your article and fix here is very concise and a lifesaver. Many thanks from the Old Mule, in Louisville, KY. I’m indebted to you for your knowledge and sharing it here.

  28. BenM

    Thanks Paul! I got this error with CU12, just saved my weekend!

  29. Charles Giles

    Thank you, I have read many articles you have written in the past also. You are a real Systems Admin. One that shares his details and thus is not all about me type person. I am currently on a 200k 2010 to 2013 migration with one of the biggest companies everyone knows. I have great praise for your time and thoughts. I love my complex job and friends that share their vast experience’s. Although I have worked at MS twice in my career their resources and services tickets fail me time and time again and the TechNet articles are slow to change to keep up. By that time me and many of the worlds engineers are stuck in “crises mode” and are forced to find a solution while the customers suffer for lack of better words “they loose money”.
    Thank you for all u do !
    Charles Giles

  30. Natalie

    I am so glad I found this before I run the CU’s!

    I have a DAG which was set up before I started and now have the task of getting them up to date. Each server in the DAG has Outbound, Deafult and Client pointing to FrontendTransport and then a Client and Default pointing to HubTransport which are left over from the 2010 migration. They both cover all IPs like their FrontEnd counter part but are set to port 2525 and 465 with the Frontend connectors using 25 and 587.

    My intial thought is to just delete the HunTransport connectors as we have the Frontend Transports connectors. Am I right in thinking that as they point to Hub Transport and this is no longer a service then they can’t be working anyway?

    Many Thanks,

    Natalie

    1. Paul Cunningham

      No … don’t modify or delete any of the connectors that Exchange setup created. This article only applies to custom connectors that have been incorrectly configured by an administrator when they were created. I’ve updated the post to clarify.

      1. Natalie

        Great, thank you! So I shouldn’t get failed update leaving them as they are and pointing to the HubTransport?

        1. Natalie

          Just read your amendment, thank you very much.

  31. patrick

    YOU made it 8-D

    Thanks a lot Paul. It’s awesome to learn from you!

    Greetings from Germany
    patrick

  32. Blake

    Adding my heaps of praise and thanks to the thread, thanks Paul! Saved my bacon for sure! Had a hard time finding anything related to the error until I limited my search and found this. As an above comment said, it was like a needle in a haystack! Very well written article, perfect step-by-step cleanup/fix. Thanks!

  33. PL

    Thank you Paul, you rescued me from my heart attack! I would have never been able to figure this out. (Single Exchange server, I am managing an installation done previously by someone else)

    Cheers,

  34. Magudeeswaran M

    Hi Paul,

    We currently have Exchange 2013 SP1 CU8, trying to install CU11.

    It says we need to update the Schema/AD domain, which I did not expect.

    Is this behavior normal or different thing, I cant see anyone told about this here in forum.

    1. Paul Cunningham

      It is normal behaviour. You can expect every Exchange cumulative update to include a schema update.

  35. Ben

    Great comment. It helped me big time. thks a lot. I got this error while upgrading to CU10.

  36. Charlie

    Whew… Thanks to you and google my heart has stopped trying to pound out of my chest. I could have restored to last backup, but I never like having to tell the boss that. Thanks!

  37. Ole Gunnar Deila

    Fantastisk! Big Thanks indeed !

  38. Dan Stevens

    Life = Saved.
    Thank you.

  39. Frank

    Needless to say. Thanks again Paul.

  40. William Henderson

    I have a question.

    I am trying to setup Exchange 2013 to receive email from a Spam Server but I get the following error message “the values that you specified for the Bindings and RemoteIPRanges parameters conflict with the settings on Receive connector.” How am I to allow incoming email from this server?

    1. Paul Cunningham

      The default receive connectors on an Exchange 2013 server are already configured and ready to accept email for internal recipients.

      Other than that, any new connector you create should be bound to Frontend Transport not Hub Transport.

  41. Andrea

    you saved me 🙂
    thanks !!!!!

  42. Tom Erdely

    Wow. This is a needle in a haystack problem and your multiple solutions (either with 2+ Exchange servers or with 1) are fantastic. Thank you Thank you Thank you.

  43. Geert

    Worked like a charm! Thanks Paul!

  44. Ayman Elsherbeny

    I faced this issue while upgrading from Exch2013-SP1 to CU9 on DAG server with two nodes ( Active/Passive ) starting with the passive node.
    I got the error ( Receive Connector Conflict )
    I had to delete the Customized Receive Connector with Hub Transport on Both DAG Nodes and restart some Exchange Services.

    Finally It’s working and resuming the upgrade process
    You Saved My Time .
    Thankssssss Paul

  45. Artur

    If one has a CU8 already installed (and upgrade from CU7), can it be assumed that this will not be an issue when installing CU9? Thanks!

    1. Paul Cunningham

      Why rely on assumption when it only takes a few seconds to check?

      1. Artur

        Fair point – to be safe I checked – the script that you provided returned no errors at all.

  46. Aaron Karstedt

    Great article, came in handy. I’ll note that SP1 does not have sufficient logic to prevent this, I just updated SP1 -> CU9 and a disabled HubTransport connector caused this.

  47. Vasilis Tikas

    Same case – single exchange- you use ADSI tool and change it in the AD. Works like a charm !!

  48. F. Adams

    Hi,

    Great post, but what to do if you only have a single exchange 2013 server with this error?

      1. F. Adams

        Hi,

        been using adexplorer from sysinterals. ADSIEdit doesnt seem to show all the information. While with adexplorer everything shows up as described in the article.

        After that the setup failed again, due to not able to find the grammars directory (C:Program FilesMicrosoftExchange ServerV15UnifiedMessaginggrammars) . I have created this folder manually and setup completed after a new run.

  49. Vasilis

    You rule !!!

  50. IT

    What about receive connectors on a mailbox server that is not co-located with the ClientAccess role? Do these need to be set to FrontEndTransport as well?

  51. Twan van Beers

    Hi Paul,

    Nice article, thanks for taking the time to write it up! Interested to hear why you’re choosing to change Test to a FrontendTransport connector, rather than changing the bindings to port 2525?

    I thought that the default FrontendTransport connector already allowed access from anyone on any source IP address anyway, so wouldn’t you normally create additional HubTransport receive connectors with the different Auth Mechanism and Permission Groups as needed for a smaller set of IPs?

    Thanks in advance
    Twan

  52. Rich W

    Great article, this has caught me out in the past. I restoring from backups to get up and running again. I’m nervous about trying again as I don’t have a test environment to try it in beforehand 🙁

    1. Paul Cunningham

      Using ADSIEdit to fix the role for the connector, or deleting the offending connector entirely, seems to be a reliable solution from all reports I’ve received.

Leave a Reply