Exchange Sync Issues with large Active Directory

Recently I worked with a large customer getting to the bottom of some issues experienced with Exchange Sync in Project Server 2010, specifically the issues originated due to the large multi-forest nature of the Active Directory environment in which the Project Server was deployed.

 

Symptoms

Exchange Sync for resources not working, little is logged in the ULS without verbose logging other than the following:

06/30/2011 12:25:49.04        Microsoft.Office.Project.Server (0x0E3C)        0x08C4        Project Server        Queue        954k   Medium  PWA:http://pwa.something/PWA, ServiceApp:Project Service Application, User DOMAINServiceAcc, PSI: [QUEUE] Retry: 1 ExchangeSyncTasks Microsoft.Office.Project.Server.BusinessLayer.QueueMsg.ExchangeSyncTasks 

To get to the bottom of what was happening Verbose logging on Project Server Exchange Tasks was required which showed far more detail which I will summaries here:

06/30/2011 12:25:48.00        Microsoft.Office.Project.Server (0x0E3C)        0x16D4        Project Server        Exchange Sync    fux2     Verbose        -! Info: System.Net.WebException: Unable to connect to the remote server —> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond xxx.xxx.xxx.xxx:443 at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress) at … …

06/30/2011 12:25:48.00        Microsoft.Office.Project.Server (0x0E3C)        0x16D4        Project Server        Exchange Sync   fux2  Verbose        =? Trying to Autodiscover using email at ‘https://autodiscover.companydomain.com/autodiscover/autodiscover.xml’ 

06/30/2011 12:25:48.04        Microsoft.Office.Project.Server (0x0E3C)        0x16D4        Project Server        Exchange Sync   fux2  Verbose        -! Info: System.Net.WebException: The remote name could not be resolved: ‘autodiscover.companydomain.com’ at System.Net.HttpWebRequest.GetRequestStream … …

06/30/2011 12:25:48.04        Microsoft.Office.Project.Server (0x0E3C)        0x16D4        Project Server        Exchange Sync   1zpd  Verbose        Error is: ExchangeSyncEWSUrlFailed. Details: Attributes: a6a779c4-a930-461f-a738-a3c78e8e826a . Standard Information: PSI Entry Point: Project User: DOMAINusername Correlation Id: a1e43a95-1a78-44a0-be1b-e8b3aa267074 PWA Site URL: SSP Name: Project Service Application PSError: ExchangeSyncEWSUrlFailed (40509)

06/30/2011 12:25:48.04        Microsoft.Office.Project.Server (0x0E3C)        0x16D4        Project Server        Exchange Sync  9fbi  Verbose        Error is: ExchangeSyncGeneralProcessingFailure. Details: Attributes: a6a779c4-a930-461f-a738-a3c78e8e826a Microsoft.Office.Project.Server.BusinessLayer.Queue.
ExchangeSyncEmailAddressInvalidException: Could not find Exchange server for resource a6a779c4-a930-461f-a738-a3c78e8e826a at Microsoft.Office.Project.Server.BusinessLayer.Queue.
ProcessExchangeSyncMessage.ExecuteSync … …

06/30/2011 12:25:49.04        Microsoft.Office.Project.Server (0x0E3C)        0x0DD0        Project Server        Exchange Sync  fux2  Verbose        ?? Starting SCP lookup for domainName=’companydomain.com’, root path=”   

[cut out multiple variations of the above as Autodiscover tries in vain to find the account]

If you’ve read through that, you can see that Project Server is attempting to use the Exchange Autodiscover service to locate the users Exchange details, as per the Autodiscover protocol which very very basically is something like this:

  1. Look for an Autodiscover service on the host (then look on the same host without the autodiscover bit)
  2. If not found try an SCP (Service Connection Point) lookup in Active Directory to find the resources

This can fail in a multi-forest Active Directory environment if your Exchange Client Access Servers (CAS) are located in a different forest. As was the case with my customer.

 

Solution

This is actually a well understood problem in Exchange circles, and the full solution is well documented:

How to Configure the Autodiscover Service for Multiple Forests

However if like in my case you don’t have the time to get a significant change such as the above completed then an alternative is needed.

 

Workaround

Looking at the above autodiscover procedure an easy “workaround” (I call this one a ‘hack’) is clear;

  • Add an entry to the HOSTS file on all Project Server application servers pointing to autodiscover.companyemaildomain.com which points to the actual autodiscover service running on the Exchange CAS.

However now you’ll see something like this in your Verbose ULS:

06/30/2011 12:40:04.17        Microsoft.Office.Project.Server (0x0E3C)        0x1930        Project Server        Exchange Sync        fux2        Verbose        -! Info: System.Net.WebException: The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel. —> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure.

This is because your Exchange CAS has something like as the Subject CN in its SSL certificate which would be expected, so what you need to do add a Subject Alternative Name to your certificate, see the following article on doing this:

Configure SSL Certificates to Use Multiple Client Access Server Host Names

Now you’re almost there, just one last thing! In order for the Exchange and Project Server to both authenticate these cross-forest users you need to make some changes on the service account on both forests by configuring the msExchMasterAccountSid property on the Exchange forest. See the following similar technet forum solution to achieve this:

EWS returns error "Failed to get valid Active Directory information for the calling account"

 

Done.

Hope that helps someone else out there!