Thursday, October 16, 2008

Could not establish secure channel for SSL/TLS with authority

My portal throws this exception when making a web method call to the web service.

Check list:

1. On server and client side, the server certs and their intermediate and root certs do not expire.

2. On server and client side, the server certs have right permission setting (e.g. giving permission to Network Service).

3. On the client side, the client cert used by WCF web method call and its intermediate and root certs do not expire. This is required because the server IIS SSL setting can be "requiring client cert".

4. On the client side, the client cert used by WCF web method call has right permission setting (e.g. giving permission to Network Service). This is required because the server IIS SSL setting can be "requiring client cert".

5. On server and client side, check the server certs are used by IIS.

6. Another experience: I found when an authenticated page expired, the WLID sign in page shows the sign in dialog in the left.htm frame. I figured out it is because the SSL cert is wrong (not for my portal's url). Then I made the IIS SSL cert to the one using the right portal url. After I changed the SSL cert for the machine where both the portal and the web service are deployed, I verified all above 5 points successfully but still got the error as the subject. Finally, I found the portal's web.config should change the WCF endpoint address from https:\\mymachinename\endpoint.svc to https:\\theportalurl.com\endpoint.svc.

7. Another experience: when my portal and the web service is deployed to 1box using installserver.bat, IIS uses SSL cert myserver.mydomain.com. Usually I use URL https://myportal-int.com and SSL cert myportal-int.com for my portal when it is deployed seperately. I always thought that to establish SSL channel between portal and WS, both side need to have the URL match the SSL cert. However, this is wrong! To establish SSL channel, only server side (WS side)'s SSL cert is verified.
My collegue help me address the issue eventually.
First, he pointed out that client side (portal side) does not need have a SSL cert which matches the portal URL. So I can get out of the trap of trying to make cert and URL match on both server side and client side.
Second, by the exception message: "The remote certificate is invalid according to the validation procedure.", he asked me to try to access web service using IE to see if there is any SSL warning. Under his expectation, IE showed SSL warning when I access the endpoint address.
Third, he logged in my machine and clicked the certificate error icon besides the IE url column. He viewed the https cert by clicking the "View certificate". He found the cert name is something like authmod, which is another site on the 1box. By this clue, he found site authmod also uses port 443 which is used by my portal site. After he removed https binding of authmod site, my portal start working.
I learned a lot from this lesson.

No comments: