BPuhl’s Blog

A little bit of everything without actually being much of anything

Archive for May, 2006

ADFS Certificate Maintenance – v1

Posted by BPuhl on May 25, 2006

Over the past several weeks, we’ve celebrated the 1 year anniversary of our ADFS deployment.  I say it this way, because the only reason I know this, is that the certificates on the servers keep expiring, and things would break unexpectedly.  Yeah, yeah, yeah…I could have, would have, and SHOULD have been a bit more proactive about this, but our use of ADFS internally is somewhat limited until this summer (aka – fiscal year and budgets) when we’re going to start onboarding a lot of new ADFS based services for our users.

So, I wanted to get an idea of where I needed to update the certs.  Since we use a common cert for both the ADFS trust policy signing certificate, as well as the IIS SSL cert, I needed to make sure I replaced them both.  We also use an FS Proxy on the internet, against SSL and policy signing cert required here.  This all made sense to me, so off I go:

  1. Change the SSL cert in the IIS console of the FS
  2. Change the SSL cert in the IIS console of the FS-P
  3. Change the token signing cert on the FS using the ADFS MMC
  4. Change the token signing cert on the FS-P using the ADFS MMC

(note:  pet peeve of mine, which I’ll probably rant about again… but we can’t remotely administer the ADFS servers using the MMC.  You have to TS onto the box, or write your own scripts to do things remotely….  grrrrrr… yes, the product group knows, yes I remind them every chance I get…unfortunately no, it didn’t make it into R2.)

Ok, so at this point, I’m thinking to myself, “cool, this annual maintenance is done.”  Probably took 7 full minutes before my phone rang that EVERYTHING was broken.  Seems that I forgot that a year ago when we set this up, I had to send a copy of the token signing cert to the federation partners.  Their FS-R’s need to be able to validate the cert.  Ok, drop that in e-mail, follow-up with a phone call…everything’s working from internal now (ie. users connecting directly to the FS server).  RDP’d back to my home computer, so I could get a view of this thing from the internet though and it’s still failing.

Perusing the event logs on the FS-P (which look like a Christmas tree…if Christmas colors were red and yellow) I find:

The Federation Service Proxy encountered an exception when it called a Federation Service Web method.
Federation Server URL:
Web method: GetProxyTrustConfiguration
Proxy certificate thumbprint:
<snipped by brian>

This may cause a user request to fail.

User Action
The exception details may give an indication of the precise problem.

Check network connectivity between the Federation Service Proxy and the Federation Service.

Ensure that the Federation Service is running.

Ensure that the Federation Service Proxy client authentication certificate has been added to the list of proxy authentication certificates in the Federation Service trust policy.

Ensure that the Federation Service Proxy client authentication certificate chains to a root that is trusted by the Federation Service.

Ensure that the Federation Service Internet Information Services (IIS) Secure Sockets Layer (SSL) server certificate chains to a root that is trusted by the Federation Service Proxy.

Ensure that the Federation Service Uniform Resource Locator (URL) that is configured in the Federation Service Proxy web.config uses the name that is the subject of the Federation Service IIS SSL server certificate.

Additional Data
Exception details:
System.Web.Services.Protocols.SoapException: System.Web.Services.Protocols.SoapException: Server was unable to process request. —> System.UnauthorizedAccessException: Attempted to perform an unauthorized operation.
   at System.Web.Security.SingleSignOn.FederationServerService.MethodInvocationCheck(MethodAuthenticationLevel DesiredAuthentication)
   at System.Web.Security.SingleSignOn.FederationServerService.GetProxyTrustConfiguration(VersionInformation proxyVersion, VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& trustConfig)
   — End of inner exception stack trace —
   at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
   at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
   at System.Web.Security.SingleSignOn.FederationServerSoapProxy.GetProxyTrustConfiguration(VersionInformation proxyVersion, VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& trustConfig)
   at System.Web.Security.SingleSignOn.LSPersistentState.GetPolicy(VersionInformation& fsVersion, ProxyInformation& proxyInformation, TrustConfigurationData[]& data)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

I’ve bolded the important part above.  When I zipped right back to the FS server where I found the following event:

The Federation Service failed a privileged Web method call because the caller’s client authentication certificate is not configured as a Federation Service Proxy certificate.
Certificate thumbprint: <snipped by Brian>

User Action
Ensure that the trust policy is properly configured with all valid Federation Service Proxy certificates.

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Back to the ADFS snap-in, and in the trust policy there is an FSP Certificates tab.  Added the token signing cert here as well so everything matched, and voila! auth started working again.

There were a couple of reasons I wanted to jot this down:

  1. As the ADFS service scales, managing the certificates is going to become significantly more important and require a lot of advanced planning.  While I’m a HUGE fan of documenting and testing operational processes, etc… The fact that this will require coordination between the FS-A and the FS-R provider, and that this once-a-year function requires so many touch points, is just scary.
  2. I wanted to point out the quality of the events that are being logged by the service.  The ADFS team put a lot of thought into getting the “right” information into the logs, so an administrator can quickly figure out what’s going on with the service.  There is still granular debug logging available, but you shouldn’t really need to use it that often.  IMHO, too many admins are jaded against the event logs anyways (rightfully so?), but I still believe that if eventvwr isn’t your first stop, then you’re probably not doing your job right.  For federation services, this is definitely the case.  ADFS Product Team – THANK YOU.

Posted in ADFS, Identity and Access | Leave a Comment »

Orgs…and Re-Orgs…

Posted by BPuhl on May 24, 2006

(Especially) If you work in a large company, then organization changes aren’t really anything new.  Microsoft certainly isn’t an exception, but for the 5 or so years that I’ve worked here, the primary function of the team I’m on has rarely changed.  Our team name changed, managers changed, people come and go, but for the most part the one thing that remained consistent was that our “org chart bubble” was moved as a whole and the teams remained intact.

For a bit of background, AD support in MS IT was divided between 2 teams.  I was part of the core infrastructure team which owns AD, DNS, ADFS, DHCP, WINS, IPSec, IPv6, and several other “smaller” services that we would keep track of.  Specifically for AD, we owned the domain controller configuration, replication, topology, performance, capacity planning, as well as being responsible for a large amount of the dogfooding work that comes around.  Because AD is so widely deployed with high availability and redundancy, we’ve always tested new OS’s, fixes, etc.. on the DC’s first.  In short, we were the “service owners”.

The other half of the AD support was the Identity Management Team.  IDMT owned the account provisioning systems, MIIS, Schema, GPO’s, Trusts, etc…  Basically, they were the “data owners” for the directory.  For the past year or more, IDMT has been part of our corporate security organization.

Like Bob Dylan says, “These time they are a-changin'”

This most recent re-org has really shaken things up a bit.  Rather than maintaining 2 separate groups for different functions, Microsoft IT now has a single Identity and Access Management team, which owns both IDMT and infrastructure functions for AD, as well as PKI, ADFS, and some other services.  Whether DNS gets org’d with AD or remains with the core team is still up in the air.

Personally, I’m pretty excited about this change.  For one thing, the director of the group is someone that I’ve worked for before, and am happy to work for him again.  The combination of duties will help us align better with the way the AD product group is organized.  But mostly, because I’ve always worked closely with the IDM Team and have a great working relationship, but when it comes down to it, they had their priorities and we had ours, and sometimes the two didn’t quite meet.  Hopefully being in the same org, with the opportunity to influence (and be influenced) by each other will be good for everybody.

One thing that did occur to me through all this, is that I have talked to very few customers about the way that “they” are organized for AD support.  I’m sure this varies from company to company, and can only imagine the enterprise type customers having an “AD team”, but I’ve started to wonder whether they split these duties or keep them together.  If you see me at TechEd, or talk to me on one of our regular conference calls, and I’ll probably ask you about what YOUR org chart looks like.  I’ve always felt that even though in several ways we’re not, that Microsoft IT should be representative of the customers.  So in this case, I suppose it would be good to know what that means.

Posted in Babbling and Blabbering | Leave a Comment »