Index of “How-to” for Windows Azure

http://msdn.microsoft.com/en-us/library/gg432998.aspx

How to use ADFS for OWA access

Ken posted a great article about how to configure OWA for ADFS authentication:  http://www.theidentityguy.com/articles/2010/10/15/access-owa-with-adfs.html

Overheard in a meeting (paraphrased)…

…the problem is, that instead of trying to make what we have work.  Every software architect believes that that their <widget> will be the solution that everyone adopts…

 

Typing this, reminded me of something else that I heard recently, which was along those same lines…

Of course my idea on the whiteboard is better than all the code that you’ve written!

Getting your Bitlocker keys out of AD

I often talk about my perspective that AD is a great publishing engine, but that it should not be authoritative for anything.  Any mission critical data should be mastered outside of AD, and then sync’d into the directory to be published/consumed.

The problem with this, is when you have services which source their information in AD directly, but that data is still mission critical.  One example of this, would be BitLocker Drive Encryption recovery keys.  The BDE service on clients will write it’s recovery keys directly into AD.

Before MSIT broadly deployed Bitlocker, we worked with an internal team to build a solution for finding new BDE recovery keys, and copying them out of AD into an external store.  We even went a step further, and put some self-service recovery options in front of that store.

I’m happy to see that MSIT was able to publish this solution out to Codeplex, so we can share it with everyone.

If you’ve got Bitlocker deployed in your environment, but are ONLY storing the recovery keys in AD – you may want to take a look.

http://keyrecoverytool.codeplex.com/

ADFS 2.0 is in the house!

Released To Web:
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=118c3588-9070-426a-b655-6cec0a92c10b&utm_source=feedburner&utm_medium=feed&utm_campaign=Feed:%20MicrosoftDownloadCenter%20(Microsoft%20Download%20Center)#tm

If you’re going to use the same password for everything…

At least let it be a good password:  http://www.cxo.eu.com/news/password-protected/

Forget your password?

Read an interesting article at http://redtape.msnbc.com/2008/08/almost-everyone.html on the issues/weaknesses of password recovery schemes. 

Most everyone remembers when Sara Palin’s Yahoo mail account was hacked, because her password recovery questions were easily discoverable.  One thing that I thought was interesting in the article though, was the idea of a “black market” for personal information – let me go buy a profile to find out the name of your dog, your favorite restaurant, etc…  How would people come up with this information in the first place, are there secret spies in black trench coats following everyone around taking notes on everything they do?  I have no idea…

…in other seemingly unrelated news – has anyone else taken all those funny Facebook quizzes where you answer questions about yourself, and they tell you how long you’re going to live, what your zodiac sign means, and things about your shopping habits and sexuality that you never even realized you knew?  There’s got to be a thousand of those things out there…  I’m sure glad that Facebook is much safer than the dangerous “internet”!

hmmmmm……

Personal Limits

We all have limits, we usually just don’t think about them as such.  Or when we do think about them, it’s usually because somebody else is pushing them and it’s making us mad.  But this is more about personal limits, especially with respect to the way that we run our servers in MSIT.  (Important to note that MSIT is probably unlike any other environment out there, so your limits may be different, but the idea is probably the same)

On Sunday, Rey Diaz included some of the following thoughts in a conversation about Wisdom versus “right”

Do you try to come as close to breaking the law as you can, without breaking it?
Do you try to push your morals as far as allowed, without being immoral?
Do you try to move as close to disaster as you can, without actually feeling the consequences of disaster?

This evening, I was reading an article in one of my flying magazines, and Rod Machado had an article where he talks about one of my favorite factoids in aviation:

On page 6-26 of the FAA’s Pilots’s Handbook of Aeronautical Knowledge, we find that, “Aircraft certification rules require accurace in fuel gauges only when they read empty

In fact, if you look up the actual Federal Air Regulation, it says:

FAR 23.1337 (b.1) Each fuel quantity indicator must be calibrated to read “zero” during level flight when the quantity of fuel remaining in the tank is equal to the unusable fuel supply determined under part 23.959(a)

Is there anyone around that actually wants to be flying in an airplane when it’s fuel gauges read the only accurate calibrated measurement that they have to?  I like flying gliders, but not like that!

Ok – but this is a geek blog – so what does this have to do with anything remotely interesting to you?

Well, in MSIT, the question often comes up about what are our DC performance.  Sure, you can go graph hundreds of counters and things, but see my earlier post about situational awareness and then tell me how easy it is to keep yourself aware of all that flaff.

Instead, we’ve got some personal limits when it comes to our DC performance that have worked pretty well for us over the past few years.  They are:

20-40% Target sustained CPU utilization
40-50% CPU utilization and we start checking for unusual causes of load, but if this is just normal trend growth then we either bring it back down with hardware replacement, or additional servers
> 50% CPU utilization – evaluate the trends, this is indicative of a potential problem – may need to start budgeting process for new servers
> 60% CPU utilization – we consider ourselves “broken”, and we go into break/fix mode to either reduce load or increase capacity

Of course, nobody’s saying that AD is broken at 60% CPU, these our just our personal limits.  After all, if I wanted to wait for AD to break before I did anything, I might as well spend my free time polishing up my resume.  The idea of course, is that you want to decide when it’s broken for you, and think about what you’re going to do ahead of time – this way, you’re not thrashing around, and management isn’t surprised when you have an out of band budget request.

A few other tidbits while I’m thinking about this. 

– These numbers are purely based on our experience in our environment.  We know that when we run over 75-80% CPU, we’re running very hot and some sensitive applications can be impacted by latency.  We also know that our standard operating procedure is to have 3-5 DC’s offline at any given time.  We have to account for the fact that when servers are offline (dogfooding, debugging, etc…) we need the headroom for the load on the other boxes.   

– We consider “sustained utilization” to be averages over 15 minutes, across all DC’s, but we’re also applying the human element to the data.  We’re looking for the trends, not the spikes…we know that spikes happen…

-  In a perfect world, you’d at least know where the load came from.  More often than not, there isn’t a single smoking gun, it’s just increased utilization as other systems in the environment are leveraging AD.  At the moment, I can only think of one time when we could trace the increased utilization back to a single project, and that was our IPSEC deployment – of course, we couldn’t roll that back anyway, we still had to increase capacity – so it’s not like it really mattered, but it was nice to know what caused a 20%+ jump across the board.

Personal limits, a good thing to bring to work with you.

Passive Safety & Situational Awareness

I find that there are a lot of concepts which I bring to my job in MSIT, from my hobby as a private pilot.  In this case, I am “borrowing” the title for this post from an article by Bruce Landsberg in one of the magazines I subscribe to.  He starts:

… Compared to machines, the homo sapiens’ conceit of being masters of the universe shows us to be consistently unreliable when it comes to repetitive tasks.  We do excel, however, in thinking up ways to get out of mindless chores to refocus our short attention spans on really important stuff…

This hit me, because earlier today I was talking to some of our engineers about our “team server” – which is the box that we use to run all of our recurring scripts from, collect data to, store utilities/tools/scripts on, and generally dump stuff.  Appropriately named Dumpster (does that make us dumpster divers?  probably).  We run a lot of scripts to collect a lot of data for our own use.  Although we’ve got the full blown monitoring infrastructure in place and we own all the settings for alerting, etc…  SCOM is owned by one team, the alerts go to our 24×7 operations center (who resolves the bulk of them), etc…  So if the administrators are abstracted from most of the chaff, how do they maintain situational awareness?

Situational awareness, another one of those terms I picked up flying.  Basically, the understanding of what’s going on around you.  Easily demonstrated with the following question to your administrator:  “How’s AD doing today?” – by default, the answer will be, “AD’s running great” (that is their job after all…) – the follow-up question though, “How do you know?” is usually the zinger.  If the answer is, because nobody from Help Desk is screaming at us, then that’s probably not a good sign.  If the answer is, because there are no trouble tickets, that’s probably also not a good sign… 

Lack of bad doesn’t necessarily equal good.

When I have a chance to talk to the MS Directory Masters classes, I usually try to work in the following story:

In 2002, I was one of a small group of AD administrators for MSIT, we were knee-deep in dogfooding Whistler, which shipped as Windows Server 2003, when one day my GM walks by, sticks his head in the door (never a good sign), and asks “How’s AD doing today?”.  Default response at the time was something like, “Looks good, couple of DC’s being upgraded, so far so good… why do you ask?”  It’s at this point that he says, because I just got a call saying that our Extranet is offline, nobody can authenticate to any applications, our partners aren’t able to do business with us, and I was wondering what you were doing about it?  If I remember correctly, it was about that time that he looked a little worried about his hiring decision, turned and walked away…

Quickly (trying) to log onto the domain controllers, all 6 of the DC’s were running at 100% CPU utilization.  Perfmon, SPA traces, expensive/inefficient query logging – nada/zero/zip – we were in trouble.

Within a couple of hours, we’re all in a big room – techies around the table, managers looking over our shoulders, and we had the AD rock stars from the product group (the developers) sitting in the room, taking apart the DC’s in the debuggers.  They were all shaking their heads, when someone mumbled under their breath, “this almost looks like normal load…just a lot of it”

That’s when we decided to pull some perf data for the past 6 months, which looked something like this:

Sure enough, we had been growing load for the past year or so, all the DC’s were running at 100% CPU, we stole 4 servers which were racked & built for some other application, DCPromo’d them and perf dropped down to a reasonable level…

oops…

As you can see from my MSPAINT representation – WE ACTUALLY HAD THE PERF DATA!  The problem was, that we had lost situational awareness of what was going on in our other environments, because we were so focused on dogfooding.

The moral of the store then, being that it’s good to HAVE data, but it’s much better to LOOK AT the data occasionally…  

The cloud doesn’t need all your passwords (just one…)

Patrick Harding and Pamela Dingle recently posted a great article, Grounding Enterprise Passwords.  If you haven’t already looked at it, I recommend you do so.

For a moment though, let’s say that you’ve already sold your management on the benefits of identity federation, and have deployed the infrastructure, and are rockin’ and rollin’ with SSO.  It’s time to sit back and relax, comfortable in the knowledge that your users passwords are securely locked inside your directory, so you’re enterprise is “safe” right?  Uhmm, maybe not.  Go grab your local CISSP and ask them when the enterprise is safe, and they’ll spout a bunch of stuff about risk management, defense in depth, and mitigating controls such as firewalls, virus scanners, and yes – your identity system & passwords.  If you dig in though, they often aren’t talking about protecting the “enterprise” – because that’s sort of an ambiguous amalgamation of many things – one of which is “enterprise data”.

Enter the cloud.  Do you care about applications moving to the cloud?  Absolutely (so does your CxO by the way)!  Do you care about how users are getting to that data?  Of course, as Patrick, Pamela, and others point out – it’s critical to ensure the identity of your users.  But we also have to be concerned about the data that resides in the cloud, and what that means to the rest of the enterprise.  Quick illustration:

Cloud Collaboration Vendor:  Move your data to my service, and I’ll save you bazillions of dollars over your on-premise suite, plus I’ll give you these value added features like letting your users view their data directly through my service from anywhere (without having to download everything locally), powerful indexing, blah, blah, blah…

CIO:  Ok, so let me play back to you what I heard, “I sign here, my users quit complaining about our VPN solution AND you save me bazillions of dollars” – GREAT!  Go work with my team and make it so…

CCV:  Ok IT guys – your CIO has signed off, now here’s the migration plan:  Train your users, copy the data, and…oh yeah – we need the private key that you used to encrypt any of that data so we an index it and decrypt it for your users when they ask…

IT Guy:  Como say WHAT?!?  That’s the key we use to encrypt ALL of our enterprise data, not just the stuff we’re hosting with you

Does your business require data encryption for some things, like high-business-impact data?  If so, how do you reconcile this with pushing the data out to a cloud service?  Or do you not?  How many instances of your data protection infrastructure do you have (is there more than one key?)  Does your vendor support data encryption at all, and if so – do they use their keys or is there a dependency on your service?  In my experience, most cloud services are loath to take too many dependencies on customer infrastructure, SLA discussions become big finger-pointing exercises. 

Back to data encryption though.  The conversation becomes even tougher when you start to talk about the “cross-premise” scenario, which is where you maintain a set of infrastructure on-premise, and host the rest of it in the cloud.  I should be able to protect my on-premise data – that a vendor should never have access to anyway – from the vendor, right?  Of course I should – so I need to have data protection FOR the vendor, and data protection FROM the vendor. 

In this thought exercise, there is an interesting tension about “who” are you protecting the data from.  In the on-premise world, the reason you protect data is so outsiders (and even some insiders) can’t get to it.  Where on the scale of trusted entities, does your vendor fall?  Even if you’ve done your due diligence, and funded new Ferrari’s for an army of lawyers, what data do you give access to?  Let’s assume you give your vendor access to all the data that is “relevant to their service”, so the vendor can decrypt any data which is hosted in their site.  What’s the process for re-encrypting the data in the case of a breach, either of the on-premise key or of the cloud key?  Often times this is a herculean task, which requires knowing/finding all of the encrypted data, and then re-encrypting it with a new key. 

If you decide to cancel your contract with a vendor – is that roughly equivalent to a compromise of the key?  Everyone I talk to says yes, that somebody with protected content and the ability to decrypt it, who is not authorized to do so – is a security problem.  As far as I can see, this is going to need to be something that the lawyers cover, otherwise the off-boarding costs of a vendor skyrocket.

These are just a few, there are a bunch of hard questions when it comes to the cloud – which is what makes this space so much fun! – I don’t have all the answers.  Here in MSIT, where we classify and encrypt A LOT of data, we’re having conversations with everyone, business owners, security folks, lawyers.  I can’t say we always tread carefully, sometimes we just “go for it”, but when it comes to adopting cloud services, we’re looking hard as we’re taking the next step, and part of that is how we protect our enterprise data IN the cloud, as well as FROM the cloud.