Lindström

My laptop has been telling me things for the last few weeks. I decided to ignore it. It kept persisting and I brushed it off. Until yesteday when the not so subtle messages were getting hard to ignore:

“Your installation of Windows Server 2008 R2 Enterprise will expire in 11 hours”

I’ve been running R2 as my laptop OS for a very long time now and it has performed flawlessly every single day. Many new builds have been released during this time but I have been so satisfied (and pressed for time) that I felt no need to upgrade or reinstall. To prepare for rather forced reinstallation due to the built in time bomb I grabbed a blank DVD disk, downloaded the latest build from connect.microsoft.com. Once downloaded I started to burn the disc only to be presented with an error message when the built in disc burning tool was trying to finalize the disc. I tried several blank DVDs but the end result was the same: all the installation files were on the disc but it sure wasn’t bootable. At this point I was down to 9 hours until expiration.

Time for Plan B

Could booting and installing from an USB stick work? I had read about this before but never tried it myself since I’m really not that into installation/deployment procedures but it turns out it’s really easy to accomplish and it also has the benefit of cutting down the installation time significantly.

Step by step instructions that work can be found here: http://www.sevenforums.com/installation-setup/1607-how-install-windows-7-usb-stick.html

At Qbranch we do quite a lot of identity management projects based on ILM. Some of our customers have their entire infrastructure in our data centers which makes monitoring of key systems critical to identity management solutions quite trivial. Since we are a consultancy we do also do these kinds of projects for customers who might have their IT outsourced to a competitor or they might be running their own data centers.

From experience we can say that no matter how large the outsourcing competitor is (and I’m including the big international ones here) we always get called in if there is any trouble with something identity management related at one of our customers. Since we’re not in charge of application monitoring in this scenario it might take quite some time until any issue is escalated to us.

Most of the time it turns out that one of the ILM connected systems is misbehaving or that someone made a human error in entering data into one of the connected systems. Regardless of what the issue may be we get asked to pinpoint the issue for our customer so it can be resolved. Sadly it may take a few hours or days until we’re notified that there might be a problem. To combat this waste of time and deliver a better service to our customers my colleague Pål Edman and I have spent some time building a proof of concept solution for application monitoring over the internet.

By leveraging message bus/message queue functionality found in the .NET Services building block of Windows Azure we had a really rough alpha version running and sending test messages within 10 minutes. Since then we have implemented a configurable event log listener and an ILM run history listener which both send messages across the service bus. Once we receive them on our end we can, depending on the type of message or whichever factor we may chose, raise alarms and take any action that we may please.

The beauty of using the service bus to tie together applications is that it will work over pretty much any network topology since connections are initiated from within the respective security and organizational boundaries.

• The WCF producer and consumer authenticate to the service bus with x509 certificates. No user name/password to worry about
• The messaging channel is encrypted
• The message payload is encrypted to ensure that only relevant parties can read the message

If you are thinking about connecting applications over the internet you should check out Windows Azure and .NET Services. SDKs are available for

• the .NET platform
• Java
• Ruby

During Microsoft Techdays in Västerås Joachim and me did a presentation on Active Directory Federation Services (ADFS) which went quite well. All our demos worked and I think we did a good job of explaining the concept of federations to the people in the room. In retrospect it’s easy to identify certain areas where we could have explained things in further detail but I guess that will always be the case regardless of the subject matter.

The event in itself was very well organized, there were plenty of interesting sessions to listen to and the keynote speach by Troed Troedsen was pure brilliance. My personal favorite among the sessions was a presentation on Security Development Lifecycle (SDL) by Johan Lindfors and Sergio Molero. If one were to boil down their presentation to one sentence it would be: Don’t treat security in your application as a last minute addition.

www.microsoft.com/sdl

SDL Threat Modeling Tool

edit: fixed some funny typos

Fixes in this release

• DSCT now works on Windows XP SP3
  It turns out that there is a slight difference in which properties are always present on objects returned in DirSync queries when running on Windows XP compared to Windows Server 2008 (XP returns less). DSCT relied on one of these properties to be present, a fix for XP has been added to work around this behaviour.
  The auditing feature is disabled when running on XP since this feature relies on event log functionality that was introduced in Windows Vista and Windows Server 2008.
  The buttons on the USNBug dialog are currently not visible since these rely on functionality that was introduced in Windows Vista. An XP fix will be added in a coming release.
• More bugs ironed out in the auditing feature
  Fixed some more credential related bugs.

Download here. Feel free to leave a comment with any feedback you might have.

There is a new version of the Directory Service Comparison Tool available for download.

Bugs fixed

• Resync action now checks that DSCT is connected to data sources before doing it’s thing
• Credential handling for alternate credentials was all backwards in the Auditing part of DSCT
• DirectorySynchronizationFlag cannot be set error reported by Tom appears to be fixed.

Thanks for the feedback Tom!

This release of DSCT is somewhat experimental since it makes use of some pre-release technology from Microsoft. It can probably be considered polite to point out this fact before you actually download and install this release of DSCT.

I am more than interested to hear your feedback so feel free to leave a comment if you run into any issues or if it just works for you.

Active Directory auditing changes are logged on the domain controller where the modification was made. The only way to get the full picture of what happened to an object is to query every domain controller for relevant event log entries. Performing these queries in a sequential way is quite expensive and time consuming.

The new Active Directory auditing integration feature in DSCT makes use of Parallel Extensions (June CTP) for .NET to efficiently query multiple domain controllers in parallel. In techno babble lingo this means that the AD auditing part of DSCT will make use of however many cores you might have in your machine. Running DSCT on a dual core machine will result in 4 (2 per core) parallel queries to retrieve event log entries. The performance increase over a sequential approach is quite remarkable by itself. When you factor in the ease of making parts (where it makes sense) of DSCT multi-threaded with the help of Parallel Extensions it’s just mind blowing.

In closing I’d like to point out that the new auditing feature in DSCT is the only part that makes use of Parallel Extensions. All operations involving Active Directory (reading, comparing, restoring values) use vanilla .NET functionality.

With that out of the way let’s get down to the good stuff.

New features in 1.3.X

• Active Directory audit integration
  Retrieve relevant AD audit event log entries for a specific object from all domain controllers in a domain. Filtering options allow you to find and inspect attribute specific event log entries.
• Group membership restore
  Display differences in group membership for a selected object (users, computers and groups). Allows for full or selective restore of group membership at any time.

 Base features

• Display differences between objects in Active Directory and Active Directory Snapshots
• Restore attribute values from snapshots to Active Directory
• Reanimate deleted objects

Download

• DSCT main page

Requirements

• .NET 3.5 or higher
• MMC 3.0
• Data sources: Active Directory and a mounted Active Directory Snapshot on Windows Server 2008 or Windows Server 2008 R2
• Active Directory audit integration: domain controllers must be Windows Server 2008 (R2 supported). Active Directory auditing must be enabled. RODCs and domain controllers running on older operating systems will be ignored.
• DSCT does not have to be installed on a domain controller nor does it have to be installed on a domain joined machined. A functional DNS infrastructure is recommended.

Testing so far

• x86 and x64 versions installed and tested on Windows Server 2008 x64
• Active Directory and snapshots on Windows Server 2008 and Windows Server 2008 R2 tested as data sources
• Locally on a DC and on a separate (non domain joined) machine
• Restore attribute values has been tested on several attributes including ntSecurityDescriptor, multi-value attributes like member and more normal ones like givenName, sn, accountExpires, etc.
• Group membership restore has been tested on users, computers and groups
• Reanimation functionality has been tested on users, groups and organizationalUnits.
• Active Directory audit integration tested with 10 domain controllers and a total 1.2 million event log entries in the security event logs. 6 of these spread over several domain controllers were of interest. DSCT, running on a dual core machine, retrieved these in roughly 6 seconds.

Known issues in 1.3.X

• Due to a bug in the Windows Server 2008 snapshot feature recently created snapshots will appear to be more up to date than the Active Directory instance. DSCT has a workaround for this scenario. Read this post for further information. This issue does not affect Windows Server 2008 R2.

AD Audit integration in DSCT (click for larger version)

It’s not quite ready for release yet but it should not be that far out. Of further note is that DSCT will require .NET 3.5 to run. Thoughts and comments are most welcome.

XPATH event log query against 20 virtual domain controllers with Parallel Extensions for .NET
Average number of of entries in the security log per server: 58799
Total number of security event log entries on all servers: 1175582
Total number of event log entries we are interested in (AD Audit for a specific object): 24

1 core / 1 thread: 36.188 seconds
2 cores / 4 threads: 7.839 seconds (x4.6 performance increase) (physical machine)
4 cores / 8 threads: 6.219 seconds (x5.8 performance increase) (virtual machine)

How much work did I have to do to take advantage of multiple cores? Probably hours and hours of coding, right? I changed one line of code by using Parallel.ForEach instead of the classical foreach when going through the list of domain controllers to query. Pretty neat, so far I’m really impressed with the Parallel Extensions!

One feature I’ve always wanted to have in the Directory Service Comparison Tool (DSCT) is event log integration, specifically when it comes to Active Directory Auditing. Having DSCT show you not only the the differences between a domain controller and an AD snapshot but also show you the related event log entries seemed like a good idea. I’ve had some test variations of this up and running that got the job done but were completely horrible from a performance perspective. If AD Auditing is enabled then the logging is done per DC. To get the full picture you have to query each and every domain controller to retrieve all the records. The old event log related classes in the .NET framework really don’t lend themselves to fast and efficient queries, something that is very much needed in a scenario like this.

This however changed quite drastically during the Windows 7 TAP Summit where I had the opportunity to show DSCT to Alain and Sanjeev from the AD team. We talked a bit around the event log scenario and I explained my findings regarding the lack of performance when it comes to querying the event log. Alain was nice enough to point out that there is a an entirely new way to interact with the eventlog in Windows Vista and Windows Server 2008 using XPATH queries.

The following XPATH query will return all records that have the EventID 5136 (a directory service object was modified) and also has a specific value in the logged ObjectGUID attribute. In this case this is the ObjectGUID of a user account.

*[System[(EventID=5136)] and EventData[(Data[@Name=\"ObjectGUID\"]=\”{31a84a37-2433-45ee-bb4a-31e26dbec47c}\”)]]

There are other EventIDs that we are interested as well, namely 5136-5139 and 5141 but the above is good enough during development of this feature.

For testing purposes I have 10 virtual domain controllers running Windows Server 2008 on a quite beefy HP ML370 server. Thanks to the Parellel Extensions for .NET DSCT can now make use of multiple cores and run several queries in parallel. Early testing shows that 2 threads per core are spawned which results in 8 parallel queries on my quad core test client.

From a performance standpoint DSCT can currently query 10 domain controllers and retrieve all revelant records in roughly 2 seconds. Since it’s still somewhat early in the development process these numbers should be taken with a grain of salt, they do however indicate that my goal to integrate AD auditing into DSCT with reasonable performance seems realistic.

Hello from Seattle! I’ve been here for a week together with Joachim to attend the Windows 7 TAP Summit, basically lots of in depth presentations and chalk talks during the entire week. Besides digging into lots of technical details I also had the chance to show DSCT to a few people and the response was quite positive. I’ve invested a lot of time in building the tool so it’s always nice when the initial response is something like “hey, that looks really useful!”.

During the evenings we met up with Joakim and Heinz from Hörby Kommun and Micke from Truesec. We ate grotesquely large steaks, took a few trips to malls around Bellevue and also spent a few hours in central Seattle.

We have roughly 8 hours until our flight leaves so we’re sitting around drinking coffee and working on our various little side projects. Joachim just released his Core Configuration Console, a very handy and lightweight tool for configuring Server Core. If you work with Server Core and are looking for a good config tool then I suggest you check it out!