TFS 2010 - Test Impact Analysis Series - Part 5

Phew, well we have come to the end of our series. This last post will look at how we setup application pools (specifically user accounts) and what that can do to TIA.

It is fairly common (at least to me) to set up an IIS App pool to run as a domain account. This means that this account can be granted access to downstream remote stores like databases. So obviously at this point it is no longer running as Network Service, Local System or any other built in account. Where I work, this is the norm. Great so what?

Well in part 4 we learned about how TIA (and IntelliTrace) rely on the CLR profiler for their magic. The profiler in turn relys on two environment variables being set on the process hosting the application.

Those who are old enough to remember environment variables will recall that processes inherit them from the user. So the environment variables are set against the user. These are stored in the user profile.

Is that the noise of a penny dropping...IIS6 does not load user profiles (See here) and so how will the environment variables get set...

According to the docs the way to do it is this.

If you decide to use a custom account for the identity that is being used for the application pool on the Internet Information Server (IIS) where you intend to collect Intellitrace data, you must create the local user profile on the IIS machine for the custom account that is being used. You can create the local profile for the custom account either by logging on to the IIS machine locally one time or by running the following command line by using the custom account credentials:

runas /user:domain\name /profile cmd.exe

Sounds all well and good. This will create the profile. But how will the profile get loaded.
Well it turns out that it won't.

What can go wrong?
If you try and run your test against a web site that is running under a custom account you will once again meet our old friend TestRunDataCollectorErrorWarning.xml. And this time he will say:-

Diagnostics and data collection from IIS failed: An error occurred during
website instrumentation: User "domain\user" does not have a writable user
environment key. Please make sure the associated application pool has the
"Load User Profile" option setup and has been recycled.

Uh oh. What is this Load User Profile - can this get us out of trouble? Unfortunately this is an IIS7 only setting. And judging by this post, it doesn't fix it anyway.

So the only thing that I have found to make it work it to.
  1. Log onto the web server
  2. Run the runas command as above using the custom account that the app pool is using
  3. Leave this window open
  4. Run the test

So we have found another limitation:- If you are using a custom account for your app pool, then someone must remain logged onto the server, and the runas command must be executed. Yuck! The only other option that I can think of right now is to write a custom data collector and use the logon api. You would also have to restart IIS as it does right now - but thats quite a lot of work for an out of the box feature. Stay tuned I may give it a try.


Impact Analysis is a great feature. But there are a number of serious limitations / gotchas associated with it. Is it worth the price? I am not sure yet. I am keen to get some feedback on other peoples perspective. If you have successfully used Impact Analysis against a remote website - what did you think?

I have asked a number of these questions to various experts and hope to hear back with more details. I will update these posts if anything changes.

See part 6 for more info.



  1. Thank you for this series on MTM: parts 2 and 4 have been much help for me. This is the first series of blog posts (i have come accross) that actually shows hands on experience. I'm wondering why you don't get comments on this series!

  2. Thanks Elger! Glad to see that they help someone. I have also posted part 6, with part 7 to come.