TFS 2010 - Test Impact Analysis Series - Part 6 - Redux

Just when you thought that the series was over....

After some consulation with experts and some investigation we can get round some of our issues.
To summarize our issues at this point:-

Multiple Testers get a warning raised - see Part 4 of our series for more info on this. No update on this yet but still trying.

App Pool with a custom account won't work without someone logged onto the server - see Part 5 of our series for more info on this. We have some progress here!

You will remember that the CLR profiler relies on environment variables to start. These can also be set in the registry. By manually setting the profiler guid onto the services the profiler will always attach. Obviously a side effect of this work around is that the CLR profiler will ignore your testsettings and always attach. If you are using the same environment for load / perf testing you will want to remove these reg entries first...

Here is how to change the environment for a service:-

Navigate to:-
Add a new multi-string value (REG_MULTI_SZ) and call it "Environment"
Double click this new value and in the dialog you can set the environment variable (one per line) using the syntax Variable=Value.

So simply goto the iisadmin (for IIS6), w3svc and WAS (for IIS7) services and add the following to the enviroment reg key


Reset each one for these changes to take effect.

Its probably a good idea to run ProcessExplorer and check that the environment variables are set properly.

When you now run a test using MTM you will still get a TestRunDataCollectorErrorWarning.xml file with the error about the custom account not having a writable user environment key. You can now safely ignore this as we have used the registry instead.

Ok so what has that bought us? The list of events that happened was

1.Run Test is selected in MTM.
2.The Test Controller restarts IIS on the webserver hosting the application to be tested. (You always wondered why this needed to be an Admin account didn't you?)
3.Tester Launches IE on local machine.
4.Proxy setting is changed on local IE (see part 3 for more details)
5.IIS worker process is started with correct environment variables set.
6.Trace profiling is started.
7.Tester ends test.
8.Tester may re-run test
9.Tester chooses Save and Close or just close in MTM
10.The Test Controller restarts IIS again
11.Trace profiling stops

Changing the registry has sorted out item 5 and 6. However items 2 and 10 still remain. The controller needs to restart IIS in order for the impact data to be written back.

So that leaves us with 2 options Option 1 (the easiest) is to manually reset IIS on the webserver once the test run is over.

This does not work for me for a variety of reasons:-

  1. Testers do not have that level of permissions on the web server.
  2. How do you know if someone else is testing at that point (if you reset IIS from underneath them in the middle of a complex test scenario you may end up running for your life!!)

However these are my restrictions. If you are the only tester (and have permissions) or can easily co-ordinate effort it may the easiest option for you.

Option 2 involves writing a custom Data Collector. See Part 7 for the details!


No comments :

Post a Comment