Visual Studio 2010 Code Coverage of IIS and Windows Service based Application

Code Coverage of IIS based application

It was very surprising to find that there was no enough information available over the internet regarding how to perform code coverage of IIS hosted application. 

All the site/ blog i visited was having half information causing me to spent lot of time debugging the issue. Anyways but good news is, i was able to solve this issue and thought of writing my first ever blog with all the detailed information i was able to learn from this

Lets Start

Assumption: 

You have an asp.net based application hosted on Windows Server and there are some windows services that you are application is interacting with

Pre-requisite: 
1. Visual Studio 2010 Performance Tools are required to be installed on the server box where your product is installed/ hosted.
2. You should have .pdb (symbols) in hand. Symbol files are must for instrumentation
3. If possible, try to get code (.cs) file on the installation box. This will be required for analyzing the code

Performance tools are available at: 

C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools
You can set this as path in environment variable 

Quick steps

1. vsperfclrenv /globaltraceon  and reboot 
2. vsinstr /COVERAGE [assembly] 
3. vsperfcmd /START:COVERAGE /OUTPUT:[ouputfile].coverage /CS /USER:"[ASP.NET worker process user]" 
4. vsperfcmd /ATTACH:w3wp.exe
5. Run the test scenario against the web application 
6. vsperfcmd /DETACH 
7. iisreset /STOP 
8. vsperfcmd -SHUTDOWN

Step 1: vsperfclrenv /globaltraceon  and reboot

This will turn on trace on for the machine. Reboot the machine after turning the trace ON.

Step 2: vsinstr /COVERAGE [assembly] 

Second step is to instrument the dll's. Now this is the most trickiest and most important steps. You can not skip this. To instrument a dll you can use "vsinstr /coverage <<your assemble name>>.dll"

You will find this information on all the blogs but there is something missing in most of the blogs:

1. You may get error saying Files are in use
2. You need to have .pdb (symbols) file at the same location your dll is present
3. Now there will the case when you have your n numbers of dll and in different folders under hosted directory
4. Above command will only instrument only one dll but how to instrument all the dlls specially when there are 100s of dll present for your application

Don't worry, i have a solution for this:

1. First copy all the .pdb where the dlls are present. This you need to do manually. If you need to automate this, ask your release management to build installer in such a fashion that when you install the build, pdbs are present with dlls
2. Stop IIS using "iisreset /stop" command and also make sure all the windows service referring to your dlls are stop. If this step is not done, some dlls which are actively being used may not get instrumented
3. Create a batch file with following commands. This bat file will take care of issue 3 mentioned above, it will search for all the dll, in a specific folder and instrument those dll's one by one. This will save lot of your times when you have multiple dlls.  

cd <location of your dll>
if not EXIST C:\Bin mkdir C:\Bin\
for /f "delims==" %%F in ('dir /b *.dll') do copy "%%F" C:\Bin\
for /f "delims==" %%F in ('dir /b *.pdb') do vsinstr /Coverage "%%~nF.dll"

     4. Run this batch file for all the folders you have. Place this batch file in each folder you have your dll's  and then run the bat file from Visual studio 2010 command prompt

Once the instrument is complete, you will find your original dll will be renamed with .orig extension existing dll will be instrumented. Now start the iis using "iisreset /start" command and start all the services again

Step 3: vsperfcmd /START:COVERAGE /OUTPUT:[ouputfile].coverage /CS /USER:"system" 

This step will start the monitoring of instrumented dll. Lets review each arguments

• /start:Coverage - This will tell the tool to start the coverage
• /Output:[outputfile].coverage - This will allow you to name the output file
• /CS: CS means cross session. This will be required so that you can attach to IIS
• /User:"system" - This is very important step. In order to monitor IIS based application, you need to have this option and user mentioned should be correct. Go to your iis and in application pool, check under what user iis is running. Same user should be mentioned /user clause

Once this step is done, just to ensure it will capture what you want, restart iis using "iisreset" command and restart all the windows services

Step 4: vsperfcmd /ATTACH:w3wp.exe

This step will tell code coverage tool to monitor iis. To do this, open your application in a browser. This will start w3wp.exe process and now run "vsperfcmd /ATTACH:w3wp.exe" command. 

Sometimes you may find there are multiple w3wp.exe command. In that case you can attach to both the process using process id (replace w3wp.exe with PID)

Step 5: Run tests

Run all your test from anywhere. You can run from same machine or from your test machine. Test can be run manually or through automation. Just to regular testing using browser

Step 6: vsperfcmd /DETACH 

Once your tests are completed, detach the process using above mentioned command

Step 7: Stop your iis and application related windows services

This step is required so that perf tool can collect code coverage data

Step 8: vsperfcmd -SHUTDOWN

This command will ask the tool to stop collecting code coverage and generate code coverage file in output location provided in step 3.

Now you can open the .coverage file using Visual Studio and understand the coverage of your test.

I will try to add screenshot soon but at this point you are ready to go.

Some tips:

1. Place the .cs file as mentioned in pdb file. Double click any function you are evaluating and it will tell you where you need to place your .cs files. Once done, you are can evaluate the code

2. vsperfcmd /Status will let help you find the status of process. Just try it and you will find some interesting information