VSTS 2008 SP1 and IE8 ...

IE8 is coming up soon as Beta 2 is already out in the open, and one of the major changes in IE8 - I actually just came across it by chance - is that IE8 opens up to 6 concurrent connections per host compared to only 2 in IE7.

That leads to the question of Does VSTS support IE8? Will it really open the 6 connections per host?

Well luckly the answer to both questions are YES ...

• VSTS 2008 SP1 has the IE8 Option in the Browser Mix
• By testing IE8 in the load test, it worked :)
  It opened 6 concurrent connections per host.
  The test was created by having a load test with one web test in it, and one concurrent user, once ran against IE7 and another run against IE8.

Patterns & Practices: Performance Testing Guidance for Web Applications

Patterns & Practices: Performance Testing Guidance for Web Applications is now available on MSDN.

Contents
Landing Page
Foreword By Alberto Savoia
Foreword By Rico Mariani
Introduction
Chapters
Chapter 1 – Fundamentals of Web Application Performance Testing
Chapter 2 – Types of Performance Testing
Chapter 3 – Risks Addressed Through Performance Testing
Chapter 4 – Web Application Performance Testing Core Activities
Chapter 5 – Coordinating Performance Testing with an Iteration-Based Process
Chapter 6 – Managing an Agile Performance Test Cycle
Chapter 7 – Managing the Performance Test Cycle in a Regulated (CMMI) Environment
Chapter 8 – Evaluating Systems to Increase Performance-Testing Effectiveness
Chapter 9 – Determining Performance Testing Objectives
Chapter 10 – Quantifying End-User Response Time Goals
Chapter 11 – Consolidating Various Types of Performance Acceptance Criteria
Chapter 12 – Modeling Application Usage
Chapter 13 – Determining Individual User Data and Variances
Chapter 14 – Test Execution
Chapter 15 – Key Mathematic Principles for Performance Testers
Chapter 16 – Performance Test Reporting Fundamentals
Chapter 17 – Load-Testing Web Applications
Chapter 18 – Stress-Testing Web Applications

This guide was produced by the following Team

• J.D. Meier, Microsoft, Senior Program Manager, patterns & practices
• Carlos Farre, Microsoft, Software Design Engineer Test, patterns & practices
• Prashant Bansode, Infosys Technologies Ltd
• Scott Barber, PerfTestPlus Inc, Chief Technologist
• Dennis Rea, Wadeware LLC

Microsoft Tester Center

Microsoft launched their Tester Center that can be found on MSDN.

The Microsoft Tester Center showcases the test discipline as an integral
part of the application lifecycle, describes test roles and responsibilities,
and promotes the test investments required to deliver high-quality
software.

Extending Load Testing Browser Support

The browser support Out Of The Box for Team Edition for Testers is minimal.

But, it can be extended by adding your own browsers (common ones you'll probably want to add are IE7, Firefox and Opera).

The configuration files for the browser types are stored in C:\Program Files\Microsoft Visual Studio 8\Common7\IDE\Templates\LoadTest\Browsers.

The files end in the extension ".browser". These files simply contain the header information a browser sends with a request.

If you aren't sure of what information to enter in this file, simply open up your favorite browser that isn't in the list and go to http://www.ranks.nl/tools/envtest.html and grab the appropriate values!

Use the IE6.browser file as a template. For example, if you wanted to add support to test Opera you might add the following and save it as Opera.browser:

<Browser Name="Opera 8.0">
<Headers>
<Header Name="User-Agent" Value="Opera/8.00+(Windows+NT+5.1;+U;+en)" />
<Header Name="Accept" Value="text/html, image/jpeg, image/gif, image/x-bitmap, */*" />
<Header Name="Accept-Language" Value="en" />
<Header Name="Accept-Encoding" Value="GZIP" />
</Headers>
</Browser>

There are obviously other values you can place here and some of this will be dependent on your locale, plug-ins you have loaded and other environmental factors. Once you save this file, you should be able to go in and create a new load test and Opera 8.0 will now show up as an option in your browser mix list.

VSTS Future Releases ...

Microsoft released their roadmap for Team System and Team Foundation Server on MSDN.

Check it to see the many additions planned for VSTS and TFS, which target architects, developers, testers, and database professionals (in addition, of course, to project managers).

Performance Testing Guidance

Patterns & Practices: Performance Testing Guidance is now available on CodePlex.

The purpose of this project is to build some insightful and practical guidance around doing performance testing and using Visual Studio 2005.

It's a collaborative effort between industry experts, Microsoft ACE, Patterns & Practices, Premier, and VSTS team members.

Usage Scenarios:

• Improve efficiency and effectiveness of your performance testing
• Learn expert techniques performance testing
• Apply expert techniques for performance testing using Visual Studio.NET 2005

Getting Code Coverage Without Tests ...

The following steps shows how to get code coverage on .NET applications without having to actually run "tests" in the VSTS context. You will need either VSTS for Developers, VSTS for Testers, or VSTS Team Suite Edition installed to have the right tools to make it work.

Using the commandline you can instrument any .NET assembly, but beware of code signing as it might cause problems. Once the code is instrumtented, you need to get the "listener" running to collect the code coverage data generated, as follows:

1. Open a VS command prompt
2. Instrument your assembly by executing:
   vsinstr /coverage Assemblyname.dll
3. Start the Code Coverage monitoring process by executing:
   vsperfcmd /start:coverage /output:yourfilename.coverage
4. Run your Application that contains the instrumented code
5. End the Code Coverage monitoring process by executing:
   vsperfcmd /shutdown
6. Open the file "yourfilename.coverage" found in your application directory in Visual Studio to view the results.

VSTS Load Test Simulating Browser Cache

Below are some notes that can allow the Load Test containg Web Tests to attempt simulating the caching behavior of the browser

• There is a property named on each request in a Web test named “Cache Control” in the Web test editor (and named “Cache” on the WebTestRequest object in the API used by coded Web tests)
• When the Cache Control property on a request in the Web test is false, the request is always issued.
• When the Cache Control property is true, the VSTS load test runtime code attempts to emulate the Internet Explorer caching behavior (with the “Automatically” setting) as described in this article:
  http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnwebgen/html/ie_introfiddler2.asp?frame=true
  (This includes reading and following the HTTP cache control directives.)
• The Cache Control property is automatically set to true for all dependent requests (typically for images, style sheets, etc embedded on the page).
• In a load test, the browser caching behavior is simulated separately for each user running in the load test.
• When a virtual user in a load test completes a Web test and a new Web test session is started to keep the user load at the same level, sometimes the load test starts simulates a “new user” with a clean cache, and sometimes the load test simulates a return user that has items cached from a previous session.
  This is determined by the “Percentage of New Users” property on the Scenario in the load test.
  The default for “Percentage of New Users” is 100 in which case all user sessions are started with a clean cache.
  This is probably not correct for most applications where there are return users, so users should consider the most appropriate value to use for this setting depending on the actual usage of the application being load tested.

There was a bug in the original release of Visual Studio 2005 Team System that is fixed by the SP1 update where the load test was not honoring the “max-age” cache control directive.

Advanced Load Testing Features of Visual Studio Team System

Bill Barnett, a Software Design Engineer in Microsoft, wrote a great article on MSDN describing all the bits and pieces of Load Test features in VSTS.

Visual Studio Team Test Edition provides two capabilities that can allow
users to create load tests of web sites quickly and easily: the Web Test
recorder and the Load Test wizard. In fact, these tools make this process so
easy that it is tempting to use the web tests and the load tests that result
from using these tools without much modification. However, to use the web
testing and load testing capability most effectively it is beneficial to
understand how to use other web test and load test properties that are not set
by these tools. This note describes some of the most important considerations
for load testing with Visual Studio Team System that are not addressed by the
web test recorder and the load test wizard.

IE Add-In for Submitting Web Bugs to TFS

Check this post (Automatically raise bugs from IE) on the Microsoft Developer Tools UK Sales Team blog for information about an Internet Explorer Add-In Neil Kidd developed for submitting Web Pages bugs with all the necessary accompanying information to TFS.
It can be downloaded from GotDotNet.

I've written an addin for IE to allow users to easily raise a complete bug
report. When the user indicates that there is a problem on the page, the
addin
will create a new Team System Work Item and populate it with the
following
information:

• A screenshot attached to the work item.
• HTML source of the page and any frames and IFrames attached to the work
  item.
• Any style sheets referenced by the page attached to the work item.
• Any script files referenced by the page attached to the work item.
• Details about the user's machine and browser (OS version, browser version,
  screen size, ...).

Load Tests Results Dilema

Been trying to read the load test results captured on a client’s machines at the client’s hosting environment on our development environment to capture what are the problems we have in the application.

It took hours to figure out why the test results keep giving unable to connect to load test repository.

The only solution found is the following:

1. Create a local Load Test on your machine and export it for reference.
2. Open your Exported Load Test trx file and search for the key m_resultsRepositoryConnectString, this key holds the Load Test Repository connection string encrypted. This key is found under Tests/LoadTestResult node.
   <m_resultsRepositoryConnectString type="System.String">
   LAs4ogoKlDfikWPmARSuWmkl+..
   </m_resultsRepositoryConnectString>
3. Copy the encrypted connection string, and open your original exported load test trx files and replace the original connection string with the new one.
   

Voila!! You are now able to view your test results

Monitoring and Analyzing a Load Test Result

A great MSDN article related to Monitoring and Analyzing a Load Test Result that is very helpful.

Microsoft Visual Studio Team Test Edition provides tools for running,
monitoring, and analyzing load tests. The load test result viewer is capable of
monitoring a running test or analyzing a completed test. This document describes
the features of the load test result viewer

IE7 Browser Template for VSTS Web Tests

Sean Lumley posted about creating an Internet Explorer 7.0 browser template for use in Team System Web tests.

This post walks you through the process of adding a new IE7 browser
template that can be used by your web tests when running them stand alone or
inside of a load test.