Windows Azure: The Testing Story, Update 1

My co-worker pointed out that IIS Express would not work for us.

In his words,

Running IIS express from the test might work, but it has a few drawbacks:

  • It will be harder to debug – when running a test, we’ll have to attach a second visual studio to the iis express process to debug.
  • It will be harder to manage – we will need to make sure we stop iis express after the tests stop running – or we’ll need a separate clean-up process. Otherwise, we’ll end up with many instances of iis express running.
  • If multiple CI builds are running in TFS, how would we manage the ports with multiple IIS express versions running?

His points are valid and sinks the idea of using IIS Express to run the Web Worker roles.

He points to a postby Chad Meyers who spun up MVC by, for the most part, mocking ASP.NET.

Our web roles are wrapped in an IHttpHandler. This gives us the flexibly to test the code outside of Azure. In theory, we can spin up an instance of ASP.NET, using Chad Meyers’s method in the above post and drop in our web role HttpHandlers and test our code.

Windows Azure: The Testing Story

Testing code outside Azure is a pain in the ass — my company is struggling with this issue. We are currently testing against Azure itself and are encountering issues: First, the tests are taking longer and longer, the latency between Azure and our network is too great. Our complete test suite takes 2 to 3 hours to run. Secondly, data is co-mingled. Unless we create a Queue for every test, the tests are not atomic. Creating a Queue for each test presents another problem: the management of Queue creation and deletion.

Ultimately we decided the optimal course is to mock and run the Azure tests locally.

Web Roles
Under the covers Web Roles uses Hostable Web Core with an Azure Wrapper. IIS Express is also a wrapper for Hostable Web Core. It is possible to launch IIS Express from a test and then make a request to it and get a response back. There are drawbacks: First, with IIS Express a port can not be used concurrently. With thousands of tests this could become an issue. Second, IIS Express requires the full path to the root of the web role. Third, it requires we take a dependency on IIS Express.

In a proof of concept, from a test, IIS Express started up, the test made a request to IIS Express, IIS Express gave a response and the test shutdown IIS Express.

IIS Express Code (adapted from Reimers.dk):

using System;
using System.Diagnostics;
using System.Threading;

namespace IISExpress
{
    public class WebServer : IDisposable
    {
        private readonly string _appLocation;
        private readonly string _port;
        private Process _iisProcess;
        private Thread _thread;

       public WebServer(string appLocation, string port)
       {
           _appLocation = appLocation;
           _port = port;
       }

       public void Start()
       {
           _thread = new Thread(StartIISExpress) { IsBackground = true };
           _thread.Start();
       }

        private void StartIISExpress()
        {
            var startInfo = new ProcessStartInfo
                                {
                                    WindowStyle = ProcessWindowStyle.Hidden,
                                    ErrorDialog = false,
                                    LoadUserProfile = false,
                                    CreateNoWindow = false,
                                    UseShellExecute = false,
                                    Arguments = string.Format("/path:"{0}" /port:{1}", _appLocation, _port)
                                };

            var programfiles = string.IsNullOrEmpty(startInfo.EnvironmentVariables["programfiles"])
                                   ? startInfo.EnvironmentVariables["programfiles(x86)"]
                                   : startInfo.EnvironmentVariables["programfiles"];

            startInfo.FileName = programfiles + "IIS Expressiisexpress.exe";


            try
            {
                _iisProcess = new Process {StartInfo = startInfo};

                _iisProcess.Start();


            }
            catch
            {
                _iisProcess.Dispose();

                throw;
            }
        }

        public void Dispose()
        {
            if (_iisProcess != null)
            {
                _iisProcess.Kill();
            }
        }

        public void Stop()
        {
            Dispose();
        }
    }
}

Worker Roles
There are two components that need mocking:the Azure Queue and the Azure Table storage. Currently the ad server components are coupled to the Azure implementation. Mocking these components and using dependency injection would be a big step in decoupling the ad server components from Azure.

Windows Azure Storage: TDD and Mocks
http://blogs.southworks.net/fboerr/2010/07/23/windows-azure-storage-tdd-and-mocks/
http://compositecode.com/2011/01/18/windows-azure-unit-testing-dilemma/

As of this post, we have not implemented anything around Azure testing.

Refactoring Our Development Process

At work we’ve been improving our development process. We are using team system as our source control and CI server. Our production servers reside in Azure, but our development environment is local. We are finding pain points with trying to run tests locally with web workers. The team brainstormed and came up with some area’s to improve the process. I’ve been tasked with gathering a list of processes and tools to help us with the refactoring of the development environment.

Are there any open source projects that will help us run more deployments side by side?

Octopus – a deployment framework that leverages the NuGet packaging system for deployments

What logging frameworks exist and what should we use?

Log4Net – Has a known file locking issue.
Enterprise Library – Microsoft’s solution is a beast to configure.
The Object Guy Logging Framework – Don’t know anything about this one.
nLog

What db deployment/migration tools are there and what should we use?

RedGate Products
Visual Studio Database Projects
Migrator DotNet http://stackoverflow.com/questions/504909/database-deployment-strategies-sql-server
DBGhost
Database Versioning

Testing
Gallio – Open source framework that will run multiple testing frameworks.

Hypersonic 2.1 Released!

A new version of Hypersonic was release today!

A new feature was added to help retrieve Identities from the insert. The headache of declaring and out parameter in C# is no longer.

public int Insert(User user)
{       
		List parameters = database.GetParameters(User);
 		parameters.SelectIdentity();

		_database.NonQuery("User_Insert", parameters); 
		return parameters.Identity(); 
}

The C# Code Generator for Hypersonic has been updated to support the new out parameter feature.