Parallel Unit Testing with PNUnit

One of the great things about the coming version of NUnit is the integration of PNUnit - the parallel Unit Testing tool developed by codice.com.

With PNunit, you can distribute your tests across machines and test it for different load and stress scenarios. This distributed testing mechanism also helps in performing "smoke tests". The other advantages, to keep it brief, is that the team that uses PNUnit need not learn any new scripting language and can extend on its existing skill sets. This is also why PNUnit is called as NUnit Extensions.

The parallel execution of testsis possible through a configuration based framework where the configuration file (with an extension of ".conf") tells the framework the assembly name, the test fixture name, the machine ip, the port number - much like how a remoting configuration file would do.

The PNUnit framework is enabled by two components - an Agent that listens on a certain port and a Launcher, which launches the Unit Tests as specified in the .conf file.

The ease of use is further enhanced with the help of the familiar C# / NUnit type syntax. The example below shows the execution of the tests on two machines.

Copy the source code and build. The output assembly should be "pnunittests.dll" or even if it is not, make sure to change the name in the ".conf" file.

Copy the configuration code below into a file called "pnunittests.conf" and place all these files into the "pnunit" folder.


// PNUnitTests.dll

using System;
using System.Threading;
using NUnit.Framework;
using PNUnit.Framework;

namespace PNUnitTests
{

[TestFixture]
public class Testing
{
private string[] testParams;
[SetUp]
public void initTests()
{
testParams = PNUnitServices.Get().GetTestParams();
}
[Test]
public void FirstTest()
{

PNUnitServices.Get().InitBarrier("BARRIER");
// wait two seconds
System.Threading.Thread.Sleep(2000);
PNUnitServices.Get().WriteLine(
string.Format(
"FirstTest started with param {0}",
testParams[0]));
PNUnitServices.Get().EnterBarrier("BARRIER");
Assert.AreEqual(Convert.ToInt32(testParams[0]), Cmp.Add(15, 4));
}
[Test]
public void SecondTest()
{
PNUnitServices.Get().WriteLine(
"Second test will wait for first");
PNUnitServices.Get().InitBarrier("BARRIER");
// will wait for the first test
PNUnitServices.Get().WriteLine(
"First test should be started now");
Assert.AreEqual(Convert.ToInt32(testParams[0]), Cmp.Add(15, 4));
}

}
}

//Distributedtest.dll

using System;
using System.Threading;
using NUnit.Framework;
using PNUnit.Framework;

namespace DistributedTest
{
[TestFixture]
public class Class1
{
[Test]
public void TestOnAnotherMachine()
{
string[] testValues=PNUnitServices.Get().GetTestParams();
Assert.AreEqual(testValues[0], "Hello World!");
}
}
}

// pnunittests.conf
<?xml version="1.0" encoding="utf-8" ?>
<TestGroup>
<ParallelTests>
<ParallelTest>
<Name>SimpleTest</Name>
<Tests>
<TestConf>
<Name>FirstTest</Name>
<Assembly>pnunittests.dll</Assembly>
<TestToRun>PNUnitTests.Testing.FirstTest</TestToRun>
<Machine>localhost:8080</Machine>
<TestParams>
<string>19</string>
</TestParams>
</TestConf>
<TestConf>
<Name>SecondTest</Name>
<Assembly>pnunittests.dll</Assembly>
<TestToRun>PNUnitTests.Testing.SecondTest</TestToRun>
<Machine>localhost:8080</Machine>
<TestParams>
<string>19</string>
</TestParams>

</TestConf>
<TestConf>
<Name>DistributedTest</Name>
<Assembly>distributedtest.dll</Assembly>
<TestToRun>DistributedTest.Class1.TestOnAnotherMachine</TestToRun>
<Machine><your ip>:8080</Machine>
<TestParams>
<string>Hello World!</string>
</TestParams>

</TestConf>

</Tests>
</ParallelTest>
</ParallelTests>
</TestGroup>



PNUnit uses a older version of NUnit so use the same NUnit framework version that comes bundled with the PNUnit binary.

You need the PNUnit, .Net 2.0 or above framework and machines to be in the same network for the code to work. Provide for the appropriate ip and port number in the "pnunittests.conf" file.

Download PNUnit and extract the binary to a location. Compile and place the test assemblies(the test code posted above) in the same folder as PNUnit extracted files.

Open Command prompt, change directory to the PNUnit folder. Run "Start Agent agent.conf". The Agent window will open; next, Run "Launcher pnunittests.conf" file, the configuration file provided above.

For testing the "distributedtest.dll" on the remote machine, extract the "PNUnit" binary in that machine, place the .dll file into the same folder as in the local machine and start ONLY the agent in the remote machine. Run "Launcher pnunittests.conf" from the local machine. Default port used in agent.conf is 8080.

Happy Parallel Testing!

ExpectedException - NotImplementedException

Recently, i got to thinking about how well suited the NotImplementedException is for Test-First approach of software development.

In Test-First or Test-Driven approach, you are required to write your tests first before writing any line of code.

For example, if you were to provide for a Calculator class, it is required that you write tests for the class first. But then, how do you write tests for a class whose operations do not exist? Simple. You test for the NotImplemented Exception!

There are many techniques available in NUnit. For instance, the Assert.Throws and .DoesNotThrow are good ways of asserting for an exception or you can simply use the [ExpectedException] attribute and decorate the test method so that the test knows that if NotImplementedException is thrown that is the right behavior of the test!

Below is a simple Hello World example demonstrating the usage of ExpectedException attribute alongwith the usage of the TestDelegate type in NUnit and writing to the Console (TextOutput pane) of the NUnit GUI.

using NUnit.Framework;
using System.Diagnostics;
using System;
namespace NUnit251_Tests
{
[TestFixture]
public class Class1
{
private WriteToNUnitConsole traceListener;
private SayHelloClass obj;
public class WriteToNUnitConsole:TraceListener
{
public override void Write(string message)
{
//throw new NotImplementedException();
Console.Write(message + "from NUnit");
}
public override void WriteLine(string message)
{
throw new NotImplementedException();
}
}
[SetUp]
public void InitTestData()
{
if (!Trace.Listeners.Contains(traceListener)){
traceListener=new WriteToNUnitConsole();
Trace.Listeners.Add(traceListener);
}
}
[TearDown]
public void DestroyTestData()
{
Trace.Listeners.Remove(traceListener);
}
[Test]
public void TestSayHelloWithWrite()
{
obj = new SayHelloClass();
Assert.DoesNotThrow(() => Trace.Write(obj.SayHello()));
}
[Test]
[ExpectedException("System.NotImplementedException")]
public void TestSayHelloWithWriteLine()
{
obj = new SayHelloClass();
obj.SayHelloThrowsException();
// Assert.Throws(typeof(NotImplementedException), (() => obj.SayHelloThrowsException()));
}

}
}

using System.Diagnostics;
namespace NUnit251_Tests
{
public class SayHelloClass
{
public string SayHello()
{
return "Hello World!";
}
public void SayHelloThrowsException()
{
//Trace.WriteLine("This method is not implemented");
throw new System.NotImplementedException();
}
}
}

As usual, i have commented out lines for you to try out different combinations.

This code has two tests - one commented and the other live. When both are uncommented, one test will pass and the other fail.

Happy TDD!

Agile NCR Conference 2009

I presented on "ROI with Agile" and the session went much better than I had anticipated.

The conference, itself, was organized as well as last year and soon, we should probably see the NCR conference going international. Its popularity and enthusiasm generated by this conference is engaging and infectious.

My colleague, a first time attendee to the conference, was eager to check the budget of such a conference so tht our company could host it as well!

The organizers, Xebia, were gracious and the sponsors, Ericsson, were excellent in their support. The presentation by Hedwig Baars was excellent as was the participation of the Ericsson employees in the conference.

In all, a highly successful conference and after this conference, i genuinely believe that i, my colleagues at the conference and my company can go Agile, with total confidence.

Generating test cases from an external class

The TestCaseSource attribute allows you to specify a method or a property, which could act as the source for your test cases. Simple and effective as TestCase attribute is TestCaseSource attribute is better.

A simpe rule that you must remember is that the class that contains a source method for the testCaseSource attribute must have a default constructor defined for it unlike the TestCase attribute.

// Class that uses another class' method as the source for its test method using the TestCaseSource attribute and taking the type and the name of the method as its arguments.

[TestFixture(21)]
public class TestCaseFromAnotherClass
{
private int expectedValue;
public TestCaseFromAnotherClass(int x)
{
expectedValue=x;
}
[Test,TestCaseSource(typeof(Enumerator_GeneratesData), "GenerateData")]
public void TestCaseSourceTest(int arg1, int arg2)
{
Assert.That(arg1*arg2,Is.EqualTo(expectedValue));
}
}

// Class that contains the source method for the TestCaseSource attribute and hence, must have a default constructor.

[TestFixture(42)]
public class Enumerator_GeneratesData
{
private int expectedAnswer;

public Enumerator_GeneratesData() { expectedAnswer = 42; }
public Enumerator_GeneratesData(int val) { expectedAnswer = val; }
public IEnumerable GenerateData()
{
for (int i = 1; i <= expectedAnswer; i++)
if (expectedAnswer % i == 0)
yield return new int[] { i, expectedAnswer / i };
}

//
[Test, TestCaseSource("GenerateData")]
public void TestGeneratedData(int x1, int y1)
{
Assert.That(x1 * y1, Is.EqualTo(expectedAnswer));
}

}