What's new in ApprovalTests.Net v3.7

[Available on Nuget]

AsyncApprovals - rules and exceptions

[Contributors: James Counts]

In the end, all tests become synchronous. This means for a normal test we recommend

	/// DON'T USE ASYNC
	[TestMethod]
	public async Task TestHttp()
	{
	var response = await new HttpClient().GetAsync("http://www.google.com/");
	Assert.AreEqual(response.StatusCode, HttpStatusCode.OK);
	}
	/// Prefer to use .Result
	[TestMethod]
	public void TestHttp()
	{
	var response = new HttpClient().GetAsync("http://www.google.com/").Result;
	Assert.AreEqual(response.StatusCode, HttpStatusCode.OK);
	}

view raw gistfile1.cs hosted with ❤ by GitHub

However, If you are looking to test exceptions everything changes and you might want to use

	[Test]
	public void TestAsyncExceptions()
	{
	int a = 0;
	AsyncApprovals.VerifyException(async () => 1/a);
	}
	// NOTE: If async is creating inconsient stack traces, consider a scrubber
	AsyncApprovals.VerifyException(async () => 1/a,
	ScrubberUtils.RemoveLinesContaining("System.Linq.Parallel.QueryTask"));

view raw gistfile1.cs hosted with ❤ by GitHub

Removed BCL requirement

[Contributors: James Counts &  Simon Cropp]

HttpClient is nice way of doing web calls in .Net. Unfortunately, at this time the BCL in nuget does unfortunate things to your project if you do not wish to use the HttpClient. 

This is a violation of a core philosophy of approvaltests 

"only pay for the dependencies you use" 

HttpClient was add ApprovalTests 3.6. Thanks to Simon for pointing and troubleshooting this error. It has now been removed. 

Wpf Binding Asserts

[Contributors: Jay Bazuzi]

This is a bonus from v3.6

It is a very hard thing to detect and report Wpf Binding Error. To even get the reports to happen you have to fiddle with the registry and then read and parse logs.

No More!  Now to you use BindsWithoutError to ensure that your Wpf binding are working.

	Binding myBinding = new Binding(TestViewModel.MyPropertyPropertyName);
	WpfBindingsAssert.BindsWithoutError(viewModel, () =>
	{
	// ... create control and initalize bindings
	// (This will happen once the reporting and logging has been setup)
	var textBox = new TextBox();
	textBox.SetBinding(TextBox.TextProperty, myBinding);
	return textBox;
	});
	/************************************************
	If this fails you will get an exception like:
	System.Windows.Data Error: 40 : BindingExpression path error: 'MyProperty' property not found on 'object' ''TestViewModel' (Hashcode)'.
	BindingExpression:Path=MyProperty
	DataItem='TestViewModel' (Hashcode)
	target element is 'TextBox' (Name='')
	target property is 'Text' (type 'String')
	*************************************************/

view raw gistfile1.cs hosted with ❤ by GitHub

What's new in ApprovalTests.Net v3.6

I finally got a new version released. It's been a lot time in the making and I wanted to highlight some of the new feature. There's a bunch so hold on tight:

VerifyNoAbandonedFiles

One issue that pops up with approvals is sometimes old approval files end up abandoned in the source tree. Now you can write a simple test that will prevent this

	[Test]
	public void EnsureNoAbandonedFiles()
	{
	ApprovalMaintenance.VerifyNoAbandonedFiles();
	}

view raw Maintenance.cs hosted with ❤ by GitHub

Asp Scrubbers

It's great that ApprovalTests can so easily capture web pages, but it can still be hard to get those pages as consistent as a Unit test needs them to be. Inspired by Geoff Bache's TextTest, scrubbers have been included.

Scrubbers are simple Functions that take a input string and return a clean version of it. Here's an example to remove the GUID used by asp for forms

	[Test]
	public void TestInternationalization()
	{
	AspApprovals.VerifyUrl("http://localhost:1359/OrderForm", ScrubAspViewstate);
	}
	public static string ScrubAspViewstate(string input)
	{
	string AspViewState = "<input type=\"hidden\" name=\"__VIEWSTATE\".+/>";
	return Regex.Replace(input, AspViewState, "<!-- aspviewstate -->");
	}

view raw gistfile1.cs hosted with ❤ by GitHub

SimpleLogger.Mark

SimpleLogger is a easy way to both capture behavior for locking test as well as see performance data for simple profiling but it was also hard to get everywhere a method would exit. Now you don't have to, simply use a using around the method like this.

	public void MyMethod()
	{
	using (Logger.MarkEntryPoints())
	{
	if (guardClauss){return ;}
	// other logic
	}
	}
	Will produce
	6/18/2014 3:43:06 AM ~000008ms => ClassName.MyMethod()
	6/18/2014 3:43:06 AM ~000004ms <= ClassName.MyMethod()

view raw gistfile1.cs hosted with ❤ by GitHub

StatePrinterApprovals

Locking down legacy code can be hard when none of the objects have a good ToString. Enter StatePrinterApprovals. Using the nuget package StatePrinter you can easily print and capture even highly complex and recursive object graphs.

	public void TestVerifyCircleReferences()
	{
	StatePrinterApprovals.Verify(anyObject);
	}
	Can produce output like
	Root = <Turtle>, ref: 0
	{
	Name = "jim"
	On = <Turtle>
	{
	Name = "kasper"
	On = <Turtle>
	{
	Name = "llewellyn"
	On = -> 0
	}
	}
	}

view raw gistfile1.cs hosted with ❤ by GitHub

Rest Calls 

Finally there is a simple object to create your own Rest Calls that are both easy to Mock (because they implement ILoader) and easy to Test. Here's the Testing Logic

and here the code to test it.

	[Test]
	public void TestGoogleQuery()
	{
	Approvals.Verify(new GoogleQuery("lolcats"));
	}
	// Make sure your RestCall extends RestQuery<T> where T is what is returned from the call
	class GoogleQuery : RestQuery<GoogleQueryResults>

view raw gistfile1.cs hosted with ❤ by GitHub

VerifyJson

It's always been easy to verifyJson, but now it will be pretty printed as well

	Approvals.VerifyJson(jsonText);
	///Produces
	{
	"GivenNames": "John",
	"FamilyName": "Smith",
	"Spouse": "Jill",
	"Address": {
	"Street": "1 Puddle Lane",
	"Suburb": null,
	"Country": "USA"
	}
	}

view raw gistfile1.cs hosted with ❤ by GitHub

Serializable Theory Test

Using Theory based testing, you can now test that any object can be serialized and consumed to recreate the original object. It's as simple as 

SerializableTheory.Verify(anyObject, Assert.AreEqual);

view raw gistfile1.cs hosted with ❤ by GitHub

WindowRegistry.Asserts

You can also easily assert that WindowsRegistry Values are in place with

WindowsRegistryAssert.HasDword(keyName: "Console", valueName:"TrimLeadingZeros", expectedValue:0,failureMessage: "You should probably Have This");

view raw gistfile1.cs hosted with ❤ by GitHub

WriterFactory

ApprovalTests is more extensible with the introduction of WriteFactory. An easy way to insert your own FileWriters if you need them

ApprovalDataScenarios

Finally, the last big change is the introduction of DataScenarios. This allows you to use data driven test cases to produce multi approved files.

	[Test]
	[TestCase("foo", "bar")]
	[TestCase("jack", "jill")]
	public void Transformer(string a, string b)
	{
	using (ApprovalResults.ForScenario(a,b))
	{
	Approvals.Verify(Transform(a,b));
	}
	}
	// Will Produce
	Class.Transformer.ForScenario.foo.bar.approved.txt
	Class.Transformer.ForScenario.jack.jill.approved.txt

view raw gistfile1.cs hosted with ❤ by GitHub

What’s new in ApprovalTest.Net v 3.0?

ApprovalTests.Net v 3.0 is here, you can get it on Nuget.
Here’s what’s new:

Version 3.0 – The final version

Versioning with signed packages is always a bit of a pain. It’s made things harder to deal with when I release new versions. Thanks to Simon Cropp, I was introduce to JSON.net's model for version numbers which solves it. You can read all about it here, but here’s the short version

• ApprovalTests & ApprovalUtilities will be v 3.0 from here on out.
• Nuget Version numbers will be actual version, and will continue to evolve.

[Contributors:  Simon Cropp]

GitHub improvement & bug fixes

One of the nice things GitHub provides is issue tracking.  As such many issues suggested where fixed including:

• Rdlc Reports with External Images 
• Approvals.VerifyPdf(file) 
• Xunit & Mspec Support

Please continue to add issues as you find them.
[Contributors: James Counts & Simon Cropp]

HtmlScrubbing

I found that asp.net pages were giving me some issues with generated keys, so I added a scrubber you can use if you need to. You can view the code here.

.gitattributes suggestions

Line endings are a pain for everyone, but git’s solution of switching them makes things worse for ApprovalTests. Fortunately, Jake Ginnivan came up with a very nice solution of setting the approved files to binary with removes this problem.  Simply add:

      *.approved.* binary

to your .gitattributes
[Contributors: Jake Ginnivan]

What I've learned about open source by pairing with Simon Cropp

Over the last 2 weeks I have be fortunate enough to pair with Simon Cropp for about 8 hours on my open source project ApprovalTests.  Simon has taught me a lot about running a better open source project, this blog is an attempt to share some of that for those not fortunate enough to be able to pair with Simon themselves.

Think about your 'brand'

Often I am writing ApprovalTests because I use ApprovalTests myself. I am happy that others find it useful as well and believe in sharing and open source, so I also make it an open source project. However, all of my thinking goes into after a programmer has it on their computer. However, there is a lot of things that happen before someone has ApprovalTests on their computer. Most of what I am about to talk about is specific to this area of open source. In the 8 hours we paired, we spent surprising little time actually programming C#.

It works on my computer

I believe this is particularly harder with .Net than other languages. There were many things that I just assumed everyone had because I never thought I did a special installed for them. Some of these were libraries that were put in my Global Assembly Cache (GAC)  by VisualStudio (2010) but not by (2012). That means adding .dll's which means configuring git.

Git and binary files

I really wish then was a simple header to all files indicating that they are a binary or text file. There isn't. This becomes an issue with git, so if you are checking in binary files, you need to set your .gitattributes to include them. For example,  I added the following to my .gitattributes:

#Binary

*.dll  binary

*.exe  binary

*.png  binary

*.tif  binary

*.tiff binary

*.snk  binary

*.ttf  binary

If you don't do this as soon as someone clones your repo, there will already be changes.

Removing Cruft 

I'm pretty good about cleaning my code, but what about my project in general? All though references you don't use anymore? That Silverlight experiment? Even dll's that are in your project but no longer referenced by your project. All of that makes it harder for someone else to clone and play with your source.

Consistency

This project started before nuget, so there were dll's that were added directly that now should be added by nuget. It's hard to remember the advantage of consistency in a project, especially as it is not bothering you. This is a double edged sword with pairing. On the one hand it's easy to code because you have your environment setup plus already know the tricks. On the other hand, it masks trouble that a lone programmer will have with your source.

Version Numbers and Signed Code

This is a long post, but the short version is that in the end I followed JSON.net's model for version numbers. This means that while the nuget version number will increase, the version number for ApprovalTests will now remain at 1.22 for all time. Because I sign ApprovalTests, this allows me better compatibility with other software.

Version Numbers & Simple Builds

An other problem was getting a single place for the version numbers. This required a bit of awesome hacking on Simon's part.  Here's the high level view

1. create a solution level file for the version number
2. add it to packages via link (there is a drop down on add)
   
3. Manually change your .csproj file to move it into your properties folder
   Properties\VersionInfo.cs
4. Use this when creating your nuget with the -Version flag

If any of this was confusing checkout these files from my project: VersionInfo.cs , ApprovalTests.csproj, ApprovalTests.CSharp.nuspec, CreateNuget.cmd

1 .dll per Nuget

Having more than one dll file in a nuget package is a bit counter productive since you now can't use only 1 of them. In my case I had 2, so now I have two nuget packages: ApprovalTests & ApprovalUtilities. Of course, ApprovalTest depends on ApprovalUtilities, but you can now use the Utilities on it's own, as originally intended.

Think long term

On the Internet things live forever. Whenever possible link to a domain you control that can move with you. When I started everything was on sourceforge, now I have moved to github and links I have posted will be broken because of that. That history that creeps into your project isn't just in you code. Your brand is the full experience. Blogs, videos, stackoverflow questions, packages, repos, code, api's. Try to make it still work 5 years from now.

More to come...

This blog is already getting pretty long, and there is still much more Simon as got me thinking about. Here's my current todo list:

1. Continuous Integration Server - I'm a big believer in this on regular projects but it seemed like overkill when I was the only committer. I just had to remember to always run my tests. But there is an other advantage here. I keeps me aware of at least 2 machines, meaning it will be easier to think of other people who clone the repo.
2. Website - On top of moving the website to github, and using a static website generator like jekyll. I am also aware of needing to structure my website to guide people to Understanding the Code, Using the Code, and Browsing the Code. A very nice example of this in the NancyFx Page.
3. Documentation - This includes what goes in the blog vs into a wiki. The main difference being that wiki should be relevant next year, while a blog is relevant NOW and not so important next year.

Again, a super big thanks for all the help Simon has given me. I have paired with more than 50 people across the globe and all of them have been very helpful to me. Simon had a particular gift for understanding the branding of an open source project and was very patience working with me to improve ApprovalTests.

What's new in ApprovalTests.Net v.22 ?

ApprovalTests.Net v.22 is here, you can get it on Nuget
Here's what's new:

UTF8 Support

[Contributors: Scott Reed & Maurice De Beijer] 

Text Files will now work with UTF with (many/most) Diff Tools. This basically comes down to making everything UTF8, and including the bom (byte order mark) header in the text files.  

**Backwards compatibility break**

This change means version22 of ApprovalTests will break compatibility with previous versions.

To  adjust for this, when you run your tests, ApprovalTests will check the .approved file and add the UTF8 bom if needed. This might result in you needing to re-check-in your approved file, despite an apparent lack of change.

GitHub

[Contributors: James Counts & Jason Jarrett]

While not strictly a feature of v.22, this version marks the move of the source to GitHub. (https://github.com/approvals/ApprovalTests.Net/) This should make it easier for people to branch and play around with the code. Although my preferred method of collaboration is still a remote pair programming session via skype. If you are interesting in pairing just ping me on twitter @LlewellynFalco 

ORM Support

[Contributors: Scott Reed] 

We have added  the Interface DatabaseAdaptor, which makes it much easier to do IExecutableQuery type approvals. This is based on the ‘Testing the Weather’ pattern. 

This goes along with added support for:

• NHibernate
• Entity Framework DbContext (V 4.1+) 

Async

[Contributors: Jake Ginnivan] 

XUnit supports testing of methods using the async and await keywords. Now ApprovalTests supports it as well. Asynchronous parallel programming offers many advantages with both IO and multicore, and it’s great to see this paradigm opening up so nicely in the .net world. 

P4Merge and better path finding

[Contributors: Jason Jarrett]

It’s always been a source of problems detecting where different systems have placed the Diff Reporter programs ApprovalTests uses.  Thanks to the addition of “where.exe” on windows7 and above, if the reporter is in your system path, it will now be found. This comes with the addition of P4Merge in the reporter options. P4Merge will also diff images giving us yet another excellent option. 

CLS complaint

[Contributors: James Counts]

This is not the same as a portable library (that’s also on the list) but it does help with language portability. Meaning if you are using .Net but not using C#, ApprovalTests is looking out for you. 

Contributors

A big thanks to everyone who contributed! I could not have done a ¼ of what I have if it wasn’t for the people who have paired with me (They have also allowed 2 new ports of ApprovalTests in the last iteration as well to Node.js & Perl).

ApprovalTests has always relied on the pairing of contributors, and I was remiss for not being more vocal about it in the past. Thanks again for all your help.

What’s New in Java ApprovalTests V 0.12

In this release of ApprovalTests (available here ), we’ve only added one thing, but it’s a big thing. Specifically, it’s a big data thing.
My thanks go out to John Heintz for helping me to create this. 
Here's a video of the whole thing:

Unit Testing HadoopMapReduce

Before we go into the details of everything involved in HadoopMapReduce, let’s go through an overview of how we’ve been testing our MapReduce jobs. In this example, we are doing this common demonstration of a word count MapReduce job. The idea is that sentences will come in, they will be split into words, count them up, and summarize in our reduce job.

Visualizing the System

Unit Testing provides many different advantages to a programmer. And all of them have different importance to different people, but one of these advantages is specification. In particular, with MapReduce, being able to understand the flow and transformation of the data can be more enlightening than the other aspects of Unit Testing.
One of the things that we strive to do with ApprovalTests, is create output that is meaningful and insightful. To demonstrate this, I’m going to start output of my word count unit test.

WordCountTest.testMapReduce.approved.txt

[cat cat dog] -> maps via WordCountMapper to -> (cat, 1) (cat, 1) (dog, 1) -> reduces via WordCountReducer to -> (cat, 2) (dog, 1)

As you can see, this output is a nice way of visually understanding what comes into the job, how the WordCountMapper transforms it, and what the final result is. Let’s take a look at the code that produced this result, and then we’ll break it down into individual segments.
Here is the entire line to create that output:

 HadoopApprovals.verifyMapReduce(new WordCountMapper(), 
                                 new WordCountReducer(), 0, "cat cat dog");

Easy right? Let’s look into the parts.

MR Unit

HadoopApprovals sits on top of MR Unit . You will need to grab it and include the jars. The javaDocs for HadoopApprovals mention all of the jars needed to run it. They are:

• hadoop-client-1.0.3.jar
• hadoop-core-1.0.3.jar
• hadoop-test-1.0.3.jar
• log4j-1.2.15.jar
• mockito-all-1.8.5.jar
• mrunit-0.9.0-incubating-hadoop1.jar

HadoopApprovals

HadoopApprovals has three main functions: the ability to test the mapper, the reducer, and the map reduce combination. To make this easier, we are going to use extra information with the generics called SmartMappers and SmartReducers. We will talk about those in the next section.

Testing a Mapper

Here’s the method to test the mapper:

verifyMapping(SmartMapper mapper, Object key, Object input)

To test a mapper, you need the mapper you’re going to test and the keyValue pair going in. Many times, mappers do not actually use the key that comes in, but regardless, you will need one anyway. Do not make it null.

Testing a Reducer

Here’s the method to test the reducer:

verifyReducer(SmartReducer reducer, Object key, Object... values)

When testing a reducer, you will need to pass in the key plus a set of values for the key. You will notice you can pass in regular strings and regular integers instead of Text and LongWritable. Again, this is because of the SmartMappers we will talk about in the next section.

Testing a full MapReduce job

Here’s the method to test the MapReduce:

verifyMapReduce(SmartMapper mapper, 
                SmartReducer reducer, Object key, Object input)

As you might expect, this is almost identical to testing a mapper. Except with the extra addition of a reducer.

SmartMappers

I wrote a blog about getting generic type information at runtime from Java. You can read it here:

The long a short of it is, if you add an extra interface to your mapping and reducing jobs, you can avoid a lot of boiler-plate code needed to state what the runtime types of the input and output are. You can do this by simply having your MapReduce jobs extend SmartMapper instead of Mapper.

“What? I don’t want to change my stuff!”

Fair point.  If you want to test against an existing mapper or reducer, and you don’t want to change it to use the SmartMapper extensions, you can always wrap it using the MapperWrapper (bonus:  fun to say out loud) or ReducerWrapper. Here’s a sample of how to do that:

SmartMapper mapper = new MapperWrapper(new WordCountMap(),
                                       LongWritable.class,
                                       Text.class,
                                       Text.class,
                                       LongWritable.class);

Once you do this, you will no longer need to wrap all your inputs in the writable classes, HadoopApprovals will wrap primitives into the appropriate context for you.

What’s new in ApprovalTests.Net v.20?

ASP routing support

As community contributor, Krzysztof Koźmic  said in his NDC talk …

Using ApprovalTests makes what was implicit,
explicit so that the item can now be tested

Inspired by his talk, Jim Counts and I created a new method in AspApprovals to make the formerly implicit ASP routing explicit and now easily able to be tested.  Here’s an example

var urls = new[] {"/Home/Index/Hello", "/"};
AspApprovals.VerifyRouting(MvcApplication.RegisterRoutes, urls);

In the sample above, the delegate (MvcApplication.RegisterRoutes) will be invokes against a mock route collection, and then fed the URLs provided.  A sample output would look like this:

/Home/Index/Hello => [[controller, Home], [action, Index], [id, Hello]] 
/ => [[controller, Cool], [action, Index], [id, ]]

Event approvals

Similar to the example above, another aspect of code which is often hidden (or implicit) is which events are wired-up to your form.  This can be particularly troublesome when changing an existing, unfamiliar piece of code.  For example, it it easy to accidentally remove a button-click event.

Event approvals allows you to easily lock down existing events which are associated with an object or a form.  To do this, simply call the following:

EventApprovals.VerifyEvents(myObject);

This new method will only allow you to verify events which are directly associated to the object which is passed in.  Unfortunately, many times you don’t want to say ‘What are the events associated with this button"? rather you actually want to say ‘What are the events associated to this form, even though the button is on the form and the event is associated to the button (rather than the form)?’  This is now no problem, since we took this scenario into consideration and also created a convenience method for the latter scenario.

To test all the events on a form and its immediate children, simply call code as shown below:

WinFormsApprovals.VerifyEventsFor(new DemoForm());

WPF support for Controls

It’s always been possible to test WPF windows with a simple:

WpfApprovals.Verify(window);

This would render the window to a .png image and verify against .approved file.

But sometimes you want to scope smaller to an individual control. Unfortunately, we had previously neglected this scenario – no longer!  You can now test an individual WPF control with the same call as shown below:

var menu = new ContextMenu();
menu.Items.Add(new MenuItem() {Header = "Add Element"});
menu.Items.Add(new MenuItem() {Header = "Delete"});
menu.Items.Add(new MenuItem() {Header = "Edit"});
menu.IsOpen = true;
WpfApprovals.Verify(menu);

This code will produce a *.png image, such as the one shown below:

Entity Framework support

People have long pondered how to test Entity Framework.  We have talked about this before, but previously testing EF with ApprovalTests had always required a large number of steps.  Now you can test EF by simply calling

using (var db = new ModelContainer())
{
    EntityFrameworkApprovals.Verify(db, CreateCompanyLoaderByName2(db, "Mic"));
}

This method will implement a ‘test-the-weather’ scenario which will verify the resulting SQL without the need to be able to connect to a database.  However, if this method fails, then it will connect to referenced database and will execute the generated query and will return extra information about WHY the test failed.  I plan to create more detailed tutorials on the inner workings of this new method soon.

Let’s give you one more look at this method signature.

public static void Verify<T>(ObjectContext db, IQueryable<T> queryable)

Email support for attachments

Previously, we added the method

EmailApprovals.Verify(message);

Unfortunately, when the email message contained attachments, then the GUIDs that those attachments created caused issues with test output because they were not consistent over multiple test runs.  This bug has been fixed in this release.  To learn more about approving email messages, watch this video.

Testing Email with ApprovalTests

FileLauncher with DelayReporter

There has been a great deal of demand to use Cassini-dev after Jim Counts demo’d it during his ASPConf video on testing ASP.MVC views. Unfortunately, when using this method it’s a bit tricky to use a file launcher, since the web server dies BEFORE the browser can hit it.  We came up with a simple solution which adds a tiny delay into the method execution so that this is no longer an issue.

[UseReporter(typeof(FileLauncherWithDelayReporter))]

Well, that’s all for this release. As always, if you have any questions about ApprovalTests, tweet them with the hashtag #ApprovalTests, I monitor that and will answer you promptly.

Happy testing.