Matt Smith's Blog

Software Engineering related topics

Refactoring to Patterns Presentation

| Comments

I gave a presentation on Refactoring to Patterns last week to my colleagues here at Towers Watson. I thought I was capturing video during the presentation so forgive the black screen, but feel free to listen if you like.

Fencepost Testing

| Comments

Fence

Unit testing software can be difficult, especially when you are considering what to test. You should test the happy path that you expect most of your code to adhere to, but what else? I use a concept I call Fencepost Testing to help me determine if I have sufficient test coverage. The concept is simple, consider your code and identify the boundary conditions it should adhere to. If what you are testing has one boundary or a “fencepost” then test that post. If you find multiple boundaries then you have multiple “fenceposts”, you should test each post and a wildcard in between. The “fenceposts” being the make it or break it components of the subject you are testing.

So, for some code that checks if a date is in the future I would breakup my tests as such:

  • Some arbitrary date in the past
  • Yesterday (Fencepost - Immediate boundary)
  • Today (Fencepost)
  • Tomorrow (Fencepost - Immediate boundary)
  • Some arbitrary date in the future

Some would argue that the arbitrary dates would not matter, but they serve as sanity checks to make sure I’m noti, for example, using the month value alone. The arbitrary dates could be random, since they should never fail. If they did, well then you’ve caught a bug.

Consider the regex ^\$\d+$ which simply tests is a string is a dollar amount. A slightly more complicated item to test, with an infinite number of posibilities that would and would not match. But, there are some fenceposts:

  • Must start with $
  • Must have a digit following the $
  • Can have many digits there after
  • Last character must be a digit

So my test cases would be:

  • ’ $1’ (There is a whitespace character at the beginning)
  • ‘a$1’ (Non-whitespace character before $)
  • ’$’ (Nothing following the $)
  • ‘$a’ (Non-whitespace non-digit character after $)
  • ’$ ’ (Whitespace character after $)
  • ‘$1’ (Simple one digit happy path)
  • ‘$1234567890’ (Simple multi-digit happy path)
  • ‘$1 ’ (One digit with trailing whitespace)
  • ‘$1a’ (One digit with trailing non-digit character)
  • ‘$1a1’ (Digits with intermittent non-digit character)
  • ‘$1,000’ (Digits with comma separated non-digit character)

In essence this should cover most of the cases that could be applied to that regex. If you do find a bug that passes this regex enexpectedly then add a new test case.

Testing the posts alone, or solely the happy path is not different than building fenceposts with no fence between them.

Fence posts only

Fetching GitHub Pull Requests

| Comments

A co-worker was doing some work today to integrate our TeamCity instance integrated with GitHub pull requests. And it got me thinking, I hate adding remotes to my local repo just to pull down pull requests. The solution:

1
2
[alias]
  req = "!f() { git fetch origin refs/pull/$1/head:pr/$1; } ; f"

Usage:

1
2
3
4
5
6
7
8
$ git req 39
remote: Counting objects: 10, done
remote: Compressing objects: 100% (10/10) done.
remote: Total 10 (delta 0), reused 1 (delta 0)
Unpacking objects: 100% (10/10), done.
From github.com:DarthFubuMVC/StoryTeller2
 * [new ref]         reqs/pull/39/head -> pr/39
$ git checkout pr/39

And now I have that pull request locally without adding a remote.

Logstash Presentation

| Comments

I gave a presentation on Logstash at work to introduce my colleagues to what it can provide you.

Introduction to Bundler

| Comments

Over the last few years I have become a huge fan of using Rake as my build scripting language of source. It is rich in many aspects, which I don’t care to dive into right now. Since it runs on Ruby you have access to all the various gems already developed.

I recently started a new job at Dovetail Software, and they were already using Rake; however, they weren’t using Bundler. This is where my headache began which prompted this post. I proceeded to install all the gem dependencies manually by simply running gem install rake. The latest version at the time that I did that was 10.0.2 and the version that was previously developed against was 0.9.2.2. Although the differences between these two different versions was minimal, there was one error that I got simply because I was on a newer version then all of my colleagues.

Bundler was designed to mitigate this problem. Bundler is a ruby gem version management solution. The best part is it only requires two additional files to the root of you project directory, one which you write and the second that is generated by Bundler. I know I just said generated, which is generally a bad thing when it comes to code, just bear with me.

The first file to create is a file named Gemfile no extension type just exactly as I wrote it. This file is used to specify the acceptable gems and versions that you intend to use. You can find detailed instructions on the content of a Gemfile here.

After you have the Gemfile you can generate the second file I mentioned Gemfile.lock. This is created by running bundle install, this command will do one additional thing on top of creating the file. Since Bundler is a gem version management utility the command will also download the gems for the versions you specified, along with those gems dependencies. The Gemfile.lock file will contain a list of all the gems installed and the exact versions that were installed.

Here is where the magic begins. Commit those files and move over to your buddies computer. Have him pull down that commit, and run bundle install The Gemfile.lock will not be changed this time, instead Bundler will install the gems with the exact versions as specified in the Gemfile.lock. Now if you leave those gems alone for a while and you either hire a new employee, get a new computer, yatta yatta yatta. Run bundle install on that new computer and it will use those same versions, even if a newer version exists.

Now when the time comes that you wish to update a gem, just run bundle update <gem>. Bundler will modify the Gemfile.lock, commit this and all your colleagues will know what version you are now using.

If you have multiple projects using various different versions of gems Bundler is there to rescue you again. When running Ruby it will attempt to use the latest version of an installed gem. If you preface your command execution with bundle exec <command>, then Bundler will ensure that the versions specified in the Gemfile.lock will be the ones that are pulled in with a require statement.

Now you only have one additional thing to do, and you will never accidentally require in the wrong version of a dependency. Simply put require 'bundler/setup' at the beginning of you entry ruby script. Bundler will then fail if the versions specified in the Gemfile.lock are not installed. It will prompt you to run bundle install. Now when you update a dependency and commit it, your colleagues will not be able to accidentally run the updated scripts.

I don’t intend for this to be an all inclusive documentation on Bundler, you can find that at the official Bundler website here, but hopefully this will server as a launching point to get started with using it. It has been and will continue to be a great tool to ensure your entire development team is on the same page.

Event Queue for FubuMVC.ServerSentEvents

| Comments

Out of the box the event queue in FubuMVC.ServerSentEvents just stores all events in memory while the web application is running. While this will work great for many people it does have it’s limitations, namely that it is a built in memory leak. It was originally designed with the idea that those who used the library would override the event queue with their own implementation. There are a couple of things you can do here, you can either persist the events to long term memory, or build in a mechanism to reduce the events in the queue, or whatever else you would like to do. (Sound like the Fubu way of doing things or what!)

When you implement your own queue you will need to implement both the IEventQueue and IEventQueueFactory interfaces. Then register them in your fubu service registry like so:

Fubu Service Registry
1
SetServiceIfNone<IEventQueueFactory<TOPIC>, YOUREVENTQUEUEFACTORY>();

You will need to do this for each of your topics for which you would like your event queue to be used.

How you implement your event queue is entirely up to you. The solution that I implemented was an in memory queue which would self reduce when the number of events surpassed some count.

Event Serialization in FubuMVC.ServerSentEvents

| Comments

The IServerEvent.Data property is defined by the interface to be an object. Hence this object needs to be serialized prior to broadcasting it down to connected clients. By default the FubuMVC.ServerSentEvents bottle will use the FubuMVC JSON serializer. So an example stream would look like this:

Example of stream with serialized data
1
2
3
4
id: <EVENT ID>/<EVENT NAME>\n
data: { "property" : "value" }\n
retry: <RECONNECTION TIME>\n
\n

You can easily change the serializer by implementing your own version of IDataFormatter then in your registry add ReplaceService<IDataFormatter, YOURIMPLEMENTATION>()

This is where appending the event name to the end of the event id comes in handy. You can route all of your events to the default message handler on the client, deserialize the data, then trigger a non-SSE event with the deserialized data, similar to how everything else is handled in FubuMVC.

FubuMVC.ServerSentEvents OOTB Support

| Comments

FubuMVC now has support for Server Sent Events (SSE)! I have recently used it for our internal call center dashboard here at Extend Health. Since it’s inception we are now seeing data on the dashboard within seconds of it happening, as opposed to the previous 5 second ajax refresh interval we were doing before.

Out of the box (OOTB) the FubuMVC.ServerSentEvents bottle already does just about everything you need on the server. The only thing you need to implement is a topic for which your events will be coordinated against. A topic is created by inheriting the FubuMVC.ServerSentEvents.Topic class. The topic will be used to coordinate how events are stored and pushed to the connected clients. This is accomplished by overriding the bool Eqauls(object item) and int GetHashCode() methods. See an example below:

SimpleTopic.cs
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
using System;
using FubuMVC.ServerSentEvents;

namespace SomeNamespace
{
    public class SimpleTopic : Topic
    {
        public Guid SomeId { get; set; }

        public override bool Equals(object obj)
        {
            return Equals(obj as SimpleTopic);
        }

        public bool Equals(SimpleTopic other)
        {
            if (ReferenceEquals(null, other)
                return false;

            return ReferenceEquals(this, other) ||
                   SomeId.Equals(other.SomeId);
        }

        public override int GetHashCode()
        {
            return SomeId.GetHashCode();
        }
    }
}

When an event is published or when a client connects, the code will pass through a TopicFamily which is keyed by the topic object you define. A separate channel is created for each topic, which is where the method overrides come in. The diagram below is a BASIC flow of the how events are stored and retrieved:

Basic Flow

Now all you need to do is publish your events and connect a client. You publish events via the IEventPublisher. I will cover the event and how you can customize it in a future post. For now though the ServerEvent class is the basic OOTB event that you can use.

Publishing Events
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
using System;
using FubuMVC.ServerSentEvents;

namespace SomeNamespace
{
    public class PutHandler
    {
        private IEventPublisher _eventPublisher;

        public PutHandler(IEventPublisher eventPublisher)
        {
            _eventPublisher = eventPublisher;
        }

        public void Execute()
        {
            var topic = new SimpleTopic { SomeId = ...someid...  };
            _eventPublisher.WriteTo(topic, new ServerEvent("1", "data-1"));
        }
    }
}

Now the client can connect to the stream via http://<YOUR HOST>/_events/<TOPIC NAME> which would be http://localhost:8080/_events/SimpleTopic in my example.

The standard protocol for SSE according to W3C specifies that events should follow the format:

W3C SSE Standard
1
2
3
4
5
id: <EVENT ID>\n
event: <EVENT NAME>\n
data: <EVENT DATA>\n
retry: <RECONNECTION TIME>\n
\n

The bottle takes a slightly different stance on this subject. However, you can override it with your own if you like by replacing the IServerEventWriter service with your own implementation. The stream that comes OOTB follows this format:

FubuMVC.ServerSentEvents SSE Standard
1
2
3
4
id: <EVENT ID>/<EVENT NAME>\n
data: <EVENT DATA>\n
retry: <RECONNECTION TIME>\n
\n

This is done so that you only need to setup one event listener on the client, from which you can build up a standard for processing events on the client. Which I plan to describe more in future posts.

Writing Unit Testable JavaScript

| Comments

What is testable?

The first thing we can do that will render our JavaScript testable would be to avoid inline JavaScript like the plague! Now what do I mean by that? Well, simply don’t write JavaScript inline with your html, like so:

Inline JavaScript DON’T DO THIS
1
2
3
4
5
<!-- THIS IS BAD -->
<button onclick="someFunction('some value')">Button Text!</button>

<!-- BUT THIS IS EVEN WORSE -->
<button onclick="$.ajax({url: 'http://someurl.com', success: function(){/* do something */}})">Button Text!</button>

I see a number of problems with this approach, that are not only related to testability. The first being that this is incredibly difficult to read. I didn’t even include any inline CSS here, but can you imagine how bad this would look if I did! If you can’t tell by now I’m a big fan of clean and elegant code. As far as testability goes, I would have to rely on a tool that can read the HTML and parse out the JavaScript I wish to test. Why go through all that work! We can clean this up a bit, to a better more elegant solution:

Script Tag DON’T DO THIS
1
2
3
4
5
6
7
8
9
10
<!-- BETTER THAN PREVIOUS EXAMPLE
  - MORE ELEGANT
  - EASIER TO READ/MAINTAIN
  - BUT... STILL NOT TESTABLE) -->
<script type="text/JavaScript">
  function someFunction() {
    /* do something */
  };
</script>
<button onclick="someFunction()">Button Text!</button>

While this is a better approach to writing elegant JavaScript, it still doesn’t provide for easy testing. This is because it is still embedded in our HTML, and we would require some kind of parser just to extract it. The good thing about this though is the single, and simple method call from the onclick event property. This is the testable example:

Separate JavaScript File
1
2
3
function someFunction() {
  /* do something */
};
HTML File
1
2
<script type="text/javascript" src="/path/to/js/file"></script>
<button onclick="someFunction()">Button Text!</button>

The reason that I call this testable is because you can take this JavaScript and use it in nearly any medium completely separate from the view, ie. a test runner. The only drawback to the above example is that there is still a very small amount of JavaScript embedded in our HTML the onlick="someFunction()" a global method call. This pattern will quickly pollute the global namespace. The only way we can completely remove this, easily that is, is with the help of either a JavaScript library like JQuery, KnockoutJs, BackboneJs, etc.