guest post

Unit Testing Lambda Expressions & Streams

Richard Warburton

This is a guest post from Richard Warburton, who has worked as a developer across varied areas including Statistical Analytics, Static Analysis, Compilers and Networking. He is a leader in the London Java Community and runs OpenJDK Hackdays.

Richard also teaches a course on modern development with Java 8 at Skills Matter, alongside Jim Gough & Raoul-Gabriel Urma. At the end of this course, you will be ready to use Java 8 on your day job and be familiar with the cutting edge programming approaches which allow you to write more flexible and concise code. You can find out more information about the course here, or head to the Skills Matter page to book your place now!


Usually, when writing a unit test you call a method in your test code that gets called in your application. Given some inputs and possibly test doubles, you call these methods to test a certain behavior happening and then specify the changes you expect to result from this behavior.

Lambda expressions pose a slightly different challenge when unit testing code. Because they don’t have a name, it’s impossible to directly call them in your test code. You could choose to copy the body of the lambda expression into your test and then test that copy, but this approach has the unfortunate side effect of not actually testing the behavior of your implementation. If you change the implementation code, your test will still pass even though the implementation is performing a different task.

There are two viable solutions to this problem. The first is to view the lambda expression as a block of code within its surrounding method. If you take this approach, you should be testing the behavior of the surrounding method, not the lambda expression itself.

Here’s an example method for converting a list of strings into their uppercase equivalents.

public static List allToUpperCase(List words) {
    return words.stream()
                .map(string -> string.toUpperCase())
                .collect(Collectors.toList());
}

The only thing that the lambda expression in this body of code does is directly call a core Java method. It’s really not worth the effort of testing this lambda expression as an independent unit of code at all, since the behavior is so simple.

If I were to unit test this code, I would focus on the behavior of the method. For example, here is a test that if there are multiple words in the stream, they are all converted to their uppercase equivalents.

@Test
public void multipleWordsToUppercase() {
    List input = Arrays.asList("a", "b", "hello");
    List result = allToUpperCase(input);
    assertEquals(asList("A", "B", "HELLO"), result);
}

Sometimes you want to use a lambda expression that exhibits complex functionality. Perhaps it has a number of corner cases or a role involving calculating a highly important function in your domain. You really want to test for behavior specific to that body of code, but it’s in a lambda expression and you’ve got no way of referencing it.

As an example problem, let’s look at a method that is slightly more complex than converting a list of strings to uppercase. Instead, we’ll be converting the first character of a string to uppercase and leaving the rest as is. If we were to write this using streams and lambda expressions, we might write something like the following.

public static List uppercaseFirstChar(List words) {
    return words.stream()
                .map(value -> {
                    char firstChar = value.charAt(0);
    firstChar = toUpperCase(firstChar);
                    return firstChar + value.substring(1);
                })
                .collect(Collectors.toList());
}

Should we want to test this, we’d need to fire in a list and test the output for every single
example we wanted to test. The test below provides an example of how cumbersome this
approach becomes. Don’t worry—there is a solution!

@Test
public void twoLetterStringConvertedToUppercaseLambdas() {
    List input = Arrays.asList("ab");
    List result = uppercaseFirstChar(input);
    assertEquals(Arrays.asList("Ab"), result);
}

Don’t use a lambda expression! I know that might appear to be strange advice in an article about lambda expressions, but square pegs don’t fit into round holes very well. Having accepted this, we’re bound to ask how we can still unit test our code and have the benefit of lambda-enabled libraries.

Do use method references. Any method that would have been written as a lambda expression can also be written as a normal method and then directly referenced elsewhere in code using method references. In the code below I’ve refactored out the lambda expression into its own method. This is then used by the main method, which deals with converting the list of strings.

public static List uppercaseFirstChar(List words) {
    return words.stream()
                .map(Testing::firstToUppercase)
                .collect(Collectors.toList());
}

public static String firstToUppercase(String value) {
    char firstChar = value.charAt(0);
    firstChar = toUpperCase(firstChar);
    return firstChar + value.substring(1);
}

Having extracted the method that actually performs string processing, we can cover all the corner cases by testing that method on its own. The same test case in its new, simplified form is shown here:

@Test
public void twoLetterStringConvertedToUppercase() {
    String input = "ab";
    String result = firstToUppercase(input);
    assertEquals("Ab", result);
}

The key takeaway here is that if you want to unit test a lambda expression of serious complexity, extract it to a regular method first. You can then use method references to treat it like a first-class function.


There’s a buzz in the air… or is it on your arm?

Can you feel it yet?

Apple Watch

This is a guest post by Boisy Pitre, Mobile Visionary and lead iOS developer at Affectiva. You can find him on Twitter here, and see his previous guest posts here.


It is the imminent release of Apple’s latest gadget-wonder… the long awaited Apple Watch. Announced last year, the wearable device is due to hit stores in a few months; its launch, sale price, and subsequent success or failure is the basis for immense speculation in and around the techno news websites and journals.

Last Fall’s announcement of the eagerly anticipated watch was true to Apple’s style of introducing something new: bold yet gentle, glamorous yet modest, confident yet demure. Touted as something truly personal to the wearer, the Apple Watch wooed and wowed the event’s audience and the wider general public. It was easy to see that this wasn’t just another technology device trying to act like a watch, but perhaps the very redefinition of the watch itself.

Predictably, it didn’t take long after the announcement for the questions to follow. How briskly will it sell? Who will buy it? And who wears a watch anymore?

What’s Old is New Again

The concept of wearable devices isn’t necessarily new; it’s been around for some time and has existed in various incarnations. Thinking back 15 years ago, I can distinctly remember attending PalmSource (yes, I’m talking about the Palm Pilot for those of you who can remember) in 2000, and witnessing an attendee walking around the show floor with Palm devices strapped to his forearms. It was reminiscent of Locutus of Borg in an episode of Star Trek: The Next Generation.

Thankfully, we’ve come a bit farther in style today. From arm bands with iPods to smartwatches like the Pebble and offerings from Samsung, you don’t have to look like a cyborg to have technology up close and personal to your body. And all indications are that with its myriad of colors, band styles, and body types, the Apple Watch will be as much of a fashion statement as a technology wearable.

Of course, as developers we are the spark and fuel that moves the pistons of Apple’s engines. Seeking new opportunities and pathways for our work is a constant motivation. So what does the Apple Watch mean for the developer?

A Totally New Platform

Just like the iPhone spurred the creation of the amazing “app economy” in 2008 with the release of the native iPhone SDK, the debut of the Apple Watch brings a whole new set of creative potential to the table. Although it has some utility on its own as a timepiece, where the Apple Watch really shines is its integration with the iPhone itself. The Apple Watch is really complete when it can pair up with an iPhone. The iPhone acts as a deliverer of both content and apps to the watch via Bluetooth. In essence, your Apple Watch becomes an extensible and conveniently accessible accessory to your iPhone.

This means if you have an iOS app already written, you can extend it to bring its functionality to the Apple Watch (assuming that there is some aspect of your app that makes sense appearing on someone’s wrist). Your iPhone is the “carrier” of the smarts that your Apple Watch will use; and in doing so, you have whole new ways to extend the usefulness of your iOS apps.

Think Different

A watch is not a phone, and a phone is not a watch. We carry our phones in our pockets and on our hips, but our watches adorn our wrists. As something that you will wear on your arm, the Apple Watch becomes a very convenient, immediate, and intimate place to view and interact with data. It opens up a whole new world of ideas for apps.

Not only is the Apple Watch as a platform more physically accessible, but its screen is considerably smaller in size than any previous iOS device. Given that the largest Apple Watch is 42mm tall (the other option is an even smaller 38mm in height), you have to carefully think about your app idea, and how it will “fit” onto such a targeted space.

The smaller design space of the Apple Watch, along with the intimacy and complete accessibility that it offers, is certain to inspire creative app extensions. It’s going to be interesting to see where developers will lay stake in this brave new world.

And It Will Get Better

Like all technology, the Apple Watch is bound to get “smarter” over subsequent revisions and generations. The perceived limitation of its tethering to the iPhone will become less and less pronounced, eventually to the point where Apple Watch may become a truly stand-alone, Dick Tracy type futuristic device. Think full audio and video interaction… a complete communications experience right on your wrist.

Challenges certainly remain to get there. Increased processing horsepower and capacity required to drive more features will require more battery life, and that will challenge Apple in interesting ways. There’s not a lot of room to put larger and larger batteries on your wrist.

Are You Ready?

Wearables are about to get a lot more popular, and the apps that will empower them are going to be more and more in demand. If you’re an iOS developer with an existing app, I encourage you to look at how your app might be able to augment your user’s experience on their wrist with their Apple Watch. Not all apps may be able to find that crossover, but many will, and with it will come the opportunity for you to become more familiar and close to your users.


Swift London Are you interested in iOS development? Swift London is a group for iOS and OS X developers of all abilities who want to learn how to use it, who meet regularly at Skills Matter. You can join them for their next meetup on Tuesday 17 February – full details here.

The organisers of the Swift London Meetup group have also put together an impressive line-up for a two-day Swift Summit which is taking place in London on 21 & 22 March. The programme includes speakers such as Chris Eidhof, Daniel Steinberg & Ayaka Nonaka. See the full agenda here.

Build a Search Engine for Node.js Modules using Microservices (Part 1)

Richard Rodger

This is a guest post from Richard Rodger, CTO at nearForm & a leading Node.js specialist. He is a technology entrepreneur and wrote the seminal book on Node: Beginning Mobile Application Development in the Cloud. This is the first of a three-part blog, with the second to follow next week and the last post after his In The Brain talk at Skills Matter on the 22 September, building a Node.js search engine using the microservice architecture.

Richard will also be speaking at FullStack, the Node and JavaScript Conference on 23 October.

In this posts Richard will take us through his experience of building a search engine using microservices and how using a microservice architecture can be used to isolate features and create separate components that can be independently developed, deployed and maintained. Part one will introduce you to microservices and start you building your components and modules. Next week, we’ll cover testing the microservice!


Finding the right Node.js module is hard. With over 90,000 modules (and counting!) it can be tricky to pick the right one. You’ll often come across multiple modules that solve your problem. They are often incomplete, abandoned or just badly documented. It’s hard to make the right choice and you have to rely on rules of thumb. When was the last commit? How many stars? How credible is the author?

It would be great if there was a search engine for Node.js modules that ranked them using these heuristics. I built one in late 2012 that tried to do just that. It was a miserable failure! The quality of the results was pretty poor – see for yourself at nodezoo.com. Search engines are harder than they look, let me tell you. Recently I decided to try again. This time, I wanted to use a microservice architecture. This lets me iterate many aspects of the engine, so that it can get better over time.

This blog post is the first in a series covering nodezoo version 2.0. I’ll also be talking about this at my upcoming In The Brain talk on September 22nd. In this blog post you’ll see how a microservice architecture can be used to isolate the features of the search engine into separate components that can be independently developed, deployed, and maintained – developer heaven!

Let’s focus on the information page for each module. After you get your search results, you should be able to click-through to a page that shows you all the useful meta data about a module, such as the number of Github stars, so that you can compare it with other modules. So today we’ll build out one of the services needed to collect that information and display it.

Why Microservices?

The search engine will use a microservice architecture. Let’s take a closer look at what that means. Microservices are small, independently running processes. Instead of a single large, monolithic application running all your functionality, you break it apart into many separate pieces that communicate with each other over the network.

There are quite a few informal “definitions” of microservices out there:

…developing a single application as a suite of small services, each running in its own process and communicating with lightweight mechanisms, often an HTTP resource API
James Lewis & Martin Fowler

…a system that is built up of small, lightweight services, where each performs a single function. The services are arranged in independently deployable groups and communicate with each other via a well defined interface and work together to provide business value.
David Morgantini

…a simple application that sits around the 10-100 lines-of-code mark
James Hughes

And even a “ManifestNO”
Russell Miles

Of course, I have my own definition! It’s really the effect on humans that matters most. Microservices are just more fun. I prefer:

An independent component that can be rewritten in one week.

This gets to the heart of microservices. They are disposable code. You can throw away what you have and start again with very little impact. That makes it safe to experiment, safe to build up technical debt and safe to refactor.

What about all that network traffic, won’t it slow things down to a crawl? In practice, no. We’ve not seen this in the production systems that we have built. It comes down to two factors. First, inter-service communication grows logarithmically, not exponentially. Most services don’t need to talk to most other services.

Second, responding to any message has a relatively shallow dependency-based traversal of the service network. Huh? Microservices can work in parallel. Mostly they don’t need to wait for each other. Even when they do, there’s almost never a big traffic jam of services waiting in a chain.

In fact a microservice architecture has a very nice characteristic. Out-of-the-box you get capacity to handle lots and lots of users. You can scale easily by duplicating services which by design have no shared state. Yes, it does make each individual request a little slower. How bad! In contrast, traditional monolithic approaches give you the opposite: better performance for one user, terrible performance for many. Throughput is what counts if you want to have lots of users and microservices make a very cheap trade-off in terms of pure performance. Here’s the obligatory chart showing how this works. Hint: you want to be in the green box. The red one gets you fired.

perf

One more thing. It’s easy to accuse microservices of falling afoul of the Fallacies of Distributed Computing. Essentially pretending that the network is just like making a local function call. That’s precisely wrong. Microservices embrace the network. You are naturally led to deal with the effects of the network when you use microservices because you can’t avoid them. You have to assume that every time you talk to another component in the system you will incur network traffic. It enforces a productive discipline and clarity of mind. Architecture astronauts are quickly and severely punished.

The Search Engine

You won’t build a search engine in one blog post. Let’s look at the module information page. This collects data from multiple sources and displays it all together. Let’s take a bias towards the Availability aspect of the CAP Theorem. Whatever else happens, we want to display a web page to the user.

We’ll keep things simple. Each Node.js module has description data stored by the NPM registry. Let’s get a subset of that information and display it. Most Node.js modules also have a Github repository. It’s useful to know how many stars and forks a module has as that gives some measure of how good it is. Down the road, we might pull information from other sources, like the module author’s blog or a Google search.

So we have a set of information sources, mostly external, certainly unreliable, and definitely slow! Whatever else happens we’ve decided that we’ll always display the information page even if we have no information or only some of it. In microservice terms, you publish a message looking for information and you display whatever you get back within a certain time-frame (or some other cut-off condition). This scatter-gather pattern is explained in detail by Fred George in this video.

This acceptance of faulty behavior, this anticipation that hard choices will have to be made and results will be imperfect, is a key part of the microservice philosophy. We aim to reduce uncertainty and the frequency of failure but we never seek to eliminate it. Solutions which cannot tolerate component failure are inherently fragile. We aim to be anti-fragile.

The Architecture

There needs to be a microservice for the information page and then one for each data source. For the moment, that means one for NPM and one for Github. And we’ll also need a web server! Let’s use KrakenJS because it’s pretty cool and has easy server-side templates.

Let’s call the information page microservice nodezoo-info, the data source ones nodezoo-npm & nodezoo-github, and the KrakenJS process can be nodezoo-web. That gives you four Node.js processes to run at a minimum, or not – you’ll see in a later part of this series how to reduce this to one process when developing and debugging locally.

Now, the nodezoo-web service delivers the information page. Let’s use the URL path /info/MODULE. The controller that builds that page should ask the nodezoo-info service for the details of the module. The nodezoo-info service should publish a message to any interested services that care to provide information about modules. The nodezoo-npm and nodezoo-github services should listen for that message and respond if they can.

Sounds nice and neat, but there’s a complication. Isn’t there always! You need the Github repository URL to query Github. That information is in NPM. And yet, the Github service is not allowed to know about the NPM service. For the moment let’s assume we can have Schrödinger services that both do and do not know about other services. That allows the Github service to ask the NPM service for the repository URL. To be continued…

This is so simple that you don’t even need an architecture diagram to understand it. What? It would be “unprofessional” not to provide one you say? But microservice deployments are necessarily dynamic and fluid, changing minute-by-minute to meet the demands of user load and new business requirements. The whole point is that there is no Architecture (capital A). The best mental model is to suspend disbelief and assume all services can see all messages, ignoring those they don’t care about. Message transport is just an optimization problem.

Oh, all right then.

arch

The lovely diagram shows you something else. Each microservice owns its own data. There is no shared database. What about consistency you say? Well think about our use case. Node.js modules are not updated that frequently. We can publish messages when they are, and any services that need to update, can do so at their own pace. But module data will be out-of-date and inconsistent in some cases! Well, shucks. That’s by design. We are not going for 100% here. We’re going for fault tolerance.

Think about the error level you find acceptable, and put remedial actions in place to achieve that. Perhaps you publish the update message multiple times over the following 24 hours. It’s wasteful of resources, but covers a lot of cases where services might be down temporarily. Perhaps you have a cross-check service that randomly samples the services to see how consistent they are. Perhaps you have a set of data services and you outsource the problem. This is the style of thinking you need to adopt. Assume things will break.

Now there are many use-cases where this cavalier approach to data is not going to fly. But there are many others where it is perfectly fine. Microservices make it easy to avoid over-engineering when you don’t have to. They can all talk to the same ACID-compliant Enterprise Database Solution if you really care that much. But then you have to install and maintain the damn thing. Expensive.

The Code

Let’s look at the NPM service first as that’s pretty simple. The code in this article uses the Seneca microservice framework. That means it’s concise, you can focus on the business logic, and you don’t need to worry about message transportation. Some may balk at the idea of using a framework for microservices. After three years of building systems this way, it makes sense for us to put our collected experience into a reusable package. But you don’t need permission to go bare bones if that’s what you want to do. The basic principles are exactly the same.

The NPM service needs to do three things:

  • return data about a module, if it has that data;
  • query the NPM registry for data about a module;
  • extract relevant data from the large NPM JSON document returned for each module.

Seneca defines microservices as collections of one or more message patterns. If a message matches the pattern, then the microservice executes it. Messages are considered to be plain old JSON documents. The patterns are simple name and value pairs. If the JSON document has matching top-level properties, then it’s a match. More specific matches win. That is, a pattern with more name and value pairs beats a pattern with fewer. And that’s it.

The pattern matching is deliberately simple so that you can simulate it inside your own head. There is no fancy stuff like regular expressions or dot-notation for sub-properties. The Principle of Least Power is very – well – powerful. If you need to get fancy, match as much as you can and do the rest in code.

As noted above, all the code for this article is open-sourced in Github. It’s available under the MIT license, so you are free to re-use for your own projects, open-source or commercial.

Here’s the message action that extracts data from the NPM JSON document.

// excerpt from: nodezoo-npm/npm.js

 seneca.add(
   'role:npm,cmd:extract',
   {
     data: { required$:true, object$:true },
   },
   cmd_extract)

 function cmd_extract( args, done ) {
   var seneca  = this

   var data       = args.data
   var dist_tags  = data['dist-tags'] || {}
   var latest     = ((data.versions||{})[dist_tags.latest]) || {}
   var repository = latest.repository || {}

   var out = {
     name:    data._id,
     version: dist_tags.latest,
     giturl:  repository.url
   }

   done(null,out)
 }

The message action pattern is role:npm, cmd:extract. This is completely arbitrary. The role property namespaces the pattern, and the cmd property names what it does. Not all patterns are commands, some are just announcements for general interest, as you’ll see in later parts.

The implementation of the pattern, the action that it triggers, is placed in the cmd_extract function. This is just good code organization. It also means you can list all the patterns at the top of the source code file, which gives you a little bit of built-in documentation.

The seneca.add method adds a new message pattern. You supply the pattern and its implementing function. Optionally you can specify the arguments you need. These are additional properties that should be present in the message JSON document. In this case, a data property must be present, and it’s value must be an object. That is, the data property should contain the the NPM JSON result. For more details of the argument validation format, see the parambulator Node.js module.

The action implementation follows the standard Node.js callback pattern:

function some_action( args, done ) {
  // context-specific seneca instance – gives you nice logs!
  var seneca = this

  ...

  // there is no error, so the first argument is null
  done(null,out)
}

The args object is the message document. This what you act on. The done function is the callback to call when you are finished. If there was an error, you should pass an Error object as the first argument to done. Otherwise, pass a null, and then your result as the second. Returning a result is not required. Some messages are fire-and-forget, and there is no result, as such. Always call done though, even if it’s just done(); as you need to signal that you’re finished processing.

The extraction logic is nothing special, it just picks values out of an object data structure. To see the full structure, try this for example.

Data Storage

Microservices do need to store data occasionally. If you want to be puritanical, then data storage should also be a microservice! Data operations are just more messages. A load message maybe, a save message, a remove message, and so forth. Purity is overrated. Seneca does do things this way, but it provides a nice and easy ActiveRecord-style interface for data entities, so you don’t have to worry about it.

This gives you all sorts of lovely side-benefits. Choose this database! No, that one! Change database in mid-project. Who cares! Need to cache data in redis or memcached? Intercept the messages! No need for the business logic code to know anything about it. Need special case validation or data munging? Intercept and do what you want!

I’ll stop now.

Here’s what the API looks like:

  • Create a new data entity: var color = seneca.make$(“color”)
  • Save some data: color.name = “red”; color.save$( callback )
  • Load some data: color.load$( {name:”red”}, callback )

For more details, see the Seneca Data Entities documentation.

For the moment, you’ll store all data as plain text JSON files on disk. Yes, plain old JSON. On disk. No database needed. Later on, in production, we can worry about LevelDB or Mongo or Postgres (no code changes required! I promise). For now, let’s make local development and debugging super easy.

OK, let’s respond to a request for information about a Node.js module:

  seneca.add(
    'role:npm,cmd:get',
    {
      name:   { required$:true, string$:true },
    },
    cmd_get)

  function cmd_get( args, done ) {
    var seneca  = this
    var npm_ent = seneca.make$('npm')

    var npm_name = args.name

    npm_ent.load$( npm_name, function(err,npm){
      if( err ) return done(err);

      // npm is not null! there was data! return it!
      if( npm ) {
        return done(null,npm);
      }

      // query the npmjs registry as we
      // don't know about this module yet
      else {
        seneca.act(
          'role:npm,cmd:query',
          {name:npm_name},
          done)
      }
    })
  }

This implements the role:npm,cmd:get pattern. You need to provide a name property in your message. Speaking of which, here’s an example message that will work:

{
  "role": "npm",
  "cmd":  "get",
  "name": "express"
}

The code attempts to load a module description with the given name. The npm entity stores this data. If it is found, well then just return it! Job done. The caller gets back a JSON document with data fields as per the role:npm,cmd:extract action.

If the module was not found in the data store, then you need to query the NPM registry. Submit a new message with pattern role:npm,cmd:query. That’s that action that does that job. Return whatever it returns.

Finally, let’s look at the query action. This needs to make a HTTP call out the the registry.npmjs.org site to pull down the module JSON description. The best module to do this is request.

  seneca.add(
    'role:npm,cmd:query',
    {
      name: { required$:true, string$:true },
    },
    cmd_query)

  function cmd_query( args, done ) {
    var seneca  = this
    var npm_ent = seneca.make$('npm')

    var npm_name = args.name

    var url = options.registry+npm_name
    request.get( url, function(err,res,body){
      if(err) return done(err);

      var data = JSON.parse(body)

      seneca.act('role:npm,cmd:extract',{data:data},function(err,data){
        if(err) return done(err)

        npm_ent.load$(npm_name, function(err,npm){
          if( err ) return done(err);

          // already have this module
          // update it with .data$ and then save with .save$
          if( npm ) {
            return npm.data$(data).save$(done);
          }

          // a new module! manually set the id, and then
          // make a new unsaved entity with .make$,
          // set the data and save
          else {
            data.id$ = npm_name
            npm_ent.make$(data).save$(done);
          }
        })

      })
    })
  }

This action constructs the query URL and performs a HTTP GET against it. The resulting document is then parsed by the role:npm,cmd:extract action to pull out the properties of interest and merged into any existing data on that module, or else saved as a new data entity. The data.id$ = npm_name line is where you manually set the identifier you want to use for a new data entity.

And that’s all this microservice does. 112 lines of code. Not bad.


Check back next week for part two, when we’ll cover testing the microservice. Can’t wait until then? Let us know your thoughts, ideas and experiences with microservices below, or give us a shout on Twitter!

FullStack

One Model In, One Model Out

jimmy-bogard-skills-matter-blog

This is a guest post from Jimmy Bogard, a member of the ASPInsiders group, the C# Insiders group, as well as being the “Microsoft Most Valuable Professional” (MVP) for ASP.NET since 2009. He is also the creator and lead developer of the popular OSS library AutoMapper.

Jimmy will be speaking at Skills Matter on 3rd September at his In the Brain talk on Object-Relational Mapper’s. The talk will look at application patterns, ORM features and more to show the correct way of using your ORM, and when to ditch your heavyweight ORM and reach for a lighter one. You can register for this free talk here.

In this post, Jimmy discusses mediator patterns, and how we can use them to ensure that our behaviours become less driven by a framework – allowing us to focus on the real work being done.


In most of the projects I’ve worked on in the last several years, I’ve put in place a mediator to manage the delivery of messages to handlers. I’ve covered the motivation behind such a pattern in the past, where it works well and where it doesn’t.

One of the advantages behind the mediator pattern is that it allows the application code to define a pipeline of activities for requests, as opposed to embedding this pipeline in other frameworks such as Rails, node.js, ASP.NET Web API and so on. These frameworks have many other concerns going on besides the very simple “one model in, one model out” pattern that so greatly simplifies conceptualizing the system and realizing more powerful patterns.

As a review, a mediator encapsulates how a series of objects interact. Our mediator looks like:

public interface IMediator
{
    TResponse Send(IRequest request);
    Task SendAsync(IAsyncRequest request);
    void Publish<TNotification>(TNotification notification) where TNotification : INotification;
    Task PublishAsync<TNotification>(TNotification notification) where TNotification : IAsyncNotification;
}

This is from a simple library (MediatR) I created (and borrowed heavily from others) that enables basic message passing. It facilitates loose coupling between how a series of objects interact. And like many OO patterns, it exists because of missing features in the language. In other functional languages, passing messages to handlers is accomplished with features like pattern matching.

Our handler interface represents the ability to take an input, perform work, and return some output:

public interface IRequestHandlerTRequest, out TResponse>
    where TRequest : IRequest
{
    TResponse Handle(TRequest message);
}

With this simple pattern, we encapsulate the work being done to transform input to output in a single method. Any complexities around this work are encapsulated, and any refactorings are isolated to this one method. As systems become more complex, isolating side-effects becomes critical for maintaining overall speed of delivery and minimizing risk.

We still have the need for cross-cutting concerns, and we’d rather not pollute our handlers with this work.

These surrounding behaviours become implementations of the decorator pattern. Since we have a uniform interface of inputs and outputs, building decorators around cross-cutting concerns becomes trivial.

Pre- and post-request handlers

One common request I see is to do work on the requests coming in, or post-process the request on the way out. We can define some interfaces around this:

public interface IPreRequestHandlerTRequest> {
    void Handle(TRequest request);
}
public interface IPostRequestHandlerTRequest, in TResponse> {
    void Handle(TRequest request, TResponse response);
}

With this, we can modify inputs before they arrive to the main handler or modify responses on the way out.

In order to execute these handlers, we just need to define a decorator around our main handler:

public class MediatorPipeline<TRequest, TResponse>
    : IRequestHandler<TRequest, TResponse>
    where TRequest : IRequest {

    private readonly IRequestHandler<trequest, tresponse=""> _inner;
    private readonly IPreRequestHandler[] _preRequestHandlers;
    private readonly IPostRequestHandler<trequest, tresponse="">[] _postRequestHandlers;

    public MediatorPipeline(
        IRequestHandler<TRequest, TResponse> inner,
        IPreRequestHandler<TRequest>[] preRequestHandlers,
        IPostRequestHandler<TRequest, TResponse>[] postRequestHandlers
        ) {
        _inner = inner;
        _preRequestHandlers = preRequestHandlers;
        _postRequestHandlers = postRequestHandlers;
    }

    public TResponse Handle(TRequest message) {

        foreach (var preRequestHandler in _preRequestHandlers) {
            preRequestHandler.Handle(message);
        }

        var result = _inner.Handle(message);

        foreach (var postRequestHandler in _postRequestHandlers) {
            postRequestHandler.Handle(message, result);
        }

        return result;
    }
}

And if we’re using a modern IoC container (StructureMap in this case), registering our decorator is as simple as:

cfg.For(typeof (IRequestHandler<,>))
   .DecorateAllWith(typeof (MediatorPipeline<,>));

When our mediator builds out the handler, it delegates to our container to do so. Our container builds the inner handler, then surrounds the handler with additional work. If this seems familiar, many modern web frameworks like koa include a similar construct using continuation passing to define a pipeline for requests. However, since our pipeline is defined in our application layer, we don’t have to deal with things like HTTP headers, content negotiation and so on.

Validation

Most validation frameworks I use validate against a type, whether it’s validation with attributes or delegated validation to a handler. With Fluent Validation, we get a very simple interface representing validating an input:

public interface IValidator<in T> {
    ValidationResult Validate(T instance);
}

Fluent Validation defines base classes for validators for a variety of scenarios:

public class CreateCustomerValidator: AbstractValidator {
  public CreateCustomerValidator() {
    RuleFor(customer => customer.Surname).NotEmpty();
    RuleFor(customer => customer.Forename).NotEmpty().WithMessage("Please specify a first name");
    RuleFor(customer => customer.Discount).NotEqual(0).When(customer => customer.HasDiscount);
    RuleFor(customer => customer.Address).Length(20, 250);
    RuleFor(customer => customer.Postcode).Must(BeAValidPostcode).WithMessage("Please specify a valid postcode");
  }

  private bool BeAValidPostcode(string postcode) {
    // custom postcode validating logic goes here
  }
}

We can then plug our validation to the pipeline as occurring before the main work to be done:

public class ValidatorHandler<TRequest, TResponse>
    : IRequestHandler<TRequest, TResponse>
    where TRequest : IRequest {

    private readonly IRequestHandler<trequest, tresponse=""> _inner;
    private readonly IValidator[] _validators;
    
    public ValidatorHandler(IRequestHandler<trequest, tresponse=""> inner,
        IValidator[] validators) {
        _inner = inner;
        _validators = validators;
    }

   public TResponse Handle(TRequest request) {
        var context = new ValidationContext(message);

        var failures = _validators
            .Select(v => v.Validate(context))
            .SelectMany(result => result.Errors)
            .Where(f => f != null)
            .ToList();

        if (failures.Any())
            throw new ValidationException(failures);

        return _inner.Handle(request);
   }
}

In our validation handler, we perform validation against Fluent Validation by loading up all of the matching validators. Because we have generic variance in C#, we can rely on the container to inject all validators for all matching types (base classes and interfaces). Having validators around messages means we can remove validation from our entities, and into contextual actions from a task-oriented UI.

Framework-less pipeline

We can now push a number of concerns into our application code instead of embedded as framework extensions. This includes things like:

  • Validation
  • Pre/post processing
  • Authorization
  • Logging
  • Auditing
  • Event dispatching
  • Notifications
  • Unit of work/transactions

Pretty much anything you’d consider to use a Filter in ASP.NET or Rails that’s more concerned with application-level behaviour and not framework/transport specific concerns would work as a decorator in our handlers.

Once we have this approach set up, we can define our application pipeline as a series of decorators around handlers:

var handlerType = cfg.For(typeof (IRequestHandler<,>));

handlerType.DecorateAllWith(typeof (LoggingHandler<,>));
handlerType.DecorateAllWith(typeof (AuthorizationHandler<,>));
handlerType.DecorateAllWith(typeof (ValidatorHandler<,>));
handlerType.DecorateAllWith(typeof (PipelineHandler<,>));

Since this code is not dependent on frameworks or HTTP requests, it’s easy for us to build up a request, send it through the pipeline, and verify a response:

var handler = container.GetInstance<IHandler<CreateCustomer>>();

var request = new CreateCustomer {
    Name = "Bob"
};

var response = handler.Handle(request);

response.CreatedCustomer.Name.ShouldBe(request.Name);

Or if we just want one handler, we can test that one implementation in isolation, it’s really up to us.

By focusing on a uniform interface of one model in, one model out, we can define a series of patterns on top of that single interface for a variety of cross-cutting concerns. Our behaviours become less coupled on a framework and more focused on the real work being done.

All of this would be a bit easier if the underlying language supported this behaviour. Since many don’t, we rely instead of translating these functional paradigms to OO patterns with IoC containers containing our glue.


In the Brain of Jimmy Bogard: ORMs – You’re Doing it Wrong

On 3rd September, Jimmy discusses Object-Relational Mapper’s at Skills Matter HQ. If you want to make your DBA (and your users happy), you’ll want to use your ORM intelligently – come along to find out how!

Register for your free ticket now!

Guest Post: Meet the London Clojurians

This is a guest post from Robert Rees, organiser of the London Clojurians. The group meets regularly at Skills Matter – you can view their previous talks here and their upcoming meetups here.

london-clojurians


Dissatisfaction is the mother of all discovery

A few years ago I was feeling dissatisfied with both Java and its leading successor Scala and having read about Clojure “a LISP on the JVM” I was eager to give it a go. I had heard of LISP before in terms of artificial intelligence and optimisation problems, I had even given Common Lisp a go but the difference between LISP and the conventions of the curly brace block languages was too great.

Clojure’s syntax was no less unfamiliar but the ease of firing up a REPL and the way the language allowed you to bodge your way through with the standard libraries of Java meant that I was able to get over the hump and start exploring the unique way that LISP allows you to compose functions.

Having played around with Clojure a bit I attended a talk by Rich Hickey talking about why he had created Clojure and giving an overview of the language. The audience was small and it represented a fantastic opportunity to learn a bit about Rich’s views on software development which were fundamental to what he was trying to do with Clojure as a language.

After the talk Rich and a small group retired to a nearby pub to continue the conversation. In hindsight this was an incredible opportunity to have a very free and frank discussion with someone who was part language guru and part software practitioner. I remember that I didn’t agree with everything that Rich said that night but it provided a massive amount of motivation to carry on and dig further into the language.

Having learnt what I know of Ruby via the dojo method I knew my chances of learning Clojure would be exponentially greater if I had a group to work with and some kind of format that would force me to get stuck in to the code. I decided to hold a Clojure dojo as part of a series of tech events I was organising as by chance a colleague was interested in the language too. At this point Bruce Durling enters the picture as he attended the dojo and had a long-standing love of LISP. Bruce was excited by the possibility that he might actually get to use LISP in a production capacity rather than as a hobby.

So with Bruce providing the LISP expertise and me the venue and the schedule we turned the one-off dojo into a regular quarterly dojo meeting.

These early dojos are now somewhat legendary as we spend the part of the year trying to program a text adventure in LISP. Our ineptitude knew no bounds but with each meeting our determination not to fall back to OO behaviour and try and create a function that generated the game world (a big map) yielded results. I remember near the end someone implemented a help function that used metadata on our command functions to reflectively generate documentation. It felt like the future!

More importantly the event became more popular and more frequent. Becoming bi-monthly then monthly and attracting both LISP fans and people want to find out more about Clojure. Bruce’s legendary friendliness developed into a sophisticated MC’ing style and we worked hard to try and incorporate the lessons of accessibility that we had seen keep the London Python dojo popular. The early dojo was the training ground for the later usergroup.

It was at the dojo that we discussed whether there would be enough interest in having some talks about Clojure and where we also found some of the first speakers. After all not many people had heard of the language so mostly we would have to learn from one another. Bruce approached Skillsmatter for a slot and again from small beginnings a popular regular event developed. One that seemed to attract more people who were “Clojure-curious” but not prepared to try and code it themselves.

Bruce made a crucial decision at this point that the simplest way to help the group grow was to say yes to anything that people wanted to do but have them run it with our support rather than try and do everything ourselves. When people had organised their event they became organisers in the community in their own right. As both Bruce and I became involved in startups and inevitably got sick or had to leave town for the occasional holiday and had to miss the increasingly more frequent events this meant the London Clojurians (as we had called it) kept on trucking even without us. In fact our absences meant that a new group of dojo MCs and event organisers got to practice their skills in the way that we had.

However I think we’ve retained our core principles from the early days. London Clojurians is open to everyone who is interested in Clojure, not just experts and gurus. We want a community that is respectful and friendly. We want people to share what they know because they know more than we do and everyone benefits from a stronger community. If we learn something then we should help teach it too.

I’m biased but I think that London Clojurians is a good place. One where you can still ask silly questions and get reasonable responses but also a place where you can show off your l33t Emacs skills to people who are interested. I think our talks are not just a technical exercise in Clojure but also examples of having fun and making a difference with code.

So please, if you think Clojure is interesting, join us.