Posted on June 9, 2015 by Chris Harrington

Clean promise chains using a pipeline

A few months ago, I started working on a project with a friend of mine who turned me on to the idea of channeling various asynchronous tasks into a pipeline. The idea is that, instead of polluting your local function with a bunch of .thens, you put each of them into a separate module, and then require them into a unifying function that does all the heavy lifting for you. I’m going to walkthrough my implementation of just that right now in a quick Node JS application.

The Pipeline

The code for the pipeline is pretty straightforward. All we’re doing is looping over a list of given promises and thenning them up to form a chain. The code uses require for managing its modules, so if you’re unfamiliar with it, I suggest you read up on the format here. I’m using bluebird for my promise library, which is probably a little overkill for what we’re doing here, and any promise library will do.

"use strict";

module.exports = function(tasks) {
	this.go = function() {
		if (!tasks || tasks.length === 0)
			return Promise.resolve();

		var promise = tasks[0].apply(this, arguments);
        if (!promise.then)
            promise = Promise.resolve(promise);
		for (var i = 1; i < tasks.length; i++)
			promise = promise.then(tasks[i]);
		return promise;

First, we're creating a function called go, which when called will kick off the promise chain. If there are no tasks given, then we just return a resolved promise with no data. Next, we're starting the promise chain by calling the first task with the arguments provided to the go method. This allows us to pass in the data we need when calling the method instead of at construct time. We're also using the apply method to spread out the arguments array into separate parameters for the task function. If the first task's return value isn't a promise, we wrap up the result in a resolved promise to allow for synchronous tasks, too. Finally, we're looping through the remaining tasks and telling them to execute in order after each subsequent promise has resolved.

Example Usage

Here's a relatively contrived example of how we'd use the pipeline. [code language="javascript"] "use strict"; var Pipeline = require("./path/to/pipeline"), Promise = require("bluebird"); new Pipeline([ _multiplyBy5, _add6, _convertToString, _addCurrentDate, _wait ]).go(10).then(function(result) { console.log("Success! " + result); }).catch(function(err) { console.error(err); }); function _multiplyBy5(number) { return number*5; } function _add6(number) { return number+6; } function _convertToString(number) { return number.toString(); } function _addCurrentDate(string) { return new Date().toString() + ": " + string; } function _wait(string) { return new Promise(function(resolve) { setTimeout(function() { resolve(string); }, 1000); }); }

As you can see here, the pipeline takes a bunch of functions (or required modules, if you want to make your code super clean). Once go is called, each of the functions is called in turn with the result of the previous function, building the result as we go. Eventually, the end result is passed to the final then, which has been tacked on to the go call where we're just writing it to the console.


I really like this pattern. I feel that it allows the coder to cleanly separate modules for any somewhat complicated task into readable, testable components with an eye toward reducing coupling. The pipeline itself is a trivial piece of code that requires no special libraries or dependencies and lets the coder break down complex algorithms into bite size components. Thanks for reading!