m2node - a mongrel2 handler for node.js

This weekend I implemented a Mongrel2 handler for Node.js. At Braintree we built a custom HTTP server to solve some high availability and load balancing problems. Although we only use it with a Rails application, the server itself is language agnostic: request dispatchers could be written for any language for any platform. Mongrel2 has a simliar design philosophy. I’ve been thinking about changing our HTTP server to use the same protocol as Mongrel2 so that we could take advantage of the Mongrel2 handlers with our server. The platform that we’re most interested in using is Node, and it didn’t have a handler yet, so I wrote one.

Example

You can install m2node using npm: npm install m2node.

To run your application require m2node and call the run function with your server and mongrel2 configuration.

var http = require('http'),
    m2node = require('m2node');

var server = http.createServer(function (req, res) {
  res.writeHead(200, {'Content-Type': 'text/plain'});
  res.end('Hello World\n');
});

m2node.run(server, {
  send_spec: 'tcp://127.0.0.1:9996',
  recv_spec: 'tcp://127.0.0.1:9997'
});

The configuration is from the perspective of the handler, so the send_spec in your mongrel2 config should match the recv_spec in your node config. For example, here is the corresponding mongrel2 configuration.

m2node_example = Handler(
  send_spec = 'tcp://127.0.0.1:9997',
  send_ident = '81b7114c-534c-4107-9f17-b317cfd59f62',
  recv_spec = 'tcp://127.0.0.1:9996',
  recv_ident = ''
)

localhost = Host(name = 'localhost', routes = {
  '/': m2node_example
})

main = Server(
  name = "m2node_examples",
  port = 9000,
  uuid = '5dc1fbe7-d9db-4602-8d19-80c7ef2b1b11',
  access_log = "/logs/access.log",
  error_log = "/logs/error.log",
  chroot = ".",
  default_host = "localhost",
  pid_file = "/run/mongrel2.pid",
  hosts = [localhost]
)

servers = [main]

m2node will also work with node applications built using the Express framework.

var express = require('express'),
    m2node = require('m2node');

var app = express.createServer();

app.get('/', function (req, res) {
  res.send('Hello World')
});

m2node.run(app, {
  send_spec: 'tcp://127.0.0.1:9996'
  recv_spec: 'tcp://127.0.0.1:9997'
});

I haven’t tested with any other frameworks, but I think they’re all built using Node’s http server library, so they should all work.

Implementation

The implementation was challenging. Mongrel2 has a simple protocol built using zeromq, but Node’s HTTP server is written to use a TCP socket directly. I ended up writing a FakeSocket class that would take the request from the Mongrel handler and send it to the server using the same interface as a TCP socket.

Here’s the FakeSocket class (written in CoffeeScript).

sys = require 'sys'
util = require 'util'
events = require('events')

class FakeSocket extends events.EventEmitter
  constructor: ->
    @writeBuffer = new Buffer('')
    @writable = true

  destroy: -> # noop
  destroySoon: -> # noop

  emitData: (buffer) ->
    if (@_events && this._events['data'])
      @emit('data', buffer)
    if (@ondata)
      @ondata(buffer, 0, buffer.length)

  setTimeout: (timeout, callback) -> # noop

  write: (data) ->
    combinedBuffer = new Buffer(@writeBuffer.length + data.length)
    @writeBuffer.copy(combinedBuffer)
    combinedBuffer.write(data.toString(), @writeBuffer.length)
    @writeBuffer = combinedBuffer
    @emit('write')

I only implemented the methods that were necessary to run the Express examples, so some functions present on a net.Socket are missing.

The emitData function is what the Mongrel handler uses to send the request. It calls the same events that the TCP socket would call when it receives data.

When data is written to the FakeSocket by calling the write function it’s stored in the writeBuffer variable. The write event is emitted to notify the handler that data has been written.

Here’s the handler.

events = require 'events'
zeromq = require 'zeromq'

{MongrelRequest} = require './mongrel_request'

class Handler extends events.EventEmitter
  constructor: (options) ->
    @pullSocket = zeromq.createSocket('pull')
    @pullSocket.connect(options.recv_spec)
    @pullSocket.on 'message', (message) =>
      @emit 'request', new MongrelRequest(message)

    @pubSocket = zeromq.createSocket('pub')
    @pubSocket.connect(options.send_spec)

  sendResponse: (request, response) ->
    header = [
      request.uuid, ' ',
      request.connectionId.length, ':', request.connectionId,
      ', '
    ].join('')
    outBuffer = new Buffer(header.length + response.length)
    outBuffer.write(header, 'ascii')
    response.copy(outBuffer, header.length)
    @pubSocket.send(outBuffer)

The handler creates the zeromq sockets. It receives requests using a pull socket and sends responses using a pub socket. When it receives a message on the pull socket it instantiates a MongrelRequest object and emits a request event to initiate the request processing.

Finally, here is the run function that ties everything together. It responds to the request event from the handler by creating a FakeSocket. It then waits for the write event on the socket. Once it receives that event, it uses the handler to send the response back to the mongrel server.

exports.run = (server, options) ->
  handler = new Handler(options)
  handler.on 'request', (request) ->
    fakeSocket = new FakeSocket()
    fakeSocket.on 'write', ->
      handler.sendResponse(request, fakeSocket.writeBuffer)
    server.emit 'connection', fakeSocket
    fakeSocket.emitData(request.toFullHttpRequest())

Github Project

If you want to be notified about updates watch m2node on github. Report bugs on the github issue tracker.