Why Webhooks?

We’ve recently done our fair share of integrations. Either for our internal standup application or for some clients that wanted to receive notifications in their app from services like LiveChat, Twillio or various other services.

Webhooks are fun to work with. You basically receive a piece of JSON (more on that later), and it’s up to your code to interpret the content and do what you have to do it.

If the integration you’re trying to build acts as a notification receiver, then using Webhooks instead of APIs long polling is a no brainer.

Need some expert guidance on scaling your existing Ruby on Rails application? Let’s chat.

How to build an integration with any Webhook?

What do we want to build

Right, for the sake of this post, let’s imagine that we want to build a webhook endpoint in our app, that can receive information from Pivotal Tracker or Heroku. I’ve chosen these two because Pivotal Tracker follows the standard procedure of posting you a JSON where Heroku post the values as URL-encoded params, so we will need to take care of that.

The routes

I would recommend your routes to be prefixed with a subdomain. This way, if the application is extremely successful and you need to decentralise the webhook-receiving part, you can do that pretty easily.

constraints subdomain: "hooks" do
  post '/:integration_name' => 'webhooks#receive', as: :receive_webhooks
end

As we can see, the route takes the integration name, and they all resolve into the same controller action. The URL might look something like this https://hooks.youapp.com/bugsnag

You will only need this split if you are integrating with multiple services at once.

The controller

Right, now we can receive our webhooks, it’s time to treat them. Here we have to cater for both scenarios where the integration will send as params or as json.

class WebhooksController < ApplicationController
  skip_before_filter :verify_authenticity_token

  def receive
    if request.headers['Content-Type'] == 'application/json'
      data = JSON.parse(request.body.read)
    else
      # application/x-www-form-urlencoded
      data = params.as_json
    end

    Webhook::Received.save(data: data, integration: params[:integration_name])

    render nothing: true
  end
end

Yep, that’s it, there is nothing more to it. We have a two splits, one for the integration sending a JSON body and the other where it would send as params.

Testing it

To write tests, I’ve found useful to get some JSON example using request bin.

You just create a new Request.bin endpoint and add it as a webhook receiver in the app you want to integrate, then fire a couple of webhooks so you can quickly extract the JSON values to store in your fixtures.

I would recommend storing each different action’s JSON results as /spec/fixtures/:integration_name/:action_name.json – this way, writing your tests will be much easier.

Now writing the specs for an integration that send the params as application/x-www-form-urlencoded is trivial.

This test might look like this:

context "Heroku types, sending params" do

  before(:each) do
    request.headers['Content-Type'] = "application/x-www-form-urlencoded"
  end

  let(:received_params) do
    {
      app: "applicationName",
      user: "[email protected]",
      head: "1234",
      head_long: "12345678",
      url: "https://applicationName.com",
      git_log: "  * My push"
    }
  end

  let(subject) { post :receive, {integration_name: "heroku"}.merge(received_params) }

  describe "#receives a deploy hook" do
    it "calls the Webhook::Received service" do
      expect(Webhooks::Received).to receive(:save).with(data: {integration_name: "heroku"}.merge(received_params), integration: "heroku").and_return(true)

      subject
    end
  end
end

Now for Bugsnag’s types, we are receiving a JSON, let’s have a look at a typical test for this:

  context "Busnag types, JSON received" do

    before(:each) do
      request.headers['Content-Type'] = "application/json"
    end

    let(json_file) { "#{Rails.root}/spec/fixtures/webhooks/bugsnag/exception.json" }
    let(subject) { post :receive, {integration_name: "bugsnag"} }

    describe "#receive #{event_type}" do

      it "creates a new BugsnagWebhook submission" do
        request.env['RAW_POST_DATA'] = File.read(json_file)
        data = JSON.parse(File.read(json_file))
        expect(Webhooks::Received).to receive(:save).with(data: data, integration: "bugsnag").and_return(true)
        subject
      end
    end

Using it in development (a.k.a receiving webhooks in development)

But for now, let’s say that your webhook has been received, and the information stored in the database – you will now want to find a way to test those webhooks data on your development machine and start designing the results on screen.

The best fit for this is a gem called Ultrahook.

You can create a free account on the website and start receiving webhooks on your local machine with the provided URL like this

gem install ultrahook
$ ultrahook stripe 5000
Authenticated as senvee
Forwarding activated...
http://stripe.senvee.ultrahook.com -> http://localhost:5000

Things to keep in mind and Part 2

Right, so now, we have the first part of our feature complete, we will talk in Part 2 about what’s going on in the service and how to save these data.

In our scenario, this was a problem worth decoupling since we are receiving webhooks from at least eight different sources.

You will probably also want to keep those endpoints as fast as possible, so I would suggest processing the actual saving in a background queue.

But for now, that’s it!

Header image via