Skip navigation

Monthly Archives: January 2009

Blaine Cook

It’s been a busy week for web hooks, eh? Well, I figured I might as well share a quick note I received while writing the Amazon post. Blaine Cook, former architect of Twitter, sent me a message after finding a post in which I said he was the one preventing Twitter from providing web hooks. You see, I gave my first public talk on web hooks at an event called SuperHappyDevHouse in 2007. Afterward, Alex Payne came up to me and said, “This is great, I’d love to put this in Twitter.” But it never happened, and I was under the impression Blaine was the one that shot the idea down.

Anyway, he sent me a message saying that although he thinks it’s unfeasible for Twitter to provide web hooks at their scale, he’s all for them in principle. In fact, he says his next project will have web hooks from the start, which is exciting to hear.

But I’m still not convinced Twitter is completely unable to provide web hooks. The fact they let you get all notifications via SMS means they’re either making a web request or at the very least, sending an email. Why can’t they asynchronously queue up the outgoing HTTP requests for users that want web hooks?

Anyway, the Twitter argument is for another time. I look forward to seeing what Blaine is up to and how he’s using web hooks.

Zendesk, the hosted help desk software, has had a feature called Targets. This allows you to send notifications to external destinations. I got really excited when I heard about this. Upon further investigation, I found that they are web hooks, but not necessarily the greatest example of web hooks.

Right now, they support two hard-coded targets: Twitter and Campfire. They also let you target an arbitrary URL, which of course they call “very powerful.” That’s a web hook, right? Yes, but unfortunately, the notification payload they send is a human readable text string. That’s not so bad, but really, code is going to process it, so why not make it more machine friendly?

My only other critique is that they shouldn’t hardcode the services they support for targets. They should do as GitHub has done and run a local web server serving open-source handler scripts. This way, they can allow others to easily write and contribute to Zendesk’s integration with other services and still support it and make it feel like hard-coded service integration.

I once had to build a shopping cart framework for a friend’s company in about three days. It was a challenge considering all the complexities involved in pretty much any instance of a shopping cart. Most shopping cart solutions would hardcode predefined rules or configurable options for determining tax, shipping, and promotional discounts. This was kind of ridiculous since those are usually the points where it varies quite a bit from cart to cart. My solution was to provide hooks because I knew these guys were programmers and were smart enough to implement their own callback functions to define whatever rules they wanted. It turned out to be very powerful, one of their favorite features of the framework, and it was trivial to implement.

So how does an architecture like that translate to a hosted e-commerce checkout solution? Obviously, web hooks! Amazon recently figured this out as they just announced a Callback API for their Checkout solution under their Amazon Payments arm. It lets people use their hosted “checkout solution” while letting them customize the rules for calculating taxes, shipping, and promotional discounts in the most powerful way: code.

This is a fairly significant example of web hooks because it breaks away from the notification use-case that people seem to be caught up with. By actually processing the results of a hook callback request, you’re opening up your application logic to the end user. This is user contributed logic, an under appreciated and mostly unrecognized trend for the emerging service platform aspect of the web.

I don’t want to tangent too much into a rant on the web I want (and I think we all want, but don’t realize it). Instead, I’ll summarize some details on this new Amazon announcement of web hooks. The full documentation for which can be downloaded in PDF format.

You define the callback endpoints in a configuration XML document used for setting up Checkout. They use HMAC authentication for hook requests with added timestamp nonsense, very similar to the way they do Amazon Web Services. The payload is somewhat surprisingly not XML, but instead a URL encoded key/value pair format. The strange thing is that they try and do nested data structures this way, which makes me think JSON would be the better choice. And just to confuse you slightly, in the documentation they use XML to describe the data structures they pass to you and expect back, even though XML is not actually involved. Nevertheless, the documentation is quite good.

Anyway, I’m happy to see a use-case emerge in a way that starts to hint at the aspect of this model I’m more interested in: customization via code. Notifications, or “push,” are really just something along the way. Although this is a trivial example, you should be able to start imagining how it could be used to build a plugin architecture for your web app. The ability to allow your users to extend and build new functionality they can share with others does not seem like an empty value proposition!

Until recently, I’ve only included Google in my list of companies that unintentionally provide an example of web hooks. Like PayPal’s IPN, Google Checkout provides a similar feature for notifications. However, I was recently informed that Google Code project hosting has a post-commit hook, which they’ve implemented explicitly using the “web hooks model.”

Google Code's Post Commit Hook

If you have a project hosted on Google Code, you’ll find this under Administer/Source. The documentation for this feature even links to our wiki. I had a feeling this would happen eventually with Google Code. This particular hook event is very popular with most project hosting services, including GitHub.

I never liked mentioning the Google Checkout Notification API because it was terribly heavyweight and didn’t seem to add much to the web hooks model. On the other hand, the Google Code post-commit hook is done quite exceptionally, and even shows a great method of authentication.

The Google Code hook payload is JSON, which is significantly more straightforward and simpler than XML. More interestingly, they use HMAC for authentication, which has been something of an unresolved issue with this model. HMAC seems quite suitable when authentication is necessary.

Like most of the cases of web hooks out there, this was done without my direct influence. However, separately I have been invited to give a private talk at Google on web hooks. That’s happening February 23rd and is open to anybody at Google. On top of this, Joshua Schachter recently joined Google and he’s been in the web hooks camp for a while, so I’m excited to see what else Google does with them.

Update: Google announced this feature on the Google Code Blog.