Skip navigation

Tag Archives: push

Push/callbacks are better than polling.

This should be an easy argument. It’s not just more efficient, it’s more real-time. I once heard something like 40% of delicious.com requests returned 304 Not Modified. I imagine similar numbers for other popular sites. The more real-time you try to be with polling, the worse it gets. Don’t call us, we’ll call you. It’s more efficient.

RPC solutions provide more value than messaging solutions.

Although you can argue that RPC can be powered by messaging, and messaging can be used for RPC, the point is that they are different mindsets. RPC is about triggering code that does something. Messaging is about putting a piece of data in another bucket. When you try to call somebody, you usually don’t want to get their voicemail, right? This analogy goes even further in that, like voicemail, it must be checked on the other end. Messaging just pushes polling somewhere else.

More importantly, the focus on triggering code is central to this value proposition. Generative systems are more valuable than sterile systems. When did the web get interesting? When it became about more than just static content, and code was put in the loop to generate dynamic content. The point is that if you prioritize code to receive a message before humans, you open up many more possibilities.

RPC is about making things happen. Messaging stops short short of that by just moving data around.

HTTP is the defacto RPC protocol.

HTTP is everywhere. There are powerful free servers, clients in every major programming environment, and people know it well. It’s proven and it just works. HTTTP’s simple design also allows it to be extremely versatile. It’s basically the TCP of the application layer.

However, the best thing is that HTTP is RPC. This subtle fact has been true ever since CGI was introduced. We’ve gone through building RPC on top of it with XML-RPC and SOAP, but wisely settled on a form of RPC that’s just HTTP and is even aligned with HTTP semantics: REST.

If HTTP is RPC and HTTP is everywhere, it is our defacto RPC protocol. Especially for web applications that breath HTTP, it almost doesn’t make sense to think of any kind of inter-application communication that isn’t HTTP. Turtles all the way down!

HTTP RPC + Indirection = Webhooks

David Wheeler said, “All problems in computer science can be solved by another level of indirection.” Webhooks, and all callbacks, are about taking a procedure call and performing it on a variable function. This is indirection and this is very powerful. This is why Unix pipes work. STDIN and STDOUT are not hardcoded values, they’re variables that you can control.

Now imagine if all the web applications you used had extension points that you could effectively hook together with any other application. Well, that’s what webhooks are about.

A while back I was interviewed on the Nearsoft blog about webhooks. It goes into more details on the whole push, pipes and plugin use cases. Real-time web is a hot topic these days, so I had to mention how the webhooks movement relates to that trend. Here’s an excerpt about using webhooks for real-time notifications:

Notifications seem to be [a big draw for webhooks] … as in, tell me when something new has been posted, or when something has changed. But with your code on the receiving end of the notification, you can decide exactly how you get notified. For example, tell me about changes over Twitter, not email. In fact, no, use this other hook script that uses cloud telephony to call my cell phone and use text-to-speech to tell me.

Real-time notifications, exactly how you want them.

Okay, it’s not really a dispute. That was sarcasm in the title. Just to be clear, since there’s always been a tiny bit of useful friction between the idea of XMPP and web hooks, it’s important to remember they are not mutually exclusive. Anybody hyping a battle between the two is trying to create or is imagining a controversy that isn’t real. Many proponents of web hooks are XMPP proponents as well. In fact, the video some are pushing around to promote this supposed throw down is of two XMPP supporters (including djabberd author Brad Fitzpatrick) demoing a pubsub system based on web hook callbacks. And it takes place at an XMPP meetup, so of course there was going to be some proselytizing.

Nevertheless, Jabber/XMPP is a messaging protocol. Web hooks provide a model for functional extensibility, so they are a platform for many different things. A push-based pubsub messaging system is just one use. Even though I originally wrote about web hooks as a notification mechanism, my mind was nowhere near pubsub. I was more focused on the idea of web service integration and orchestration. With the commoditization of CGI-enabled web hosting, I was thinking about how the popularity of web programming combined with the easy invocation of HTTP requests could be used to make a more useful and functionally extensible web. A more programmable web.

With that said, there is nothing wrong with discussing what you can and can’t do with web hooks and XMPP, but there is not some angry fight between camps. Done. Over. Moving on.

As it turns out, the project demoed in that video, which is called pubsubhubbub, is something I had previously stumbled upon while browsing projects a friend of mine was involved in. There wasn’t much of a description, and I didn’t dive too deep into the code, so I bookmarked it to come back to it. After that demo happened, it seemed I misheard (from several people) that it was some XMPP pubsub system, which totally confused me because… it’s not. It’s a neat, distributed pubsub implementation built on web hook callbacks created by Brad Fitzpatrick and Brett Slatkin (a Google App Engine developer). Hopefully they’ll put some more documentation up as it develops, but it’s just really neat to see some XMPP folks build an open pubsub system with web hooks. Cheers to them!

Blaine Cook

It’s been a busy week for web hooks, eh? Well, I figured I might as well share a quick note I received while writing the Amazon post. Blaine Cook, former architect of Twitter, sent me a message after finding a post in which I said he was the one preventing Twitter from providing web hooks. You see, I gave my first public talk on web hooks at an event called SuperHappyDevHouse in 2007. Afterward, Alex Payne came up to me and said, “This is great, I’d love to put this in Twitter.” But it never happened, and I was under the impression Blaine was the one that shot the idea down.

Anyway, he sent me a message saying that although he thinks it’s unfeasible for Twitter to provide web hooks at their scale, he’s all for them in principle. In fact, he says his next project will have web hooks from the start, which is exciting to hear.

But I’m still not convinced Twitter is completely unable to provide web hooks. The fact they let you get all notifications via SMS means they’re either making a web request or at the very least, sending an email. Why can’t they asynchronously queue up the outgoing HTTP requests for users that want web hooks?

Anyway, the Twitter argument is for another time. I look forward to seeing what Blaine is up to and how he’s using web hooks.