Skip navigation

Tag Archives: polling

Push/callbacks are better than polling.

This should be an easy argument. It’s not just more efficient, it’s more real-time. I once heard something like 40% of requests returned 304 Not Modified. I imagine similar numbers for other popular sites. The more real-time you try to be with polling, the worse it gets. Don’t call us, we’ll call you. It’s more efficient.

RPC solutions provide more value than messaging solutions.

Although you can argue that RPC can be powered by messaging, and messaging can be used for RPC, the point is that they are different mindsets. RPC is about triggering code that does something. Messaging is about putting a piece of data in another bucket. When you try to call somebody, you usually don’t want to get their voicemail, right? This analogy goes even further in that, like voicemail, it must be checked on the other end. Messaging just pushes polling somewhere else.

More importantly, the focus on triggering code is central to this value proposition. Generative systems are more valuable than sterile systems. When did the web get interesting? When it became about more than just static content, and code was put in the loop to generate dynamic content. The point is that if you prioritize code to receive a message before humans, you open up many more possibilities.

RPC is about making things happen. Messaging stops short short of that by just moving data around.

HTTP is the defacto RPC protocol.

HTTP is everywhere. There are powerful free servers, clients in every major programming environment, and people know it well. It’s proven and it just works. HTTTP’s simple design also allows it to be extremely versatile. It’s basically the TCP of the application layer.

However, the best thing is that HTTP is RPC. This subtle fact has been true ever since CGI was introduced. We’ve gone through building RPC on top of it with XML-RPC and SOAP, but wisely settled on a form of RPC that’s just HTTP and is even aligned with HTTP semantics: REST.

If HTTP is RPC and HTTP is everywhere, it is our defacto RPC protocol. Especially for web applications that breath HTTP, it almost doesn’t make sense to think of any kind of inter-application communication that isn’t HTTP. Turtles all the way down!

HTTP RPC + Indirection = Webhooks

David Wheeler said, “All problems in computer science can be solved by another level of indirection.” Webhooks, and all callbacks, are about taking a procedure call and performing it on a variable function. This is indirection and this is very powerful. This is why Unix pipes work. STDIN and STDOUT are not hardcoded values, they’re variables that you can control.

Now imagine if all the web applications you used had extension points that you could effectively hook together with any other application. Well, that’s what webhooks are about.

My involvement at NASA inadvertently got web hooks written about on O’Reilly Broadcast yesterday. Kurt Cagle did a nice write-up on his take on web hooks, and it’s possible there will be more coming from Cagle on the topic. Although the post at first seemed pretty framed around syndication and push, the fact that he says things like “server-side mashups” and sees web hooks as a means to “create orchestration of web services” shows he gets the greater significance of this simple mechanism.

I just wanted to cover a few things that were brought up by Cagle and a few others that have written about web hooks recently.

Replacing Syndication

It seems a lot of people see web hooks as an alternative to poll-based feeds and syndication. Although I’ve claimed before that “feeds are not the answer,” it was in context of the vision of pipes for the web. Feeds were not invented for pipelining. They were invented for simple content syndication, and I think they do a pretty good job at that use-case: answering the question, “Hey, what’s new from you?” That said, Cagle seems to be spot on about web hooks and syndication working together.

One of my original arguments for web hooks was that polling sucks. “Hey, what’s new from you?” becomes “Are we there yet? Are we there yet? Are we there yet?” The thing is, web hooks alone don’t let you ask, “What’s new from you?” Nor do they provide a persistent reference to data. Web hook payloads should be ephemeral.

What seems like an obvious solution in this use-case is to provide a feed and a web hook for notifications of updates to the feed. This way you have the feed, which is nice for people that like polling, but also to have a persistent resource on the web for that content stream. And then you have the optional web hook for getting notified of updates, potentially with the updated data so if you don’t want to retrieve the feed, you don’t have to.

There’s a slightly verbose standard spec (as many are) called GetPingd that shows a way you can do this, but I imagine there are simpler approaches. One thing GetPingd points out is that this is all very similar to the ping services for blogs to notify search engines of new content. The missing element of that system is the ability for anybody to subscribe to the notifications. That is part of the essence of web hooks, as Timothy Fitz recently tweeted:

Remember, HTTP callbacks are nothing new. It’s exposing them to the user that makes it a web hook, and that’s where the emergence is.

Anyway, I can understand asking the question, “Will web hooks replace feeds/syndication?” as a thought experiment in trying to understand this new paradigm. But I have to say, after thinking about this for a long time, they won’t. They might replace certain use-cases for feeds, but if feeds were broken enough that web hooks would replace them, it would have happened already.


Now this is where we get into some interesting waters. A lot of people bring up good points on both sides. My stance is simply that web hooks are simpler and just as effective for the majority of use-cases, and therefore the obvious winner. There are less pieces, simpler APIs, existing infrastructure, and it’s debatable whether XMPP is inherently any more performant.

You have to consider the use-cases, though. Part of the vision of web hooks is to have a standard HTTP event mechanism for the web. I just don’t see every web service throwing up an XMPP stack along with their HTTP stack. The two can and will work together when necessary, but as Cagle notes, “web hooks in general may be superior for orchestration.” Remember, web hooks are about more than message passing.


Cagle briefly touched on the standards issue. I’m sure that having a nice standards document would make for great adoption propaganda, and I know quite well the significance of agreed upon conventions in technology. However, I’m not in a hurry to over engineer anything, and I’m not going to assume we’ll know everything about the implications of this mechanism that we can encode them in a document that will either be ignored or adopted by everybody, making it harder to adapt to change. The longer we can put off standardizing, the better.

In the article, Cagle compares it to AJAX, in that the community isn’t very standards oriented. I’m not exactly sure what the AJAX community would benefit from standards. I’m quite happy that AJAX wasn’t limited by a standard to only use XML. There is nothing wrong with options. That’s kind of the whole point of technology: to provide new avenues, options, and choices for empowerment. Tools will always be used however the tool user finds useful, which is not always how the toolmaker intended.

I would much rather provide examples, and rigorously defined patterns of usage and implementation than try to define a standard. When a globally accepted convention is necessary, then we can work one out (with a useful, ideally proven, implementation), but it will probably be about some aspect of web hooks, not the model as whole. I think the biggest aspect ripe for standardization is for machine-friendly announcement of hooks and mechanism for registering callbacks (aka subscribing). But this is not preventing web hooks from being useful, otherwise nobody would be using them already.