Skip navigation

Tag Archives: http

Push/callbacks are better than polling.

This should be an easy argument. It’s not just more efficient, it’s more real-time. I once heard something like 40% of delicious.com requests returned 304 Not Modified. I imagine similar numbers for other popular sites. The more real-time you try to be with polling, the worse it gets. Don’t call us, we’ll call you. It’s more efficient.

RPC solutions provide more value than messaging solutions.

Although you can argue that RPC can be powered by messaging, and messaging can be used for RPC, the point is that they are different mindsets. RPC is about triggering code that does something. Messaging is about putting a piece of data in another bucket. When you try to call somebody, you usually don’t want to get their voicemail, right? This analogy goes even further in that, like voicemail, it must be checked on the other end. Messaging just pushes polling somewhere else.

More importantly, the focus on triggering code is central to this value proposition. Generative systems are more valuable than sterile systems. When did the web get interesting? When it became about more than just static content, and code was put in the loop to generate dynamic content. The point is that if you prioritize code to receive a message before humans, you open up many more possibilities.

RPC is about making things happen. Messaging stops short short of that by just moving data around.

HTTP is the defacto RPC protocol.

HTTP is everywhere. There are powerful free servers, clients in every major programming environment, and people know it well. It’s proven and it just works. HTTTP’s simple design also allows it to be extremely versatile. It’s basically the TCP of the application layer.

However, the best thing is that HTTP is RPC. This subtle fact has been true ever since CGI was introduced. We’ve gone through building RPC on top of it with XML-RPC and SOAP, but wisely settled on a form of RPC that’s just HTTP and is even aligned with HTTP semantics: REST.

If HTTP is RPC and HTTP is everywhere, it is our defacto RPC protocol. Especially for web applications that breath HTTP, it almost doesn’t make sense to think of any kind of inter-application communication that isn’t HTTP. Turtles all the way down!

HTTP RPC + Indirection = Webhooks

David Wheeler said, “All problems in computer science can be solved by another level of indirection.” Webhooks, and all callbacks, are about taking a procedure call and performing it on a variable function. This is indirection and this is very powerful. This is why Unix pipes work. STDIN and STDOUT are not hardcoded values, they’re variables that you can control.

Now imagine if all the web applications you used had extension points that you could effectively hook together with any other application. Well, that’s what webhooks are about.

Just because I think web hooks will change the world doesn’t mean I have to take it seriously. I sort of default to a serious tone in my writing. Hopefully this graphic will lighten things up a bit. It was thrown together by a friend after I came up with the tag line he used in it.

I’ve heard several reports of people getting excited about web hooks from my talks or posts here, and then when they find out what’s actually going on they somehow feel cheated. “As if something that simple could do all that!” I don’t know. Perhaps that’s what happened with this guy. Anyway, I’m considering using it as the unofficial web hooks tag line.

Webhooks: So simple youll think its stupid

I’ve got a story for you. And it ends with a fairly unique take on the future of the web. I’m not about to call it Web 3.0 because I detest this long running meme of “let’s define an era before it happens,” but it is the future. And it’s two parts because it’s late and I’m tired.

So one of the key characteristics of Web 2.0 was this idea of “two-way media” or “conversational media” where users could easily publish their own content and comment on the content of others. This gave rise to the phrase “user-contributed content,” where we let the end-user create the content. It took over ten years for this idea to catch on and become a reality. It’s so great, and we’d never go back.

Some of us are privy to the fact that this idea of a two-way, read-write web was the original vision for the web. Tim Berners-Lee’s first web browser included an editor, but publishing didn’t work because the write verbs in HTTP weren’t implemented in early web servers…

Wait, come to think of it, even to this day web servers don’t implement write methods.

Does Apache itself handle PUT or POST requests as intended? No, Apache delegates the proper handling of these to CGI scripts or a module that essentially runs CGI scripts more efficiently. Most web servers still do not implement the write methods of HTTP. They don’t because before we got around to it, something happened.

In fact, I’m not sure many of us realize this fundamental change (other than the original developers of the web) because we’ve all just accepted the way things are as the way things are. Something big happened in 1993 that changed the entire conceptual model of the web.

webhooks08_nodes1138

Web of linked HTML documents

In the beginning, the web was conceptually about serving up these HTML files that would link to each other. If you recall, the path of the URL was just a subset of the server’s filesystem that was mounted to be served by CERN HTTPd or whichever web server. Apparently the killer feature of the web was rendering HTML (so you had inline hyperlinks, among other things), which you had to get somehow, and so GET was all that mattered from HTTP. If you wanted to put something online, you didn’t need a browser, screw HTTP, you just had to FTP a file to the server. Easy enough, right? It was, back then.

Nobody cared to do anything interesting with PUT or POST. However, they were interested in using this fancy new web protocol to access other protocols. So they started hacking the web servers to run scripts that would query WAIS or some other obscure protocol, usually for search because that was a big problem then.

Eventually Rob McCool drafted a spec for CGP or Common Gateway Protocol that would make a standard way for these little scripts to be run by web servers. It was eventually renamed to CGI (since it’s more of an interface than a protocol), implemented in Apache, and that was that. Now you could run scripts on web requests! Most people used CGI for search, which makes a lot of sense, but the rest of the content was still directly served up HTML files because, well, that’s how the web works, yeah?

Slowly, more and more people started doing clever things with CGI. A few people decided to respect the HTTP spec and utilize the proper verbs for write actions, but this CGI thing was kind of a hack, I don’t think ever seriously intended by Tim Berners-Lee, and so people were abusing the semantics, doing destructive things with GETs, ignoring DELETE and PUT. Well, even today people still do this, just less so… anyway…

Eventually our websites got complicated enough and ambitious enough with CGI that almost all requests would go through CGI instead of serving up static HTML documents. And instead of silly filename paths of files that aren’t really on the server, we can make up useful, descriptive paths with the date and title in them.

Today our web is not about serving up files on the server, but generating files to serve. Our web of pages is now generated by “higher-order” web applications that are no longer simple scripts, but complex software.

higherorderweb

Apps are "higher-order" nodes

If you ask me, those little hacks got a little out of hand, but I guess it’s for the best. Otherwise, we wouldn’t have Gmail or Amazon or Wikipedia… we wouldn’t have web applications that did useful things, we’d just have a bunch of static HTML documents linked to each other managed by webmasters and central authorities. None of this democratized media business.

Thanks to CGI we got the read-write web, but we also made the web way more useful than it was intended. Suddenly browsing to a URL would run some code. And code… well, code can do anything.

Next we’ll build on this idea and see how web hooks can change the game again!