Skip navigation

Category Archives: News

Our beloved webhook debugging tool known as PostBin just got a little bit more useful. You can now include a URL in the query string when registering your PostBin callback that will get invoked when PostBin is posted to. This way you can use PostBin and still have your hook script run. Here’s an example of what it would look like:

picture-19

Notice you can just prepend your existing callback URL with a PostBin URL and a question mark. In this way, PostBin becomes a decorator to your hook scripts.

A lot of work went into this and a lot of plumbing and debugging. PostBin uses Hookah to invoke your passthrough callback, making PostBin the first real user of a Hookah instance. Hookah was rewritten in Twisted and got a lot of tweaks to make this work. Please consider using Hookah to handle asynchronous callback dispatching in your apps!

PostBin has a number of features planned, including private bins, request replaying (so you can capture events, then replay them to your passthrough endpoint for debugging), filtering, and more.

Now that I’ve told you how cool AppJet is for hook scripts, allow me to obsolete it. This morning (as I try to avoid falling asleep as I type) I’m announcing Scriptlets, a tool for throwing online quick web scripts in any of several languages. It’s perfect for writing hook scripts. In fact, it’s what I’ve been dreaming of for web hooks for several years.

When I started thinking about web hooks I figured, as the early adopters, most advanced users would have access to some kind of cheap PHP web hosting for hook scripts. Only later did these cloud code runners appear like App Engine and AppJet. AppJet in particular is the closest to my ideal, as I mentioned before. But it was limited to a particular language and still felt slightly too heavyweight for simple hook scripts (by the name alone it was obviously intended for apps, not scripts).

The ideal I envisioned was a site like a paste bin (I’m keen on Pastie). You know, where you paste some code, pick a language, and you’d get a nicely syntax highlighted, numbered row display of the code with a simple, disposable URL you could use to show the kids in IRC your code that wasn’t working. I wanted that, but that also gave you a URL to run the code. This way I could use it for one-off web scripts that accepted request parameters and everything. No more random one-off PHP scripts on my server. And of course, it would be perfect for hook scripts.

Actually, when I started talking with friends about this, none of this cool cloud computing stuff was big yet. App Engine and AppJet weren’t even around. The closest thing was Ning, which used to host PHP code for you for free. One of my friends (actually, the one that coined “scriptlets”) was interning at Google and knew, but couldn’t tell me that App Engine was coming. Once it came out, he and I started a project like Scriptlets for just Python… but we ran into a wall when I misread the Python documentation and thought App Engine wasn’t letting us eval anything more than one line long. It turns out this was completely wrong, but we’re both pretty busy and we moved on to other things.

But when Google announced Java support on App Engine, even though I hate Java, I knew that the JVM meant they now supported a bunch of languages. I revisited our code, solved the stupid problem, and got several other languages working. And tonight I launched it… with support for 3 languages: JavaScript (uh oh, AppJet!), Python (uh oh, App Engine!), and PHP (uh oh.. uhh… commodity PHP hosting?). Ruby is on its way, and potentially others.

All Scriptlets languages provide an environment with access to the web request and response, and most of the features of that language. All the constraints of the App Engine environment are inherited, meaning no long running processes, no abitrary file system access and no socket connections. (I’m so glad I didn’t have to solve the sandboxing problem by myself!) But you do get to make web requests, and this is exposed with a custom function in JavaScript and PHP called fetch(). It’s a pretty limited environment and not even intended for state maintaining scripts (yet), but it should still prove useful.

Anyway, think of Scriptlets as another community service for the web hooks ecosystem. I have a bunch of these in mind that I’ll be building out. Like PostBin, Scriptlets is open source on GitHub and waiting for you to submit patches. Enjoy!

One feature of web hooks is that they’re somewhat self documenting. With web APIs you need to know what you’re requesting and how to request it, but with web hooks, you’re just handed the data. Some of it might need explanation, but you can get started pretty quickly by just capturing actual data and seeing what it gives you. When I first started playing with web hooks, I wrote a handler script that would email me the data. This worked okay, but it could easily be better.

So today I’ve released PostBin, a tool for capturing and inspecting data from web hooks. It’s sort like of a cross between pastebin and Mailinator, but for HTTP POST requests. It’s very straightforward and simple. No signups, no configuration. Just click a button, get a bin with a unique URL, capture POST requests with that URL, then view those requests and the data in them at that URL.

The source is available and patches are welcome. If you have useful source changes to push back to me, they’ll likely be deployed to the production app running on Google App Engine. If you just have suggestions, you can leave them on our UserVoice page.

Hope you guys find this useful!

I like to claim you can implement web hooks with one line of code. The reality is you need to do more than make inline HTTP requests. You need to manage user callback URLs, you need to make the requests asynchronous, you probably want a retry scheme, you might want authentication or verification, and in some cases a response handling mechanism. To do these things requires quite a bit more work, and those that have done it so far haven’t yet disclosed the details of how they’ve implemented this. I hope to change that, and to speed things up, I’ve started a simple community project I’m announcing today called Hookah.

The goal of Hookah is to take most of the work out of implementing web hooks. Ideally, Hookah will allow you to have all the features you need for successfully deploying web hooks and only need to add a few lines of code to your app. In fact, one line for each event! On top of that, the interface for Hookah is HTTP, so it’s not that different than directly invoking web hooks and works easily from any language.

The project is on GitHub and, as you can see, it’s very simple. Currently what we have in the master branch does asynchronous event dispatching with retries. This should be enough to get anybody started with web hooks. It’s written in Python for simplicity and readability, but Python is pretty ubiquitous and because it’s an HTTP server it doesn’t require you to know Python to use it. Again, it’s very simple and very early, but I hope it can be the start of a collective effort to make web hooks easier to implement.

For now we can bootstrap off the Web Hooks discussion group to talk about it, so if you have anything to say or ask, join up. There is a rough roadmap in the README and if you’re interested in contributing, let us know.

Today Gnip announced that they’re winding down XMPP support. Why? They say it’s not worth the work to support it right now. The XMPP world still lacks a leading server standard, leaving them to deal with a scattered array of various implementations. On top of this, many of their consumers are unwilling to deal with running their own XMPP server and end up using Google Talk or Jabber.org, which throttle throughput since they’re popular servers.

Web hooks don’t have these problems. This is probably why Gnip continues to use them as their primary mechanism for real-time push notifications. As ideal as XMPP sounds for real-time notifications, it lacks the simplicity and ubiquity of HTTP. Not only is it a new server implementation to worry about, it’s another type of server in the mix!

Anyway, this means I need to go back and edit my slides about Gnip. Despite this seeming like a victory for web hooks (when I contacted Jud Valeski about this, the Gnip CTO responded “Web hooks reigns!”), I was looking forward to Gnip providing a gateway between the world of web hooks and the world of XMPP. They’re likely to return to XMPP eventually, but this may be an opportunity for somebody to provide a focused XMPP to web hooks bridge.

As a sidenote, I’ve thought it interesting that Gnip is aware of “web hooks” as they use it in their diagrams, but their public use of the phrase has otherwise been lackluster. In their write-up of the announcement, ReadWriteWeb seems to have taken it upon themselves to call Gnip’s use of web hooks “Restful push.” Not bad. I’d probably call it that myself if it wasn’t about more than push. But that’s what the social media guys are into, and I don’t have a problem with that since social media is a trendy vehicle for web hooks.