I always try to point out that web hooks are nothing new. In fact, most “new” technology is not new. The mouse, for example: in 1984, when Apple introduced the Macintosh 128K, the mouse and GUI exploded in popularity and quickly became the standard model for interfacing with computers. Little did people know, the mouse had already been around for 21 years! It was invented at SRI by our pal Douglas Engelbart way back in 1963.
It’s interesting to note that the mouse is sort of useless without the GUI. The two are pretty strongly coupled. Perhaps the latent success of the mouse was in part due to the GUI taking about 20 years to be perfected and made ready for mainstream acceptance. In any case, most new technology is not new. Remember that, kids!
The name “web hooks” might be new, but the idea isn’t. The technology certainly isn’t; it’s just HTTP. In fact, you can say the same thing about REST, which wasn’t coined until 7 years after HTTP.
So what was the first major application of web hooks?
The classic example of a web hook is PayPal’s Instant Payment Notification feature, also known as IPN. This feature allows online retailers to handle real-time purchase confirmations from PayPal. After giving PayPal a URL for IPN, they perform an HTTP POST to it whenever the seller receives a payment. This allows the retailer to better integrate the rest of their system with PayPal.
It was a pragmatic solution to achieve simple server-to-server communication. What blows my mind is why it hasn’t been used for much more since then, hence this blog and my slight obsession with the idea. There have been others that have made use of this more recently, and I’m going to cover those guys on this blog, but still… IPN has been around for over 5 years.
I’m not sure how long PayPal has had the feature, but I know it dates back to at least 2003. And PayPal wasn’t the only one. There used to be (and still are) a lot of alternatives to PayPal which provide the same feature. Even Google Checkout has an IPN equivalent.
But the popularity and success of PayPal has given them loads of experience with IPN. They’re basically web hook veterans. I’d love to talk to some of their engineers about their experience providing the IPN feature. How it evolved, maybe how it scales, how the community likes using it, what kinds of tools have been made around it…
Actually, one thing I didn’t realize until I researched it for this post, is that there are 3rd party services that consume the IPN web hook to provide extra features for the retailer. For example, Scrobbld extends the functionality of PayPal with a nice web UI to manage post-sale affairs without even entering your PayPal username. It’s based entirely on consuming the PayPal IPN.
Now an interesting thing to note about PayPal’s implementation of IPN is their confirmation step. Instead of just sending the IPN in a fire-and-forget fashion, they want you to perform an HTTP POST back to them with all the data from the IPN. They use this to prevent spoofing and let you know the IPN did, in fact, come from them.
The HTTP status code you respond to the IPN with is used to determine if they should resend the notification. If you return anything other than 200 OK, they try to resend the IPN in full for up to four days.
And although some people like the idea of a structured payload document in their web hook posts, PayPal IPN data is straight up simple key/value request parameters. This eliminates the need for the consuming script to parse anything beyond what their CGI library probably already takes care of. In the case of PHP, all the data is right there in $_POST.
So in general, I like PayPal’s approach to IPN. They’ve had plenty of time to get it right and do a good job at keeping it simple. Although they might not be the very first example, they’re definitely the most used example of web hooks in action.
Now, I was just thinking about the mouse and GUI analogy I made at the beginning here. Web hooks are a complement to web API’s in much the same way the mouse is a complement to the GUI. In fact, like the mouse, without their API counterpart, I think web hooks are much less useful. Without a GUI, how useful is a mouse?
Perhaps the reason web hooks are taking longer to be adopted is similar to the mouse waiting for the GUI to develop. Even though IPN was around in 2003, it wasn’t until 2005 that web API’s started to become popular. Now that API’s are pretty commonplace, I guess it makes sense web hooks are starting to grow in popularity…
Web Hooks 