Webhooks are a mechanism for receiving notifications about events from disparate systems without having to continuously poll them. Also called Reverse APIs, they can be thought of as providing “API specs” that the consuming system needs to implement. One challenge of working with webhooks is testing the code that you write to process webhooks in your local development environment. In this article, I’ll show you how to inspect the payload, expose your endpoint using ngrok, Localtunnel, or Pagekite, and test your webhooks in different local environments
How Do Webhooks Work?
As the world becomes more and more interconnected, it becomes more likely that your application needs to receive information from other systems. One way to do this is to periodically poll the other system to see if any new information is available, but you’d have to poll very frequently to get something close to real-time data. Webhooks are a more efficient way to do this.
A webhook is essentially an HTTP request sent by the source system to a registered URL in the destination system. In programming terms, webhooks can be thought of as an implementation of the Observer Pattern. This is where the consuming system registers to be notified of events in the source system by registering an endpoint or URL.
Most source systems support security measures when registering an endpoint, such as allowing you to configure Basic Authentication or authentication using Bearer Tokens. When an event occurs that a system has registered to be notified of, the source system sends an HTTP request to the registered endpoint with a payload that contains information pertaining to the event.
Webhooks are widely used by modern web applications. For example:
- PayPal uses Instant Payment Notifications (IPNs), which are essentially webhooks to send notifications whenever a payment is received.
- GitHub uses webhooks to inform CI/CD systems when a push event occurs.
- Slack implements incoming webhooks that allow any system to send messages that are published to selected Slack channels.
Working With Webhooks
As mentioned before, webhook URLs can be thought of as reverse APIs that the consuming system needs to implement. Typically, webhook requests are processed by writing code in a programming language such as PHP. Before you write any code to process information sent through a webhook, you need a clear understanding of what the payload looks like. Once you write some code to process the payload, you need to be able to test that your code does what it should when it receives the webhook, such as returning the right status code.
Inspecting Payloads and Webhook Testing
Tools such as Request Bin allow you to create a publicly accessible endpoint you can configure in your source system. Whenever an event that you have registered for occurs, the source system sends an HTTP GET or POST request to your bin, allowing you to inspect the payload. Request Bin also allows you to view items like headers, making debugging and troubleshooting easier. To see how this works, head over to Request Bin and click on the link that says Create a public bin instead. You will be presented with a URL you can configure as the endpoint in your source system.
For example, I added a webhook to my WooCommerce site following the steps described in the WooCommerce documentation. In the “Delivery URL” field, I entered the URL that Request Bin presented me with.
When I created an order, WooCommerce sent an HTTP request to the configured endpoint and I was able to inspect the payload immediately by accessing the URL of my bin in Request Bin.
Typically you would not want to create a public bin, as it’s accessible to anyone that has the URL of the bin. It’s usually better to create a private bin, which requires you to create an account with Pipedream.
Receiving Webhooks Locally
Before deploying any code that processes webhooks, you need to test out your code in your development environment. For that you need to be able to expose your local endpoint as a public URL that you can configure as the destination URL in the system that sends the webhooks. There are quite a few tools available for you to do that.
Exposing Local Endpoints With ngrok
Arguably the most widely used tool to expose a local endpoint to the internet is ngrok. To start using ngrok you need to create a free account.
Installing ngrok is pretty straightforward. Head over to the downloads page, get the appropriate binary for your operating system, unzip it, and copy over to a directory on your path. On a Mac you can alternatively use Homebrew to install ngrok:
brew install ngrok/ngrok/ngrok
The next step after installing ngrok is to add an authtoken. Once you create an account and log in, you can obtain your token by visiting your dashboard.
ngrok config add-authtoken (token)
Now you’re ready to expose your local endpoint URL using ngrok. Suppose you have a web application running on port 80. In your terminal, type ngrok http 80. Your output should look like this:
ngrok (Ctrl+C to quit)
Visit http://localhost:4040/ to inspect, replay, and modify your requests
Session Status online
Account Sanjucta Ghose (Plan: Free)
Version 3.0.6
Region Asia Pacific (ap)
Latency 89ms
Web Interface http://127.0.0.1:4040
Forwarding https://deec-110-227-84-74.ap.ngrok.io -> http://localhost:80
Connections ttl opn rt1 rt5 p50 p90
0 0 0.00 0.00 0.00 0.00
You now have a publicly accessible URL, https://deec-110-227-84-74.ap.ngrok.io, you can use as the destination URL when configuring your webhook.
Ngrok offers switches and options that allow you to configure how it works. You can see which options are available with ngrok -help.
Using ngrok With Virtual Hosts
It’s likely that you serve your local site through a virtual host, a mechanism that allows you to serve multiple domains from a single web server. Web servers typically implement this by checking the HTTP request’s Host header to determine which website should be served. To expose a site served through a virtual host, you need to ask ngrok to rewrite the Host header of all tunneled requests to match what your web server expects. Suppose you have a local site running at http://hellfishmedia.test. To expose this site, you could use the following command:
ngrok http --host-header=hellfishmedia.test 80
Using ngrok With WordPress
If the URL you’re trying to expose using ngrok is a WordPress endpoint, the documentation describes a couple of things you need to do to ensure it works.
Using ngrok With Laravel Valet
You don’t need to install ngrok separately if you are a macOS user and use Laravel Valet as your local development environment. Valet comes bundled with a copy of ngrok which it uses to expose publicly accessible URLs. You can simply navigate to your site directory and use the valet share command to obtain a publicly accessible URL.
You need to mention the domain name when using the valet share command if you have associated your site to a domain which is different from the name of the site directory. For example, if you use valet link hellfishmedia to make your site available at http://hellfishmedia.test, then you should also use the following command to obtain a publicly accessible URL:
valet share hellfishmedia
Using Local as Your Local Development Environment
Local, a free development tool by WP Engine, is a very popular local development environment for WordPress developers. Local makes it very easy to expose a local endpoint using the Live Links feature without requiring any additional software.
However, Live Links requires that you input a username and password before it will expose the site. For any webhook you want to test, you’ll need to include them in the URL in the format http://username:[email protected]/.
Alternatives to ngrok
Several alternatives to ngrok exist that allow you to expose a local endpoint to the internet. I’ve tried out two of them below.
Localtunnel
Localtunnel is an open source alternative that allows you to expose a local endpoint without needing to create an account. You will, however, need to have Node.js installed.
To install Localtunnel, use the command:
npm install -g localtunnel
To expose an application running on localhost:80, run the following command on the terminal:
lt --port 80
If your local application is running through a virtual host at http://hellfishmedia.test, then you can use the localhost switch
lt --port 80 --localhost hellfishmedia.test
Pagekite
Pagekite is another service that allows you to expose a local endpoint. It’s written in Python and requires you to have Python installed on your system. Assuming you are on a Mac and have Python 3.x installed, you can get started quickly by typing the following on your terminal:
curl -O https://pagekite.net/pk/pagekite.py
This will fetch the pagekite.py program for you. After that you can create your first “kite” using the following command and replacing (yourname) with the name you want to use to identify your kite:
python3 pagekite.py 80 (yourname).pagekite.me
You will be prompted to enter your email address to create an account when you run this command for the first time. Subsequently, when you run this command you will need to log into your account by entering your password. The above command will make localhost:80 available at (yourname).pagekite.me.
To expose your local site running at hellfishmedia.test through a virtual host, you can use the following command:
pagekite.py deliciousbrains.test:80 (yourname).pagekite.me +rewritehost
Conclusion
Webhooks are a convenient and now almost ubiquitous way for systems to communicate over HTTP. As a developer, you need to be able to inspect the payload to process webhooks, which is where services like Request Bin come in handy. Testing webhook processing code means you need to expose your local endpoint to the internet. This is made possible by a host of services including ngrok, Localtunnel, and Pagekite.
Have you ever had to write code to process webhooks? If so, what tools did you use to test your webhooks? Let me know in the comments.
