APIs have long been useful for many types of situations that developers regularly encounter. They have enabled you — for example — to expose endpoints and then make the desired requests. However, while this technique has often proven sufficient, it has some limiting factors.
For instance, suppose you want to view the commits in a specific project and repository on GitHub. To do this, you can load the GitHub API and explore the relevant commits. You can also check whether someone has committed code by checking the current hash against one that you’ve previously recorded. However, GitHub only allows a certain number of API calls each time. Therefore, you inevitably waste bandwidth and resources.
Webhooks simplify this problem
Webhooks are an event-driven approach that enables applications to send automated messages or information to other applications. They provide a valuable way for software to communicate with other systems by sending out webhook notifications in response to designated events.
In the context of our earlier example, webhooks eliminate the need to make API calls. Instead, you can just tell GitHub to notify you when there are any new commits within the project or repository.
Developers also call webhooks “reverse APIs” because they function on non-structured infrastructure. This makes for a lightweight data-sharing platform and a quick setup process that is less resource-intensive and architecture-specific than traditional APIs.
To ensure the best experience from webhooks, there are several best practices to understand and a few caveats that you should consider. In the following sections, we will introduce some of these practices and some things to keep in mind as you begin to implement your webhooks.
Best Practices for Adding Webhooks to Applications
Handling Authentication
Authentication and authorization are essential aspects of keeping webhooks safe from spoofing, alteration, and infiltration within the network. They are critical because developers don’t receive any replies within webhooks after sending data to an endpoint. The client receiving the webhook should ensure that someone has not altered the data in transit and that the server sending the webhook endpoint is legitimate. There are certain security and authentication measures users should perform to make webhooks secure.
Sign-in Webhooks
For your clients or consumers to verify that you are who you claim to be and that the data you sent in a webhook event is legitimate, you must sign your webhook payload with a secret key or signature. Companies like Stripe use symmetrical secret keys in the request header to sign in webhook payloads. This allows users to access the key in their console to confirm the signature at their webhook endpoint.
Here is an example of a Python verification snippet:
if client_signature != server_sig:
return ('Signature does not match', 403)
else:
server_response = "<Response><Say>Huzzah! The Signature has been validated!</Say></Response>"
return Response(server_response, mimetype='application/xml')
Encrypt Data Sent by Webhooks
Since webhook requests are like any other HTTP requests, they are visible and readable during a webhook event from source to destination. This access makes them susceptible to data infiltration, leading to interception of authentication tokens, critical messages, and corruption of the whole data set.
By using the secure HTTP protocol communication, HTTPS, developers can reduce the message’s readability. Providing a secure webhook URL and installing SSL certificates in the client provides a secure transport layer that encrypts all data in the network.
Please note: This is transport level encryption, however actual encryption can also happen for the payload (at application level) where a user will get the data encrypted by a symmetric key (mentioned above).
Authenticate Connections and Certificate Pinning
Proving the identities of your connections when adding webhooks to your application is imperative. Doing so reduces any information leakage if someone redirects a request. Using Mutual TLS, both the client and the server verify and authenticate each other.
The most secure way to handle webhook payload security is by using certificate pinning. The two connections provide certificates during the TLS handshake to prove their identities. Pinning ensures that an attacker cannot compromise your data, even if an attacker has a malicious root certificate. If an attacker redirects a webhook endpoint, the authentication fails, and the attacker doesn’t access the information within the webhook.
Handling Non-sensitive Data
Webhooks are publicly accessible on the Internet, making their security a challenge. You should primarily use webhooks for non-sensitive data, such as letting users know about a status change or non-authenticated tokens. Avoid sending sensitive or private information by webhooks, even when using encryption.
Implement Error Handling and a “Retry” Policy
When making POST requests to a client URL, some of the requests fail due to poor routing and domain name system (DNS) issues. As a developer, you’ll want to try a few more times, but only for a limited period with a limited number of attempts.
Using exponential backoff provides a better way to approach this problem by reducing the number of wait periods between events and increasing the time between retries. You should mark a request as broken in your endpoint when it has not been responding with the correct status code — for example, 2xx. Additionally, you should terminate sending requests to it. Inaccessibility of broken endpoints should result in a notification to a developer to fix them.
Also, you can implement a “back off” policy to gradually reduce the number of webhook notifications sent to the destination if the Retry feature creates a continuous loop. Any response by the status code 2xx means it was a success. Otherwise, it is a Retry.
Implement Logging
Admin users can use webhook logs to monitor the activity of their webhook account. Webhook logs usually record each broken and successful endpoint during each webhook event. This record helps developers know which of the payloads sent out has reached the consumer, making it easy to debug and fix any errors that are likely to happen down the road.
Subscriptions with Expiry Date
The use of expiration time limits is an efficient way to save on time and resources used by webhooks. Subscribed clients should have a period (a week, a month, etc.) after which their subscription expires if they don’t renew it. This approach is best as some consumers are no longer interested or do not want to receive the data anymore. Since it’s not safe to assume, it’s best to send automated renewal instructions before retiring their subscriptions. If your application has different types of users, it’s an excellent choice to prioritize the premium clients by extending the webhook expiration period policy.
Handling Duplicate Events
Occasionally, webhook endpoints might receive the same event multiple times. To safeguard against duplicated event receipts, developers should ensure that the event processing is idempotent. The best way to achieve this is to log the events you’ve received and avoid processing events that are already logged.
Next Steps
Adopting webhooks into your future application is a great way to better explore them. While there are no standards regarding the security and best practices to follow when adding a webhook to an application, the methods that we have introduced can help you secure your application and ensure your connections are safe when sending data. They can help you achieve security, scalability, and ease of implementation with less bandwidth use and less manual oversight.
