Webhook Infrastructure Management

Summary

Below we'll discuss three webhook management use cases for Atlas Connex which prevents loss of data, allows control over the rate of events to endpoints, and shows how Atlas Connex filters and transformers can be used to propogate useful event data to many types of webhook endpoints in their specific format.

Introduction

Webhooks are a method to asynchronously deliver data about an event that has occurred to another networked system. For webhook receivers, processing the incoming webhooks synchronously can lead to missed events due to the system being busy. Another core issue for webhook receivers is the loss of events during planned or unplanned downtime such as network failures or system maintenance. For webhook senders and receivers, the format of the event data may be an integration issue as well. When a sender is providing a JSON message, but the receiver only speaks XML, one end or the other must be modified to accomplish the conneciton. Atlas Connex provides a way to manage this relationship, and perform the queue management as well as transformation on-the-fly so that receivers aren't overwhelmed, and the endpoints can be left unmodified.

What does a webhook look like?

A webhook is an HTTP request, typically a POST request, with an associated body containing the message to be passed along. A typical webhook message looks like the following:

                    
POST /webhook_receiver/ HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Host: some.domain.com
Content-Type: application/json
X-Signature: [hmac-signature-for-verification]

{
    "type":"customer.created",
    "data": {
        "id": "cus_1234567890",
        "name": "Johnny B. Goode", 
        ....
    }

}
                    
                
Webhooks are simple, straightforward HTTP requests and are powerful when used well.

Use Case 1: Atlas Connex as a simple webhook broker

In this use case example, Atlas Connex acts as a broker, sitting between the sender and receiver of webhook events. The Atlas platform receives the events from the sender via its highly reliable ingestion engine, and logs the received data right away. It then optionally filters the event based on the user-defined route and filter definition, and finally attempts to deliver it to one or more destinations. In the event a destination is not responding, a network error is encountered, or other issue arises making the receiver unavailable, Atlas Connex will retry the delivery based on the destination settings, until a maximum retry is reached, at which point the event is queued and the user is notified to fix the broken endpoint. Once the endpoint is operational, the user can replay the queued events, and no data will have been lost. If the receiving endpoint is going to be down for planned maintenance, simply tell Atlas Connex to pause your destination. All incoming events will be queued up for delivery when the maintenance is complete. See the destination documenentation for more details on endpoint management, payload verification, throttling, and more.

Use Case 1: Atlas Connex as broker
Atlas Connex as broker only

Configuration

To configure this use case is 4 simple steps:

  1. Create an HTTP Source in the web Console
  2. Point your webhook sender to this new source
  3. Create and configure the Destination in the web Console
  4. Draw the route from Source to Destination in the web Console

Use Case 2: Atlas Connex as a webhook adapter

Use case 2 is an extenstion of use case 1. We get all the benefits of managed delivery, pausable independent destinations, and custom throttling, but we add the ability to manipulate the event payload in flight. In this use case, instead of mapping a source directly to a destination in the web Console, we first map it to an intermediate filter, which transforms the payload format.

Use Case 2: Atlas Connex as adapter
Atlas Connex as broker and adapter

Configuration

To configure this use case is only one more step from use case 1:

  1. Create an HTTP Source in the web Console
  2. Point your webhook sender to this new source
  3. Create and configure the Destination in the web Console
  4. Create and configure a Filter in the web Console
  5. Draw the route from Source to Filter to Destination in the web Console

Use Case 3: Atlas Connex as a full webhook management solution

Use case 3 is an extenstion of use cases 1 and 2. We bring together both the broker and adapter concepts, and use the fan-out capabilities of the Atlas platform to deliver customized, pertinent messages to interested endpoints in their native format, in a completely managed way. In the example image below, when a new tag is created in the git repo, a regression test is started if it's a release or release candidate, and both the management and dev teams are notified.

Use Case 3: Atlas Connex as full management solution
Atlas Connex as full management solution

Ready to get started?

Start your 30 day, no credit card required trial.

Try it Free – Get started in 5 minutes