Optimize Shopify Webhook Reliability in a Headless Stack

At the time of writing this article, Shopify commands over 10% of the global e-commerce market, and it’s no surprise why. Listening to a podcast? You’ll likely hear Shopify ads. Stuck on the highway? A Shopify billboard is there to keep you company. Binging the latest season of House of Dragon? Time for another 30 second commercial pitch. The messaging: set up a no code, all-in-one, optimized shop in minutes.

We love it, and more often-than-not we recommend Shopify for upstart retailers for its simplicity and ease-of-use (among other benefits). However, that simplicity vanishes when you have complex integration requirements, such as an established supply chain, CRM managing your existing customer base, as well as other external systems that all have to share and agree on customer and transaction data.

That’s where headless (or composable) e-commerce is critical. An important pillar to a successful headless Shopify implementation lives within the integrations of transactions. If you are building a headless site with many touch points, this becomes an even more woven integration layer with webhooks at the core.

Let us break this down for you:

1. In Shopify, you aren’t just adding to cart and purchasing: a plethora of webhook events are being emitted every time a product is being interacted from cart, to checkout, to customers, to orders.

2. An order is not static: triggering this can involve fulfillment obligations, inventory changes, subscription management, and external integration triggered effects (i.e. CRM updates).

3. Failures are not an isolated incident: a failed webhook will have effects on downstream integrations that rely on data from these events to be ingested for operations such as updating a record.

4. Retries triggering race conditions: any delays within webhook processing or downstream dependencies can cause overlapping webhook executions. Now will have to handle duplicate process or events completing out of order.

5. Shared development environment horrors: if more than one developer testing the same webhook, get ready for duplicated webhook events emissions matching the BP oil spill.

6. Reconciliation is part of your workflow: Shopify webhooks fail for many reasons. Shopify may have broken core webhook code, a new release introduced a regression, or an asteroid passed close enough to generate an EMP causing a critical hiccup at Vercel data centers. Downstream updates aren’t going to resolve themselves.

Core Values

Now that we have established why webhooks are a riddle wrapped in a mystery inside an enigma, it doesn’t mean it’s unmanageable. 9thCO has developed an established pattern that not only solves Shopify webhook problems when integrating with 3rd parties, but is as easy to follow as the maze on a children’s menu!

Validate the webhook before trusting it

Every webhook payload includes the header x-shopify-hmac-sha256 to authenticate and protect your API endpoint. Create a reusable function that validates every event your endpoint receives. Our approach is called at the beginning of every webhook request.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import { createHmac, timingSafeEqual } from 'node:crypto';
// this function does not require Next but requires some prep work to call it
export function authenticateShopifyWebhook({
	body,
	hmacHeader,
	secret,
}: {
	body: string;
	hmacHeader: string | null;
	secret: string;
}): boolean {
	if (!hmacHeader) return false;
	const generatedHmac = createHmac('sha256', secret).update(body, 'utf8').digest('base64');
	// Both are base64 strings, convert to buffers
	const generatedBuffer = Buffer.from(generatedHmac);
	const hmacBuffer = Buffer.from(hmacHeader);
	if (generatedBuffer.length !== hmacBuffer.length) {
		return false;
	}
	return timingSafeEqual(Buffer.from(generatedHmac), Buffer.from(hmacHeader));
}
// this function lets a dev pass the request and secret directly
export async function authenticateNextJsShopifyWebhook(
	req: Request,
	secret: string,
): Promise<Request | null> {
	try {
		const clonedBody = await req.clone().text();
		const hmacHeader = req.headers.get('x-shopify-hmac-sha256');
		// HMAC verification
		if (!authenticateShopifyWebhook({ body: clonedBody, hmacHeader, secret: secret })) {
			console.warn('Shopify webhook HMAC verification failed');
			return null;
		}
		return req;
	} catch (error) {
		console.error('Error processing Shopify webhook:', error);
		return null;
	}
}

Handle endpoint actions intentionally

This is where the complexity between external systems that rely on the context of the payload coming in from Shopify start to bite. The systems rely on contexts within the payload provided by Shopify as well as internal states that can result in different outcomes.

For example, an order/create could run into these scenarios:

• Order created but contact has not been created in your CRM yet.

• Order created and the CRM contact already exists and only needs an update.

• The type of order such as a subscription, digital product, or physical product trigger different user creation/updating funnels.

Below is a simplified example of how we handled a contacts state by using both data provided by a Shopify event and a context state derived from a CRM.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// check if contact already exists in CRM
const existingContact = await getContactByEmail(email);

if (!existingContact) {
  // If contact doesn't exist, create new contact records
  const creationResult = await createContact(mappedData);
  ...
} else {
  await createShopifyPurchaseOnOntraport({
    contactId: existingContact.id,
    createdAt: parsedData.data.created_at,
    orderId: parsedData.data.id,
    amount: parsedData.data.total_price_set.shop_money.amount,
    currencyCode: parsedData.data.total_price_set.shop_money.currency_code,
  });
}

By intentionally managing business logic at the integration layer of a webhooks endpoint, you are ensuring the correct data is created, updated, or left unchanged.

Make every webhook idempotent

Shopify webhooks are aggressive and will retry when an endpoint doesn’t return a 2xx response and won’t stop until it maxes out at 8 (Verify webhook deliveries). These conditions can be met when:

• The endpoint times out

• Network failures

• Multiple subscriptions trigger the same event

Now, just because the event retries it doesn’t mean we can’t include solutions to prevent rerunning of logic causing multiple entries or unintended mutations. Our concepts follow the guidelines provided by Shopify and have implemented a persistent storage step by including Redis to track and monitor the states of these events. The core concepts involve:

• Storing Shopify webhook delivery IDs for duplicate delivery detection

• Storing business identifiers like order id, topic, status, recievedAt

• Checking whether an order, refund, or transaction has already been processed

• Avoiding creating duplicate records, duplicate CRM updates, or duplicate access changes by backing off if an event process has already been established within Redis.

Still trying to mental model this? Take a look at a snippet we have implement ourselves.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
// Initialize Redis client
const redisClient = redis();
// Check if the order ID is already in the Redis cache
const cachedOrder = await redisClient.get(`shopify:order:${parsedData.data.id}/order/create`);
if (cachedOrder) {
    console.log(`Order ${parsedData.data.id} already processed, skipping...`);
    return NextResponse.json({ success: true }, { status: 200 });
} else {
    console.log(`Order ${parsedData.data.id} not found in cache, processing...`);
    // Set the order ID in the Redis cache
    await redisClient.set(
        `shopify:order:${parsedData.data.id}/order/create`,
        JSON.stringify({
            orderId: parsedData.data.id,
            topic: 'orders/create',
            status: 'received',
            receivedAt: new Date().toISOString(),
        }),
        {
            nx: true,
            ex: REDIS_CACHE_EXPIRATION_TIME_IN_SECONDS,
        },
    );
}

Build reconciliation into the system

It has been established already that Shopify webhooks are good but cannot be fully relied on due to outages from Shopify itself, failed webhooks, and network issues. Don’t fret as we are already prepared to handle this. Running automated audits to compare Shopify records against any internal records or 3rd party integration data to identify mismatches is the way to go. What you do with it is determine whether it’s a cron that triggers a Slack channel notification, report email, or populates a database where orders are reconciled is up to you.

Want to see the end result of an email triggered reconciliation notification? We have you covered! This cron that runs daily, cross-compares Shopify order data against a CRMs data, then triggers a Mailgun email with the appropriate template based on the comparison result:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
if (
    purchasesForDate &&
    purchasesForDate.length === flatShopifyOrders.length &&
    orderIdsOnlyInCRM.length === 0 &&
    orderIdsOnlyInShopify.length === 0
) {
    await sendEmailWithTemplate({
        to: notificationEmailList,
        subject: 'Daily Report: All Orders exist',
        template: 'alert-reconciliation-success',
        'h:X-Mailgun-Variables': JSON.stringify({
            reconciliationDate: queryDate,
        }),
    });
    return new Response(
        JSON.stringify({
            success: true,
            orderCount: ontraportPurchasesForDate.length,
        }),
        {
            status: 200,
            headers: { 'Content-Type': 'application/json' },
        },
    );
}
// reconciliation failed
const report = {
    orderIdsOnlyInCRM,
    orderIdsOnlyInShopify,
};
await sendEmailWithTemplate({
    to: notificationEmailList,
    subject: 'Daily Report: Missing orders in Ontraport',
    template: 'alert-reconciliation-missing-orders',
    'h:X-Mailgun-Variables': JSON.stringify({
        reconciliationDate: queryDate,
        orderIdsOnlyInCRM: orderIdsOnlyInCRM
            .map((orderId) => orderId.replace('gid://shopify/Order/', ''))
            .join(',<br/>'),
        orderIdsOnlyInShopify: orderIdsOnlyInShopify
            .map((orderId) => orderId.replace('gid://shopify/Order/', ''))
            .join(',<br/>'),
    }),
});

Shared Development Environment Strategy

Shopify environments are structured such that developers share the same environment, which can easily lead to conflict. This is most commonly triggered by developers testing concurrently within the same endpoint. This causes multiple tunnels to be subscribed to the same webhook, being emitted because 5 different endpoints have subscribed to order/create triggering 5 different order created emails and CRM data being mutated into the incorrect state.

Why could this be problematic?

• False positives appear.

• Testing data shows errors.

• Race conditions trigger catastrophic errors caused by multiple webhook events emitted for the same action.

• Time is wasted on debugging a bug that isn’t a bug.

• Developers have feelings too and our hairline can only reced so far back before our skulls become exposed.

We want to pass on some learnings that reduce friction and improve Developer Experience:

• Use ngrok or a tunnel for local webhook testing. Best if it is a shared URL!

• Testing against real deployed webhook URLs and not Shopify’s cli events. They are not production-equivalent, don’t waste your time on app webhook trigger.

• As mentioned in a previous point, have persistent-storage set up even for staging. Prevent duping at the development level to prevent testing strain.

These may seem like small recommendations, but when you are in crunch time (on your fifth coffee of the day), and are seeing regressions only on your machine, or the same email being sent 4 times from one of your endpoints, following these practices will prevent your next migraine when it’s go time.

Shopify Webhook Checklist

This was a bit of a wild ride but hopefully it will help simplify your next Shopify webhook integration. Just to leave you with something to take away, here is a starter checklist based on everything we have covered.

• Validate every webhook with Shopify HMAC.

• Check existing data before making changes.

• Prevent duplicate webhook processing.

• Handle retries and out-of-order events.

• Store webhook and business IDs for tracking.

• Check Shopify against connected systems for missing data.

• Control webhook subscriptions in shared development environments.

• Test webhooks using real Shopify actions before going live.