Backend

I Spent 3 Days Debugging a Payment Gateway. Here's What Actually Worked.

December 10, 20247 min readBy Mohammad Kanaan

It's 3:47 AM. My phone won't stop buzzing. HyperPay webhooks are failing, and 23 transactions are stuck in limbo. Users can't complete checkout. The Toytopia Black Friday sale is dying.

If you've ever integrated a payment gateway in the Middle East, you know the pain. The docs are outdated, the support is in a different timezone, and one wrong header kills everything.

The Problem: Silent Webhook Failures
#

HyperPay sends a POST request to your webhook URL when payment status changes. Sounds simple. But here's what was happening:

HyperPay was sending webhooks. Laravel was receiving them. But they were getting rejected with a 419 CSRF error - and HyperPay's dashboard showed 200 OK.

The transaction would complete, HyperPay would charge the card, but my database never updated. Order stuck in "pending" forever.

The Obvious Fix (That Didn't Work)
#

First thing every Laravel dev does: disable CSRF for the webhook route.

PHP

// app/Http/Middleware/VerifyCsrfToken.php
protected $except = [
    'api/webhooks/hyperpay',
];

Still failing. Why?

The Real Issue: Middleware Stack
#

I logged the raw request. Here's what I found:

PHP

// PaymentWebhookController.php
public function handle(Request $request)
{
    Log::info('Raw webhook', [
        'headers' => $request->headers->all(),
        'body' => $request->getContent(),
        'method' => $request->method(),
    ]);
    
    // ...
}

The logs showed something wild:

HyperPay was sending Content-Type: application/x-www-form-urlencoded, but the data was actually JSON. Laravel's middleware was mangling the request body.

The Fix That Actually Worked
#

Created a dedicated route group with minimal middleware:

PHP

// routes/api.php
Route::post('/webhooks/hyperpay', [PaymentWebhookController::class, 'handle'])
    ->withoutMiddleware([
        \App\Http\Middleware\VerifyCsrfToken::class,
        \Illuminate\Foundation\Http\Middleware\ConvertEmptyStringsToNull::class,
        \Illuminate\Foundation\Http\Middleware\TrimStrings::class,
    ])
    ->middleware('throttle:60,1'); // Rate limit for safety

Then manually parsed the request:

PHP

public function handle(Request $request)
{
    // Don't trust Laravel's parsing
    $rawBody = $request->getContent();
    
    // HyperPay sends JSON disguised as form data
    $data = json_decode($rawBody, true) ?? [];
    
    if (empty($data)) {
        // Fallback to form parsing
        parse_str($rawBody, $data);
    }
    
    // Validate the webhook signature
    if (!$this->verifySignature($data, $request->header('X-Signature'))) {
        return response()->json(['error' => 'Invalid signature'], 401);
    }
    
    // Process in a queued job
    ProcessPaymentWebhook::dispatch($data);
    
    return response()->json(['status' => 'received'], 200);
}

The Signature Verification Nightmare
#

HyperPay's docs say "verify the signature using HMAC-SHA256." Cool. But they don't tell you the payload order matters.

PHP

private function verifySignature(array $data, ?string $signature): bool
{
    if (!$signature) {
        return false;
    }
    
    // HyperPay requires fields in THIS EXACT ORDER
    $payload = 
        $data['id'] . 
        $data['amount'] . 
        $data['currency'] . 
        $data['timestamp'];
    
    $expectedSignature = hash_hmac(
        'sha256',
        $payload,
        config('services.hyperpay.webhook_secret')
    );
    
    return hash_equals($expectedSignature, $signature);
}

Always use hash_equals() for signature comparison. Regular === is vulnerable to timing attacks.

Queue Everything
#

HyperPay gives you 5 seconds to respond. If your webhook takes longer, they mark it as failed and retry. This creates duplicate processing.

PHP

// Jobs/ProcessPaymentWebhook.php
class ProcessPaymentWebhook implements ShouldQueue
{
    use Queueable;
    
    public $tries = 3;
    public $backoff = [10, 30, 60]; // Retry delays in seconds
    
    public function handle()
    {
        DB::transaction(function () {
            // Use database lock to prevent duplicate processing
            $order = Order::where('payment_id', $this->data['id'])
                ->lockForUpdate()
                ->first();
            
            if (!$order || $order->status !== 'pending') {
                // Already processed or doesn't exist
                return;
            }
            
            if ($this->data['result_code'] === '000.100.110') {
                // Success codes from HyperPay docs
                $order->markAsPaid();
                $this->notifyCustomer($order);
                $this->syncToNetSuite($order);
            } else {
                $order->markAsFailed($this->data['result_description']);
            }
        });
    }
}

Testing Webhooks Locally
#

You can't test HyperPay webhooks on localhost. Here's what actually works:

BASH

# Install ngrok
brew install ngrok

# Start tunnel
ngrok http 8000

# Use the HTTPS URL in HyperPay dashboard
# https://abc123.ngrok.io/api/webhooks/hyperpay

Then create a test endpoint to simulate webhooks:

PHP

// routes/api.php (only in local environment)
if (app()->environment('local')) {
    Route::post('/test/hyperpay-webhook', function () {
        $data = [
            'id' => 'test_' . Str::random(16),
            'amount' => '100.00',
            'currency' => 'JOD',
            'result_code' => '000.100.110',
            'result_description' => 'Transaction succeeded',
            'timestamp' => now()->toIso8601String(),
        ];
        
        $signature = hash_hmac(
            'sha256',
            $data['id'] . $data['amount'] . $data['currency'] . $data['timestamp'],
            config('services.hyperpay.webhook_secret')
        );
        
        Http::withHeaders(['X-Signature' => $signature])
            ->post(url('/api/webhooks/hyperpay'), $data);
        
        return 'Webhook sent';
    });
}

Monitoring in Production
#

Set up a dashboard to track webhook health:

PHP

// Monitor webhook failures
Schedule::call(function () {
    $failedWebhooks = DB::table('webhook_logs')
        ->where('status', 'failed')
        ->where('created_at', '>', now()->subHour())
        ->count();
    
    if ($failedWebhooks > 5) {
        // Send Slack alert
        Notification::route('slack', config('services.slack.webhook'))
            ->notify(new WebhookHealthAlert($failedWebhooks));
    }
})->everyFiveMinutes();

Key Takeaways
#

1. Never trust middleware - Payment webhooks need raw request access

2. Always verify signatures - But read the docs 10 times about payload order

3. Queue heavy operations - Respond to webhooks in under 1 second

4. Use database locks - Prevent duplicate processing when webhooks retry

5. Monitor everything - Webhook failures are silent killers

After these fixes, we processed 1,200+ transactions during Black Friday with zero webhook failures. Sleep is nice.

Pro tip: Set up a /webhooks/test endpoint that triggers your webhook handler with fake data. Makes debugging 10x faster.

Mohammad Kanaan

Senior Full-Stack Engineer specializing in Next.js, React, Laravel, and scalable web architectures. Building production-grade applications with modern tech stacks.

Get In Touch More Articles

Enjoyed this article?

Get notified when I publish new content about web development, architecture patterns, and tech insights.

No spam, ever. Unsubscribe at any time.

The Problem: Silent Webhook Failures#

The Obvious Fix (That Didn't Work)#

The Real Issue: Middleware Stack#

The Fix That Actually Worked#

The Signature Verification Nightmare#

Queue Everything#

Testing Webhooks Locally#

Monitoring in Production#

Key Takeaways#

Tags