Major incident on our Payment production platform

Incident Report for Lemonway

Postmortem

Dear customers, here is the complete postmortem & timeline of the incident we had on November 25th & 26th :

  • Incident Summary: Multiple Network incidents on Payment Platform (November 25-26, 2025)
  • Duration: 19 hours (from November 25, 2025 4:37 PM to November 26, 2025 11:37 AM)
  • Severity: Critical
  • Impact: All endpoints behind Netscaler on the Payment Platform were disrupted or slowed down
  • Teams Involved: Lemonway Platform, Lemonway Pay Squad, Lemonway Comply Squad, NTT and Citrix.

Summary:
Network equipment outage on payment platform caused 19 hours (Nov 25 4h37 pm – Nov 26 11h20) of disruptions, impacting all payment platform endpoints due to CPU overload. Root cause was identified; a change on Compliance platform triggered mass retries and flooded network equipment (Netscaler) on payment platform.

Timeline:

  • November 25th

    • 16h37 (4:37pm) :

      • First alerts from our monitoring indicating a service degradation, the investigations start.
    • 17h15 (5:15pm) :

      • Services are partially down, the investigations continue with our integrator.
    • 20h15 (8:15pm) :

      • Following several remediations, services are restored
    • 22h20 (10:20pm) :

      • Following unforeseen circumstances, services are degraded/unavailable again, the on-call engineer resumed the investigations with our integrator until 3am
    • 3h50 (3:50am) :

      • Services are finally restored in degraded mode until the root cause can finally be identified.
    • 11h37 (11:37am):

      • The incident’s root cause is finally identified and remediated immediately. Network services are back in optimal conditions.

Resolution process:

The incident was resolved by a capacity upgrade to contain the traffic overload, to allow the resuming of the payments process and identify the root cause.
Once the cause of the change had been identified, a rollback was performed to return to a normal situation.

Posted Dec 18, 2025 - 20:38 CET

Resolved

Dear customers,

We have finally identified the root cause of the incident. The situation is stable again.

Thank you for your patience & udnerstanding.

Lemonway's Production Team.
Posted Nov 26, 2025 - 12:05 CET

Monitoring

Dear customers,

The issue has been identified by our integrator and is being investigated.

In the meantime, our services are back and we will keep a close eye on our monitoring.

Thank you,

Lemonway's Production Team.
Posted Nov 26, 2025 - 03:57 CET

Identified

The issue has been identified and a fix is being implemented.
Posted Nov 26, 2025 - 03:34 CET

Investigating

Dear customers,

We are currently facing an incident with one of our network components on our Payment production platform.

The issue has been raised with our integrator and is currently being investigated.

During this time, you may encounter issues to process payments and access your Dashboard.

We apologize for this major inconvenience and will do our best to get everything back in order as soon as we can.

Thank you,

Lemonway's Production team.
Posted Nov 25, 2025 - 23:11 CET
This incident affected: Production (Production API, Production Webkit, Production Dashboard).