Challenge

HSBC was processing retail transactions through periodic batch jobs, an approach that introduced systemic delays into data analysis and downstream business action. In a financial services environment where timing is everything, the inability to monitor, analyse, and act on transaction data in real time represented both an operational constraint and a competitive liability. The core technical challenge was substantial: migrating from a batch-based architecture to an instantly responsive, real-time streaming solution capable of handling extreme input peaks of up to 10,000 ordered transactions per second without degradation or data loss.

Solution

We collaborated closely with HSBC's engineering teams to design and implement a new real-time transaction pipeline on Google Cloud. Incoming transactions were ingested via Google Pub/Sub, providing a scalable and reliable messaging layer capable of absorbing high-volume peaks. A Dataflow job then automatically identified and enriched each transaction at scale before passing the data to a custom-built events rules engine. Critically, this rules engine was designed for accessibility as much as technical capability, allowing non-technical business users to configure their own consumption rules for downstream systems without engineering intervention, putting real-time data directly in the hands of the people who needed it most.

Outcomes

Business units gained the ability to monitor incoming transactions in real time and act on them immediately, replacing the delays that had previously limited their responsiveness. The shift from a batch-based to an automatically scalable streaming architecture also delivered substantial cost savings, as HSBC no longer needed to maintain the fixed infrastructure overhead associated with its legacy processing approach. The solution gave the bank a modern, elastic data pipeline capable of growing with demand while reducing the total cost of ownership. Key stats include:

10,000 transactions per second handled at peak load by the new real-time streaming pipeline

Real-time decisioning enabled across business units, replacing batch delays with instant data availability

Substantial cost savings achieved by moving to an automatically scalable, serverless streaming architecture on Google Cloud