Blockchains were designed to focus on immutability, decentralization, and security. This serves many great purposes as a security-hardened database but also incurs some trade-offs. One such trade off is that blockchains are not inherently suited to fast data retrieval.
This presents problems to service providers (exchanges, custodians, applications…) who generally require real-time business operations. For example, if a customer sends a transaction on a blockchain that deposits their assets on your platform, then the user experience needs to be as real-time as possible to avoid frustration. Or perhaps they are a trader looking to reduce market exposure as the price is unexpectedly crashing rapidly, every second of delay is literally costing them money.
The benefits of real-time data are not just important to the user experience, but also to the service provider itself. The operational efficiency of rapid reconciliation, fewer manual checks, and reduced support ticket volumes is worth it alone! When offering a managed service, say a regulated custodian, it’s also important to catch that any transaction going out the door was indeed performed by the custodian’s operations team and not some fraudulent or malicious activity. Quickly identifying this involves pulling on-chain data — and you want that to happen as fast as possible.
How to Achieve Real-Time Blockchain Data Access
First you can have webhooks which are essentially event-based alerts such as: “Notify me when this address receives funds / a deposit”. This can take a number of seconds. Your blockchain API provider sends a HTTP POST request to your backend endpoint, your system processes the deposit, updates the balance on the address, triggers user interface confirmation with a success notification of deposit received.
Then you have WebSockets which have a live connection open to stream events as they happen which is essentially: “keep me in the loop all the time”. These streamed events could be new pending transactions (in the mempool), smart contract events (e.g. DepositReceived), address activity (deposit and withdrawals). From a UI/UX perspective you could add an additional front end notification for “incoming deposit detected” and continue listening until the transaction is confirmed. This is obviously great for fast moving financial platforms and avoids any latency caused by confirmations or polling intervals.
Comparing Real-Time Data Approaches for Blockchain Applications
The idea of “just use WebSockets and webhooks” is correct but in reality the implementation varies a lot between chains and the developer experience is rarely uniform. So let’s take a concrete example with Sui and compare it.
Sui is a blockchain that has two RPC interfaces. A public GraphQL RPC which is highly rate limited, doesn’t expose block-wide data easily, and has limited support for real-time indexing. The paid JSON-RPC version offers lower level access, but no single call can get a full block of transactions, and requires individual transaction requests which can get expensive and slow.
Importantly, Sui does not have a public mempool or pre-confirmation state like, say, Ethereum. So you can’t get your “incoming deposit detected” visibility here. Conversely, Solana lets you fetch a full confirmed block via one RPC call and subscribe via WebSocket for account or slot updates. Sui makes you wait for events or have to pull transaction data by brute force - this equates to latency and cost/scalability issues.
There are some different options to solve this. If you want full control, and control is a key part of Cordial’s ethos and product DNA, you need to spin up your own Sui full node and use event subscriptions from JSON-RPC to track the events you care about. If you’re lucky, an indexing service might offer something you can purchase — but you won’t have control, and at the very least, you’ll need to verify it independently. Or, use some hybrid polling and event listening.hybrid polling and event listening.
Then there are of course further optimizations such as caching, load balancing, intelligent polling, idempotency and fallback systems. We mention it here to further show how complex the problem space is, but save the analysis for another time.
Blockchain Data Security and Failure Handling Strategies
If your blockchain API provider says an event has happened, you may still want to independently verify that - goes back to Zero Trust security principles which we talk about A LOT at Cordial Systems. Beyond that, you may also want to take further measures like validating webhook signatures to avoid spoofed data.
Then we must think about how everything fails, and all the time. We must assume that we will need to deal with failures and often it’s exactly when you don’t want them to happen. For example, an event might occasionally be missed so you want to have some “retry logic” in case a webhook fails. If we’re talking about an extended period of downtime then you may want some “backfill logic” to catch any missed events during the downtime. This should all be part of the data integrity through confirmation and fallback strategies.
How Cordial Powers Real-Time Digital Asset Infrastructure
At Cordial, we have iterated and iterated some more, all to bring real-time data capabilities to over 50 different layer 1 blockchains we currently support. Whenever you create an address in Cordial Treasury it is automatically subscribed to for event listening. You can also subscribe to addresses not associated with keys inside Cordial Treasury, e.g. a whitelisted destination address for your client, or a unique deposit address you own at an exchange. This allows you to detect address activity and on-chain data in near-real time.
So how do we do it? For transactions, Cordial Treasury monitors for the on-chain activity and sees the hash of a transaction that is related to one of our subscribed addresses (typically either a “to” or “from” address). We record the transaction hash in our database, which can be used by customers for further reconciliation efforts of the kind described earlier in the article. Then we “unfold” the hash into a full-blown transaction information packet (block, timestamp, amount, fees, movements etc), and return the full information blob to the user interface and also store that in our database. About 80% of the 51 blockchains we cover are on our fast data capability infrastructure.
We discussed Sui above so here is Sui’s performance, on an average basis throughout the day, when customers have been receiving a Sui transaction into Cordial Treasury (the y-axis is measured in seconds, from when a relevant transaction is picked up in the chain’s lead block and then fully unfolded within Cordial Treasury).
What we see is that it’s just under 10 seconds from spotting the transaction hash on-chain which is included in the lead block, and then being unfolded into a full transaction packet within Cordial Treasury and then updated on the UI.
As mentioned above, Sui does not allow mempool querying, so from a user experience perspective there will be some additional seconds from when they broadcast their transaction, no ability to have a “deposit detected” notification, and instead we just go straight to picking it up by streaming the lead block of the chain and 10 seconds later the user sees the balance updated. We also have other dashboards which allow us to monitor data performance further and inform our optimization strategies.
Need Real-Time Blockchain Data? Build on Cordial.
For now, this completes the introductory course on the importance of data capabilities and how Cordial Treasury handles this. Exchanges, custodians, and other applications building on top of Cordial Treasury infrastructure can rest assured that they will have positive user experiences and be able to handle all internal reconciliation, fraud, and control checks in an expedited manner. We currently have over 50 blockchain L1s supported, 80% of which are on the fast data capability, and these numbers only go up.
Ready to build with real-time blockchain data? Contact Cordial Systems to learn how we can support your infrastructure.
