EVM RPC failover: how to prevent stale or inconsistent data

Start with the job your team wants to keep or hand over. If you build and operate a provider pool yourself, you own upstream selection, capability checks, routing, health monitoring, failover, node operations where applicable, and incident recovery. Adding another endpoint increases the system you must maintain.

With SolidRPC, those operational responsibilities belong to the service. Your application uses one integration across the supported chains and methods instead of managing a primary provider and several fallback paths. An upstream incident is SolidRPC’s responsibility to investigate and recover from.

Your team still specifies the data, historical depth, throughput, freshness, and consistency the product needs. It also owns how returned data becomes an application decision or a durable database record. Use the distinction below when reading the operator requirements in this guide.

A useful failover contract has four parts:

• Reachability: a request completes before its deadline.
• Capability: the selected upstream supports the method, tracer, block range, and response size you need.
• Freshness: the node is close enough to the chain head for this workload.
• Consistency: all reads in one logical operation describe the same block or an explicitly accepted finality level.

These requirements differ by workload. A dashboard may accept a slightly older finalized balance. A mempool monitor needs the newest view it can get. An indexer can process unfinalized blocks only if it stores enough information to roll them back. A payout service may require an explicit independent-verification policy before acting; ordinary failover should not be mistaken for a consensus guarantee.

Your application team should write these requirements down per request class and qualify the RPC service against them. Without it, a router can improve HTTP success rate while silently weakening the application’s data guarantees.

The RPC operator must classify outcomes before deciding whether to try another upstream. A timeout, connection reset, upstream throttling, or transient 5xx may justify another attempt within a bounded deadline. A customer account or API-key limit instead requires request pacing or a capacity change; rotating upstreams does not increase the customer’s allowance.

A valid JSON-RPC result is different. null, 0x0, 0x, and an empty array can all be correct answers for particular methods. Treat an empty result as wrong only when the method’s semantics and surrounding evidence prove it.

The same distinction applies to errors:

• -32601 identifies an unavailable method; the operator must check the supported capability and route.
• -32602 indicates invalid parameters; the application must fix the request.
• an execution revert is an application result, not an infrastructure outage.
• pruned-history errors require a route with the necessary history, where that capability is supported.

For a customer using SolidRPC, upstream error classification and route selection belong to SolidRPC. The application should retain the response for diagnosis, handle deterministic errors, and bound any retries to the same service endpoint. A persistent failure of an advertised capability is a service issue to raise with SolidRPC, not a requirement to build a second provider path.

Qualification is the RPC operator’s job. If your team chooses to operate its own pool, build a capability matrix per chain and upstream covering:

• standard methods used by the application
• retained block and state history
• eth_getLogs range and response limits
• exact debug_* or trace_* methods and tracer options
• required transport and batch behavior
• rate, concurrency, and timeout limits

Method names alone are not enough. EVM clients expose different tracing namespaces, pruned nodes cannot answer every historical query, and two providers may share underlying infrastructure. A useful secondary must reduce shared failure risk and satisfy the required workload.

A team buying managed RPC qualifies the customer-facing service instead. Ask the provider to demonstrate a controlled preferred-path failure, then repeat the required method, history, freshness, and response-shape tests through the same endpoint. The provider owns internal qualification and requalification; the customer retains an acceptance suite for the service it actually uses.

The RPC operator must distinguish a responsive process from fresh blockchain data. A liveness probe such as eth_chainId proves only that an endpoint speaks JSON-RPC for the expected chain. An operator should compare block numbers, timestamps, and hashes across qualified references, exclude stale capacity, and verify recovery before returning it to service.

Block height alone is insufficient: during a reorganization, nodes can report the same height with different hashes. The application should specify its permitted freshness and consistency envelope and test the service against it, rather than maintaining a second production router to compensate for stale serving.

Finality remains an application choice. Ethereum’s JSON-RPC block parameter includes latest, safe, and finalized; other EVM networks can have different models or tag support. Test the exact chain and choose the block reference that matches the product’s requirements.

Apply the same intent to application caches. Cache finalized historical data where safe, but keep head-sensitive responses short-lived or uncached. Monitor the age of data your product actually uses; escalate a persistent service freshness failure with the chain, block, time, and affected request.

Resolve an explicit block before a workflow that makes several related reads. Record its number and hash, then issue every compatible state query against that same block. If the endpoint supports EIP-1898, a block-hash parameter with requireCanonical: true can make a non-canonical result an explicit error instead of allowing the workflow to mix forks.

This matters for workflows such as reading several balances, walking contract state with repeated eth_call, or combining a block, receipts, logs, and traces. Using latest on every call lets the target move between requests even when no failover happens. Switching upstreams makes that ambiguity harder to notice.

For logs, the Ethereum execution API allows an eth_getLogs filter by blockHash. For range backfills, store the hash of every processed block and validate parent continuity before advancing. If the service cannot serve the pinned block within the operation’s deadline, stop or retry the logical operation according to its recovery policy. Keep the block reference fixed; do not quietly continue at a different block or add application-level upstream switching.

There are two retry budgets to coordinate: attempts within the RPC service and retries made by the application. The operator owns upstream attempts and any hedging. The application owns its total deadline, bounded retries to the service, and concurrency. Multiple unbounded retry layers can multiply one request into a traffic surge.

A hedge sends another request when the first has not completed after a delay. It can reduce tail latency for idempotent reads, but duplicates work. That trade-off differs sharply between a small current-state read and a multi-second trace or wide eth_getLogs query. A reliable operator must account for method cost and capacity when setting its policy.

Your application should retry only unresolved, retryable work within its own deadline, use backoff and jitter for throttling, and reserve capacity for live traffic. With SolidRPC, keep those retries on the same integration; upstream selection and recovery remain with the service. The 429 guide explains how to separate demand limits from upstream failures.

Managed RPC does not maintain the indexer’s database. The indexer’s durable cursor should contain a block number and block hash, not just the number. Before committing block N, verify that its parent hash matches the committed hash for N - 1. Store indexed rows with block identity so a fork can be reversed deterministically.

A robust loop is:

1. Read a bounded block range ending behind your chosen confirmation boundary.
2. Fetch and verify the block hashes that anchor the range.
3. Process logs, receipts, and traces against those explicit blocks.
4. Commit application data and the block cursor atomically.
5. On a parent-hash mismatch, find the common ancestor, remove orphaned data, and replay the canonical range.

WebSocket failover needs the same reconciliation path. A replacement subscription starts from “now”; it does not prove that nothing was missed while the connection was down. Reconnect, resubscribe, and backfill from the last committed block before returning to live events. Geth’s subscription documentation also notes that a reorganization can resend old-chain logs with removed: true and can emit the same transaction’s logs more than once, so consumers must be able to undo and deduplicate.

SolidRPC currently serves HTTPS JSON-RPC rather than WebSockets. Its live-indexing path uses polling and explicit ranges; a hard subscription dependency needs a separate transport compatibility decision before migration.

Separate the write path from ordinary read failover. After a timeout from eth_sendRawTransaction, the transaction may have reached the first node even though the response did not reach you. Treat that outcome as unknown, not as a confirmed failure.

Sign locally, persist the raw transaction and its hash before broadcasting, and query by that hash after an ambiguous result. If rebroadcast is required, send the identical signed bytes; do not create a different transaction with the same nonce as an automatic retry. A “known transaction” response can be evidence that the bytes were already accepted, not an outage that needs further fan-out.

Read operations used to make a write decision—nonce, fee data, balances, and contract calls—deserve stricter consistency than cosmetic UI reads. Pin them to an intentional block or require agreement where a wrong answer could create financial loss.

The RPC operator needs internal visibility into routing decisions, upstream health, throttling, attempt counts, latency, and recovery. That evidence lets the operator investigate incidents and detect a fallback or capacity problem before it becomes a prolonged service failure. Operating those monitors belongs to SolidRPC when you use its managed service.

The application team should monitor the outcome it receives:

• method, chain, request time, and returned error
• end-to-end latency and retries to the service
• returned block identity and data age
• indexer backlog and oldest uncommitted block
• useful results committed within the product’s deadline
• affected users or application operations

These checks establish customer impact without requiring access to individual internal upstreams. If a supported request fails persistently, send the relevant method, parameters without credentials, timestamp, returned error, and observed impact to SolidRPC. The service owns investigation of its routing and infrastructure; the application team owns recovery of any unfinished application work.

Use a controlled test environment and agree on failure scenarios with the RPC operator. A self-managed team can exercise its own upstreams; a managed-service customer should ask the provider to demonstrate failure and recovery through the same customer endpoint.

Include timeouts, throttling, stale data, unavailable required history, unsupported methods, and slow heavy requests. Verify deadlines, the exact historical block and method, response shape, throughput, and recovery. Record any capability that is not covered rather than inferring universal continuity from an archive label.

Separately, test application behavior for reorgs, duplicate delivery, process restarts, malformed responses, and ambiguous transaction-submit timeouts. Confirm cursor continuity, rollback, and safe replay. These application tests remain necessary even when upstream failover works.

The operator owns internal requalification after routing or node changes. Keep customer acceptance probes for the methods and outcomes the product depends on, and rerun them when those requirements or the published service capabilities change.

RPC operator responsibilities

• Qualify supported methods, history, capacity, and failure behavior.
• Maintain routing, upstream health checks, bounded attempts, and recovery.
• Monitor freshness and service failures and investigate infrastructure incidents.
• Demonstrate the supported workload during a controlled failure test.

Application responsibilities

• Define method, history, throughput, freshness, finality, and consistency requirements.
• Test the customer endpoint against those requirements.
• Pin related reads to an explicit block where supported.
• Handle valid empty results and deterministic errors correctly.
• Keep application retries, deadlines, and concurrency bounded.
• Persist block-number-and-hash cursors and implement reorg rollback.
• Treat transaction-submit timeouts as unknown outcomes and rebroadcast identical signed bytes only when appropriate.
• Monitor user-visible results and recover unfinished application work.

With SolidRPC, the first list belongs to the service. Your team maintains the application controls in the second list without operating a provider fallback pool.

SolidRPC replaces the customer-managed RPC provider pool with one integration for supported HTTPS JSON-RPC traffic. SolidRPC owns upstream routing, failover, monitoring, node operations, and recovery. Your engineers do not need to select backup providers, maintain a provider router, or coordinate upstream incidents.

Start with your current setup: how many providers do you maintain, who responds when one becomes slow or falls behind, and how much engineering time goes into keeping the routes reliable? Those are the jobs to include in the comparison, alongside price and request performance.

Confirm the required chains, methods, history, and transport in the network catalog, then qualify correctness, throughput, and failure behavior with a representative workload. Use a staged migration with a bounded rollback period; after qualification, retire the old provider router.

Your application still owns consistent multi-call reads, reorg handling, durable storage, and transaction safety. SolidRPC owns the RPC service those workflows depend on. Bring your current provider setup to a workload review to compare the complete stack and the operational work it takes to maintain it.