Is it the Solana network, your endpoint, or you?

When a Solana trade fails, four layers could be responsible: your own request, the endpoint you sent it to, the network path between you and that endpoint, or the cluster itself. They produce overlapping symptoms, and the layer people blame is rarely the layer that failed. The sequence in this report convicts one of them in minutes by changing exactly one variable at a time.

It is deliberately boring. That is the point. A diagnosis that depends on judgement will be made differently under stress than it was in calm conditions, and the whole reason for having a sequence is that you will run it while irritated.

The four layers

Your code. A blockhash old enough to be rejected, a commitment level you did not choose, a transaction that has grown past the size limit, an account list that does not match what the program expects, a slippage tolerance calibrated to a stale quote. Everything here is yours and every part of it is fixable.

Your endpoint. Public and hosted nodes apply rate limits, and a node can also fall behind the cluster while continuing to answer confidently. An endpoint that trails will reject transactions built from blockhashes it has not yet seen and will simulate against state that no longer exists.

The path. Everything between your process and the provider: your connection, intermediate networks, and any proxy in between. Loss and latency here produce timeouts that feel exactly like a busy chain and have nothing to do with the chain.

The cluster. A condition affecting everybody, which is the only layer where the honest response is to wait. It is also the layer people jump to first, because it is the one that absolves them.

One variable at a time

The rule that makes the sequence work is that each step changes one thing and holds everything else fixed. Change your endpoint and your transaction at the same time and you learn nothing from either result. This sounds obvious and is violated constantly, usually because the operator is trying several fixes at once in the hope that one of them works.

The second rule is that a negative result counts. "Endpoint B behaved identically" is as valuable as "endpoint B succeeded", and it is the observation people forget to write down. A sequence of recorded negatives is what narrows four layers to one.

The third rule is that you finish the sequence before you fix anything. A change made mid-diagnosis destroys the conditions you were measuring, and if the symptom then disappears you will never know whether your change or the condition was responsible.

The isolation sequence

1. Capture the evidence. Wall-clock time, endpoint, the exact request, and the signature if one was produced. Four items, thirty seconds, and everything downstream depends on them.
2. Repeat on the same endpoint. Send it again unchanged. A different outcome the second time means the condition is transient. An identical outcome means it is deterministic and therefore probably yours.
3. Repeat on a second endpoint. Use an unrelated provider, not another node from the same one. Success here convicts the first endpoint or the path to it. Identical failure moves suspicion upstream.
4. Send a trivial request to both. Ask for the current slot. If the trivial request works while your real one fails, the fault lives in the request. If both trivial requests fail, the fault is in the connection or beyond it.
5. Resolve the signature on chain. Look it up in an explorer. Included and failed is a completely different verdict from never included, and only one of them is about the network.
6. Check the official status property. Last, not first, and record the answer with the time you checked it.

Steps two through four take about two minutes with a terminal open. Step five is the one that most often ends the investigation immediately, because it converts an ambiguous story into a fact.

Step six is deliberately last. Checking the operator status property first tends to end the investigation prematurely in both directions: a declared incident becomes the explanation for a problem it did not cause, and a clean status page becomes proof that the fault must be yours. Both conclusions are unearned until the earlier steps have run.

What each result proves

The last two rows are the ones worth memorising. They are the difference between a parameter problem and an inclusion problem, and they lead to changes that point in opposite directions.

Error messages that mislead

Client libraries compress a lot of different situations into a small number of messages, and several of them actively point at the wrong layer.

A timeout is a statement about your own patience. It says you stopped waiting, not that anything failed, and the transaction it refers to may still land afterwards. Treating a timeout as a failure and immediately replacing the transaction is the single most common way traders create duplicate fills.

A blockhash error can mean your blockhash has expired or that the node you asked has not seen the block it came from. These are opposite problems. The first is your timing, the second is a lagging endpoint, and only a second provider tells you which one you have.

A simulation failure describes the state the node used for the simulation. Against stale state, a perfectly good transaction simulates badly. Against current state, the same transaction is fine. Simulation is a useful tool and a poor witness.

Rate limit responses are usually explicit, and they are the friendliest error in the set because they name their own cause. What they do not tell you is that the limit may be applied per method or per weight rather than per request, so a flow that looks modest in request count can be expensive in weight.

The endpoint layer in detail

An RPC endpoint is a node with an opinion. It has its own view of the chain, its own defaults for commitment, its own limits and its own health. Two properties matter most during diagnosis.

The first is lag. A node that has fallen behind will answer every question you ask, and every answer will describe a moment that has passed. Nothing errors. You detect this only by comparing the slot it reports against an independent provider, which is why the two-endpoint habit is worth building before you need it.

The second is limiting. Providers protect themselves, and the protection is frequently invisible until you cross a threshold, at which point a working system starts failing without any change on your side. An automated flow that scales its request rate with market activity will find this limit precisely when the market is most interesting.

This is a recurring problem for anyone running an automated Solana volume bot or a similar scheduled routine: the request rate rises exactly when conditions tighten, so an endpoint limit and a congested market arrive together and are easy to confuse. Instrumenting your own request rate, separately from your outcomes, is what keeps the two apart.

Your own layer in detail

Most of what fails here is timing. A blockhash is only valid for a bounded window of blocks, and the published limit combined with the target slot duration puts an arithmetic ceiling of about a minute on any attempt. A signing flow with a human in it, a retry loop with generous backoff, or a queue that holds transactions before sending can all exceed that ceiling without anybody noticing.

The rest is usually assumption. Commitment defaults are stricter than people expect, so a client that never sets a level explicitly may be waiting for the strongest promise the node can offer while its author believes it is waiting for the fastest. Setting the level on every call and every subscription is one line of code and removes an entire category of confusion.

Size is the third recurring fault. Transactions have a hard byte limit, and complex routes with many accounts approach it quickly. The fix is address lookup tables, which let a transaction reference accounts compactly, and the mechanism is described in the core protocol documentation rather than in any provider's guide.

The path layer in detail

The path is the layer with the least visibility and the most excuses. It contains your local connection, everything in between, and any proxy or gateway you route through. Its signature is inconsistency without pattern: requests succeed and fail in a way that correlates with nothing you can see in the chain data.

The cheapest test is to run the same trivial request from a completely different network, such as a phone connection or a remote machine in another region. If the second location is healthy while yours is not, the path is convicted and no amount of Solana knowledge will help you.

This test is skipped constantly, and it is the one that most often produces an embarrassing answer. Traders spend hours reading about validator scheduling for a problem that lives in their own connectivity.

The path layer also has a slower failure mode worth naming, because it does not look like a fault at all. Latency that has crept upwards over weeks changes what your deadlines mean without ever producing an error. A submission window that comfortably fitted your round trip when you set it may now be marginal, and the symptom is a gradual decline in landed attempts that correlates with nothing in the chain data because nothing in the chain data changed.

You catch this by measuring your own round trip on an ordinary day and writing the number down, exactly as you would for slot agreement. A latency baseline is the only way to know whether today's figure is unusual, and it costs one measurement to establish. Without it, every latency reading is just a number with no reference, and numbers without references are what make people argue about providers instead of testing them.

Resolving an unknown outcome

The dangerous state is not failure. It is not knowing. A transaction whose outcome you have not established can still be included after you have given up on it, so any action you take in the meantime has to assume it might land.

Resolve it before doing anything else. Take the signature and look it up in the Solana explorer. Three outcomes are possible and each one has a clear next move. It is present and succeeded: your position has changed and you should read your own balances before acting. It is present and failed: you have an on-chain reason and can address the parameter that caused it. It is absent: it was never included, and once its blockhash has expired it never will be.

Writing the verdict down

A diagnosis is finished when you can state the layer, name the observation that convicted it, and describe the one change you are making. Anything less is a hunch, and hunches are what fill a codebase with settings nobody can justify.

• State the layer in one sentence, without hedging across two of them.
• Quote the observation. "Endpoint B returned the current slot while endpoint A trailed" beats "A seemed slow".
• Record the time window, because it is what you will match against any official write-up later.
• Name exactly one change, and make it after the condition has ended rather than during it.
• Record what you deliberately did not change, so the next investigation starts from a known baseline.
• Note whether official status showed anything, including a clean result, with the time you checked.

Kept over a few months, these notes become the most valuable operational document you own. They tell you which endpoint has disappointed you repeatedly, which of your own assumptions has been wrong more than once, and which conditions genuinely came from the cluster. That record is what separates an operator who improves from one who keeps rebuilding the same system in a slightly different shape.

Questions the desk keeps being asked