# Errors, retries, and recovery

> Recover your API trading strategy from rejected requests, uncertain submissions, stream resets, disconnects, and process restarts.

When a request or connection fails, avoid creating duplicate or unwanted
exposure, then reconcile your orders, balances, and market data before quoting.
The same rule applies to HTTP, WebSocket, and process failures: **pause first,
reconcile second, resume last**.

<Callout type="info" title="API order placement">
For API trading, use [pre-signed orders](/docs/api-traders/pre-signed-orders).
Server-side signing may slow order ingestion. Follow the
[one-time setup](/docs/api-traders/pre-signed-orders#one-time-setup) and grant
`orders:place_signed` for signed single-order and batch submissions.
</Callout>

## Safety rules [#safety-rules]

1. A `201` or `202` response means the request was accepted, not completed.
2. No HTTP response is an unknown outcome, not a rejection.
3. Branch on a stable `code`, never on `title`, `detail`, or message text.
4. WebSocket `op: "error"` is a stream-control failure. An `op: "update"`
   carrying `data.failure` is a terminal trading event.
5. Resume quoting only after both market data and account state have been
   reseeded and reconciled.

## One HTTP failure dispatcher [#one-http-failure-dispatcher]

Origin failures use [Problem Details](/docs/errors#http-problem-details). Keep
the `request_id` in logs and follow `recovery.strategy`:

| Strategy                 | Client action                                                                                |
| ------------------------ | -------------------------------------------------------------------------------------------- |
| `none`                   | Do not repeat the unchanged request. Correct input or state, or reconcile the durable object |
| `retry`                  | Back off with jitter, then retry only after receiving this complete typed response           |
| `retry_after`            | Wait at least `after_seconds`; for `429`, also honor `Retry-After`                           |
| `refresh_identity_token` | Refresh the identity token once, then retry or reconnect                                     |
| `check_status`           | Read the identified resource before taking another action                                    |
| Unknown value            | Log it and take no automatic action                                                          |

Require a `code` your client recognizes before following the strategy. For an
unknown code, ignore any advertised automatic recovery, log the response and
`request_id`, pause the affected mutation, reconcile state, and update your
client.

`field_errors` is optional. Accept it when present, but do not require it or
infer retry behavior from it.

A received `429 rate_limited` confirms that request was not processed, so the
identical request is safe to resend after waiting. A connection timeout does
not provide that guarantee.

## Quote-readiness state machine [#quote-readiness-state-machine]

Use one public [market stream](/docs/streaming) and one private
[account stream](/docs/account-stream). Treat quoting as enabled only while
both sides are ready:

1. **Pause placements.** Continue draining any live socket into a buffer.
2. **Open both sockets.** Subscribe to the required book, quote, market-status,
   and account channels.
3. **Fence the public orderbook.** A public `subscribed` acknowledgement can
   precede subject validation. Wait for the initial orderbook `snapshot` so the
   subject is active, keep buffering, then fetch
   [`GET /trade/v1/orderbook/{token_id}`](/docs/orderbook). Parse the REST
   `hash` as the baseline sequence, discard buffered frames with
   `sequence <= hash`, and process later frames in arrival order. A `snapshot`
   replaces the whole book; a `delta` changes it. The initial WebSocket
   snapshot alone is not a readiness fence. Neither this book fence nor the
   acknowledgement proves that the market is accepting orders.
4. **Seed account state through REST.** Fully page the complete order list, then
   read the portfolio open-order view, positions, summary, trades, and
   activities. Follow every cursor to the end. Positions and any trades page
   can be partial even with a `200`. Retry the whole positions read while
   `unavailable_exchanges` includes `AGARA`. Check that field on every trades
   page; if a page includes `AGARA`, discard the entire accumulated walk and
   restart without a cursor. For a multi-exchange seed, require the array to be
   empty on every page.
5. **Reconcile buffered account records.** Use `order_id` and `order_hash` to
   correlate an order, but never discard a positive-sequence transition merely
   because either value repeats. Process those transitions in sequence order,
   deduplicate fills only by `fill_id`, and deduplicate a `sequence: 0`
   `order_rejected` notice by `order_id`.

   A live frame can arrive before REST reflects it. For a fill, poll the order's
   trades until its `fill_id` appears, then poll the order until its state
   reflects all listed fills or a later terminal transition. A fully filled
   LIMIT or MARKET order is `MATCHED`. A partially filled GTC or GTD LIMIT
   order is `PARTIALLY_FILLED` with `is_terminal: false`. After a
   `FAK_REMAINDER` cancellation for a partially filled FAK LIMIT or MARKET
   order, wait for `PARTIALLY_FILLED` with `is_terminal: true` and retain its
   fills. The cancellation concerns the unfilled remainder. A FOK order never
   partially fills. For
   another order transition, poll the order until it reflects that event or a
   later state. Apply the same
   order-versus-fill check to the REST seed, then redo the fully paginated order
   list and refresh portfolio open orders. Every fill, position, or collateral
   event is also a signal to refetch positions and summary. Require an
   event-specific result. Compare summary
   with a collateral frame only when no later fill, mint, merge, redemption,
   deposit, or withdrawal could also have changed cash; otherwise reconcile the
   complete ordered set of effects or stay paused. Every positions response
   used as evidence must exclude `AGARA` from `unavailable_exchanges`; require
   the array to be empty for a complete multi-exchange view. For mint or merge,
   require the expected settled or unwound position state. For redemption,
   require a cleared resolved-market position. When collateral, mint, or merge
   carries `batch_hash`, also require final batch status and `unwound_at` after
   a failure. Two identical reads alone are not proof. Stay paused if REST does
   not catch up or the final condition cannot be verified. If a lower batch is
   `FAILED_DIVERGENT`, do not treat a higher failed row's `unwound_at` as proof
   of restoration; keep mutations paused and follow the
   [stranded-sequence recovery
   rule](/docs/api-traders/account-batches#what-you-observe-on-failure).
6. **Read current market state through REST while buffering lifecycle frames.**
   Use the market UUID from discovery, not the stream's `condition_id`.
7. **Apply the entire ordered lifecycle buffer after the REST read.** Include
   frames received both before and during the read. Never let a stale REST
   response override a later halt, outcome proposal, or resolution. If an
   observed tradeability-changing frame and REST disagree, stay paused and poll
   until REST catches up.
8. **Treat any lifecycle gap as unresolved.** `market_status` is live-only and
   has no snapshot, replay, or subscription-time baseline marker. After
   startup, disconnect, or reset, seeing no lifecycle frame does not prove that
   none was missed, and a REST response can briefly lag a change. The current
   APIs cannot prove the latest lifecycle state automatically. A strict market
   maker must stay paused until a human or its documented risk policy accepts
   the current REST state.
9. **Resume placements** only after that lifecycle decision, when REST reports
   `is_accepting_orders: true`, no observed lifecycle frame vetoes quoting, and
   the public orderbook and private account view are current. Do not use
   `best_quote` alone for readiness: it reports the best prices and no depth.
10. On a reset, disconnect, or `action: "reconnect"`, return to step 1.

The endpoints used during the REST seed are:

| State                                               | Endpoint                                                                                                                                                                                                    |
| --------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Current public orderbook and sequence baseline      | [`GET /trade/v1/orderbook/{token_id}`](/docs/orderbook); fetch it only after the orderbook subscription is active and buffering                                                                             |
| Current market lifecycle and tradeability           | [`GET /api/v1/markets/{id}`](/docs/api-reference/market-data/markets/get-api-v1-markets-by-id), where `{id}` is the market UUID from discovery, not its `condition_id`; require `is_accepting_orders: true` |
| Complete order lifecycle, including pre-book orders | [`POST /trade/v1/orders/list`](/docs/orders#list-your-orders); follow every cursor                                                                                                                          |
| Resting and active portfolio view                   | [`POST /trade/v1/portfolio/open-orders/list`](/docs/portfolio#open-orders)                                                                                                                                  |
| Positions and inventory                             | [`POST /trade/v1/portfolio/positions/list`](/docs/portfolio#positions)                                                                                                                                      |
| Collateral and account summary                      | [`GET /trade/v1/portfolio/summary`](/docs/portfolio#summary)                                                                                                                                                |
| Fill history                                        | [`GET /trade/v1/portfolio/trades`](/docs/portfolio#trades)                                                                                                                                                  |
| Durable account activity                            | [`GET /trade/v1/portfolio/activities`](/docs/portfolio#activities)                                                                                                                                          |
| One order                                           | [`GET /trade/v1/orders/{order_id}`](/docs/orders#look-up-one-order)                                                                                                                                         |
| One signed order by hash                            | [`GET /trade/v1/orders/by-hash/{order_hash}`](/docs/orders#look-up-an-order-by-hash)                                                                                                                        |
| One order's fills                                   | [`GET /trade/v1/orders/{order_id}/trades`](/docs/orders#list-an-orders-fills)                                                                                                                               |

## Ordinary orders [#ordinary-orders]

### Immediate rejection [#immediate-rejection]

An order rejected before acceptance returns a complete Problem Details body:

```json
{
  "type": "urn:agara:problem:insufficient-balance",
  "title": "Insufficient balance",
  "status": 422,
  "code": "insufficient_balance",
  "detail": "Available balance is lower than the order value.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "none"
  }
}
```

| Status and code                                                    | Client action                                                              |
| ------------------------------------------------------------------ | -------------------------------------------------------------------------- |
| `400 malformed_json`                                               | Correct JSON serialization                                                 |
| `409 sequence_conflict`                                            | Reconcile account state before another mutation                            |
| `422 validation_failed`                                            | Correct request fields or client-side validation                           |
| `422 order_value_below_minimum` or `422 order_value_above_maximum` | Resize the order into the notional band                                    |
| `422 order_size_below_minimum` or `422 order_size_above_maximum`   | Resize the market sell into the share band                                 |
| `422 order_price_out_of_range`                                     | Send a `price_micro` above 0 and below 1,000,000                           |
| `422 order_expiration_too_soon`                                    | Move the GTD expiration at least 30 seconds out                            |
| `422 insufficient_balance`                                         | Reconcile collateral and open BUY commitments; reduce size                 |
| `422 insufficient_shares`                                          | Reconcile positions and open SELL commitments; restore or reduce inventory |
| `422 market_closed`                                                | Stop quoting that market                                                   |
| `422 order_not_fillable`                                           | Refresh the book, then resize or reprice                                   |
| `422 post_only_would_cross`                                        | Refresh top of book, then move the quote away from the spread              |

### Accepted order [#accepted-order]

After `202`, follow the durable order row and the account stream. A terminal
`FAILED` or `REJECTED` row carries `failure`; other states carry `null`.
`order_rejected` is emitted for both terminal states and contains the same
typed failure, but REST remains authoritative.

### No placement response [#no-placement-response]

Ordinary order placement has no idempotency key. If the connection ends before
you receive a response:

1. Pause new orders for the affected outcome.
2. Fully page the order list and compare `token_id`, side, type, and creation
   time with the ambiguous intent. For a LIMIT order compare price and original
   shares; for a MARKET BUY compare `collateral_amount_micro`; for a MARKET SELL
   compare original shares. Use the lifecycle history to follow candidate IDs;
   an active-order snapshot alone cannot tell you how a terminal candidate ended.
3. If you cannot identify the result safely, cancel the plausible candidates
   or use cancel-all.
4. Keep reconciling: cancel-all covers orders visible when it begins, but it
   does not fence a placement that is still completing. Cancel a late order or
   repeat cancel-all if one appears afterward.
5. Resume only after the ambiguous placement is resolved and a fully paginated
   order list shows no unwanted nonterminal order. Refresh portfolio open orders
   before sizing new commitments. If you cannot prove that, keep the outcome
   paused.

Never blindly resend an ordinary order after a timeout.

## Signed orders and signed batches [#signed-orders-and-signed-batches]

### One signed order [#one-signed-order]

You know `order_hash` before sending, so a lost response is recoverable through
`GET /trade/v1/orders/by-hash/{order_hash}`. If a replay reaches the existing
identity check, one possible response is:

```json
{
  "type": "urn:agara:problem:duplicate-order",
  "title": "Duplicate order",
  "status": 409,
  "code": "duplicate_order",
  "detail": "An order with this identity already exists.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "none"
  }
}
```

Look up the existing order. Do not treat `duplicate_order` as a new rejection
of the trading intent. The server repeats current market, wallet, signature,
and admission checks before checking for an existing identity, so a replay can
instead return a current validation or trading failure. The hash lookup, not a
particular replay response, is authoritative for whether the order exists.

The server deliberately returns generic `422 validation_failed` for invalid
signed envelopes. Before submission, validate chain ID, exchange address,
maker, hash, low-s signature, and recovered owner locally; do not parse public
text to identify the failed check.

### Mixed signed batch [#mixed-signed-batch]

The HTTP request can succeed while individual entries are rejected:

```json
{
  "results": [
    {
      "index": 0,
      "outcome": "accepted",
      "order_id": "9c4a3e7d-12b4-4f8e-9a3c-d2c7f0a45e1b",
      "source": "AGARA",
      "status": "PENDING",
      "pending_operation": "SUBMIT",
      "as_of": "2026-05-21T17:20:01.633Z"
    },
    {
      "index": 1,
      "outcome": "rejected",
      "failure": {
        "code": "duplicate_order",
        "title": "Duplicate order",
        "detail": "An order with this identity already exists.",
        "recovery": {
          "strategy": "none"
        }
      }
    }
  ],
  "as_of": "2026-05-21T17:20:01.633Z"
}
```

Valid per-item failure codes are `validation_failed`, `order_value_below_minimum`,
`order_value_above_maximum`, `order_price_out_of_range`,
`order_expiration_too_soon`, `insufficient_balance`, `insufficient_shares`,
`market_closed`, `order_not_fillable`, `post_only_would_cross`, and
`duplicate_order`.

If the whole batch response is lost, look up every submitted hash. Retry only
the signed envelopes whose hashes are absent. Exact replay cannot create a
second identity. An existing hash can return `duplicate_order` if replay reaches
the identity check, or a current preflight failure before that check; reconcile
it as the existing order found by the hash lookup.

## Account batches [#account-batches]

An account batch has a deterministic `batch_hash`. Keep the hash before
sending. An exact replay cannot create a second batch identity. However,
current wallet, market, deadline, value, and signature checks are reevaluated
and can return another typed failure instead of `201`.

| What you observe                 | Action                                                                                                                                                          |
| -------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `201`                            | The batch exists; poll it to a final state                                                                                                                      |
| No response                      | Read the precomputed hash first. If it is absent, a replay still has to pass the current preflight checks; do not require `201` as proof of the earlier outcome |
| `502 dependency_outcome_unknown` | Follow `recovery.resource.batch_hash`; do not submit another mutation first                                                                                     |

An uncertain submission identifies the object to read:

```json
{
  "type": "urn:agara:problem:dependency-outcome-unknown",
  "title": "Submission status is uncertain",
  "status": 502,
  "code": "dependency_outcome_unknown",
  "detail": "Your request may have succeeded. Check your positions before trying again.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "check_status",
    "resource": {
      "kind": "batch",
      "batch_hash": "0x51cda675bcc49a94f2c4ace692aa542d182dcb43ffe000da5d343bb49a441574"
    }
  }
}
```

### Terminal batch failure [#terminal-batch-failure]

Poll `GET /trade/v1/batches/{batch_hash}`. A failed batch exposes a typed
`failure`, not private failure text:

```json
{
  "batch_hash": "0x51cda675bcc49a94f2c4ace692aa542d182dcb43ffe000da5d343bb49a441574",
  "status": "FAILED",
  "seq": 4,
  "deadline_unix_seconds": 1784022000,
  "origin": "PRESIGNED",
  "tx_hash": null,
  "executed_at": null,
  "failure": {
    "code": "sequence_conflict",
    "title": "Sequence conflict",
    "detail": "Order submission failed, please retry.",
    "recovery": {
      "strategy": "none"
    }
  },
  "superseded_by_batch_hash": null,
  "heals_batch_hash": null,
  "unwound_at": "2026-07-14T09:14:41Z",
  "created_at": "2026-07-14T09:12:03Z"
}
```

| Terminal code                                  | Action                                                                                                                                                                                                                                                                   |
| ---------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `batch_superseded`                             | Follow `superseded_by_batch_hash`                                                                                                                                                                                                                                        |
| `sequence_conflict`                            | Inspect lower queue rows. If a lower batch is `FAILED_DIVERGENT`, stay paused and escalate; do not trust this row's unwind marker or sign again. Otherwise wait for unwind, then reconcile the on-chain sequence, balances, positions, and account events before signing |
| `chain_execution_failed`                       | Wait for unwind and correct the operation before signing fresh                                                                                                                                                                                                           |
| `insufficient_balance` / `insufficient_shares` | Reconcile collateral and positions before signing fresh                                                                                                                                                                                                                  |
| `settlement_divergent`                         | Halt account mutations and escalate; do not self-retry                                                                                                                                                                                                                   |
| `internal_error`                               | Retain the batch hash and request ID, then escalate                                                                                                                                                                                                                      |

Keep polling the durable batch even when account-stream events show position or
collateral changes.

## WebSocket recovery [#websocket-recovery]

WebSocket failures combine two instructions:

* `failure.recovery.strategy` describes the underlying failure.
* `action` requests `resubscribe`, `reconnect`, or no follow-up action. An
  `action: "none"` failure can still close the socket; treat the close code and
  actual socket state as authoritative. See the full
  [WebSocket error matrix](/docs/streaming#error).

Honor these instructions only for a `failure.code` your client recognizes. For
an unknown code, treat both as `none`, pause the affected work, log the frame,
and update the client. A transport close still requires reconnect-and-reseed.

### Rebuild one subject [#rebuild-one-subject]

```json
{
  "op": "error",
  "failure": {
    "code": "stream_subject_unavailable",
    "title": "Stream subject temporarily unavailable",
    "detail": "This stream subject is temporarily unavailable.",
    "recovery": {
      "strategy": "none"
    }
  },
  "action": "resubscribe",
  "channel": "orderbook",
  "token_id": "21742633..."
}
```

Pause quotes for that subject, discard its local book, resubscribe, and begin
buffering. After the initial orderbook `snapshot` proves the subject is active, fetch
the REST book and apply the sequence-baseline procedure above. Do not resume
from the WebSocket snapshot alone. Other subscriptions can keep buffering.

### Rebuild both sides [#rebuild-both-sides]

```json
{
  "op": "error",
  "failure": {
    "code": "server_restarting",
    "title": "Server restarting",
    "detail": "The server is restarting. Reconnect after a short delay.",
    "recovery": {
      "strategy": "retry"
    }
  },
  "action": "reconnect"
}
```

This frame is followed by close code `1012`. Return to the full
[quote-readiness state machine](#quote-readiness-state-machine).

| Failure                      | Action        |  Close | Client response                                                                                                                                                                                                                                                      |
| ---------------------------- | ------------- | -----: | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `identity_token_expired`     | `reconnect`   | `4001` | Refresh once, reconnect the account socket, and reseed both views                                                                                                                                                                                                    |
| `dependency_unavailable`     | `reconnect`   | `1013` | Back off, reconnect, and reseed both views                                                                                                                                                                                                                           |
| `internal_error`             | `reconnect`   | `1011` | Reconnect and reseed; retain diagnostics                                                                                                                                                                                                                             |
| `server_restarting`          | `reconnect`   | `1012` | Reconnect after a short backoff and reseed                                                                                                                                                                                                                           |
| `slow_consumer`              | `reconnect`   | `4008` | Make frame reading fast enough before reconnecting                                                                                                                                                                                                                   |
| `stream_subject_unavailable` | `resubscribe` |      — | Recover by channel: rebuild `orderbook` with the active-subscription + REST-watermark procedure; resubscribe `best_quote` and take its first update as current; accept the public-trades gap; apply the REST, buffer, and lifecycle-gap decision for `market_status` |
| `market_closed`              | `none`        |      — | Stop quoting that market                                                                                                                                                                                                                                             |

On `sequence_reset`, the subscription remains active but messages were missed.
For an orderbook, discard local state and wait for the first post-reset
`snapshot` while buffering. Its arrival proves the live path is active again,
but it is not a state fence. Continue buffering, fetch the REST book, parse
`hash` as the baseline sequence, discard buffered frames with
`sequence <= hash`, then process newer frames in arrival order. A frame with
`data.kind: "snapshot"` replaces the entire baseline; a `delta` changes the
current baseline. Restart recovery if another reset arrives. If you resubscribe
instead, use the same snapshot-then-REST order. For account events,
keep buffering and redo the complete REST seed. Correlate order transitions by
`order_id` or `order_hash`;
never discard positive-sequence transitions because those IDs repeat.
Deduplicate fills only by `fill_id`, and deduplicate a `sequence: 0`
`order_rejected` notice by `order_id`. Before rebuilding open orders, wait for
each transition to become durable and for the order state to reflect all its
durable fills or a later terminal transition. Then redo the fully paginated
order list and refresh portfolio open orders. Use position and collateral
effects only to trigger fresh positions and summary reads. Require the
event-specific collateral, batch, or redemption condition from the readiness
procedure; repeated identical reads alone are not proof that REST includes the
event.

Read the current market through REST while buffering lifecycle frames, then
apply the entire ordered buffer. If REST and the latest lifecycle frame
disagree, stay paused and poll until they agree; never let stale REST state
override a later stop event. A transport close can arrive without an error
frame; treat it as reconnect-and-reseed. Because `market_status` is live-only,
this does not prove that no lifecycle event was missed during a gap. Do not
resume automatically without the lifecycle decision in the readiness
procedure.

Read and enqueue each frame promptly. Perform strategy calculations outside
the socket-reading path; reconnecting without fixing a `slow_consumer` loop
will repeat the failure.

## Crash and restart runbook [#crash-and-restart-runbook]

1. Stop the strategy loop and block new placements.
2. Request [cancel-all](/docs/api-traders/operations#cancel-all), but do not
   treat it as a fence for a placement that may still be completing.
3. Resolve each ambiguous ordinary order, signed order hash, and batch hash.
   Cancel a late order or repeat cancel-all if one appears after the first
   sweep.
4. Require a fully paginated order list to show no unwanted nonterminal order,
   then refresh portfolio open orders. A `202` cancel-all response is not
   completion; if an ambiguous placement remains unresolved, stay paused.
5. Open and buffer both WebSocket streams.
6. Seed every authoritative REST source in the table above.
7. Reconcile buffered records. Process every positive-sequence order
   transition; deduplicate only fills by `fill_id` and `sequence: 0`
   rejections by `order_id`. For a fill, require both its durable trade row and
   the corresponding order state to reflect all listed fills before redoing the
   fully paginated order list and refreshing portfolio open orders. Refetch
   inventory after every fill, position, or collateral event and require its
   event-specific final condition. Rebuild the orderbook with the
   active-subscription + REST-watermark procedure; stay paused if any durable
   state remains behind.
8. Read current market state while buffering lifecycle frames, then apply the
   entire ordered buffer, including frames from before and during the read.
   Poll any disagreement until REST catches up, and never let a stale REST
   response override a later stop event. This does not reconstruct events lost
   during the crash, so keep quoting paused until the lifecycle risk decision
   from the readiness procedure is satisfied.
9. Re-enable quoting only when public and account state are both ready.

A WebSocket disconnect does not cancel resting orders. When local state is
uncertain, cancelling and rebuilding is safer than assuming silence means the
book is empty.