Errors, retries, and recovery

When a request or connection fails, avoid creating duplicate or unwanted exposure, then reconcile your orders, balances, and market data before quoting. The same rule applies to HTTP, WebSocket, and process failures: pause first, reconcile second, resume last.

Safety rules

1. A 201 or 202 response means the request was accepted, not completed.
2. No HTTP response is an unknown outcome, not a rejection.
3. Branch on a stable code, never on title, detail, or message text.
4. WebSocket op: "error" is a stream-control failure. An op: "update" carrying data.failure is a terminal trading event.
5. Resume quoting only after both market data and account state have been reseeded and reconciled.

One HTTP failure dispatcher

Origin failures use Problem Details. Keep the request_id in logs and follow recovery.strategy:

Require a code your client recognizes before following the strategy. For an unknown code, ignore any advertised automatic recovery, log the response and request_id, pause the affected mutation, reconcile state, and update your client.

field_errors is optional. Accept it when present, but do not require it or infer retry behavior from it.

A received 429 rate_limited confirms that request was not processed, so the identical request is safe to resend after waiting. A connection timeout does not provide that guarantee.

Quote-readiness state machine

Use one public market stream and one private account stream. Treat quoting as enabled only while both sides are ready:

1. Pause placements. Continue draining any live socket into a buffer.

2. Open both sockets. Subscribe to the required book, quote, market-status, and account channels.

3. Fence the public orderbook. A public subscribed acknowledgement can precede subject validation. Wait for the initial orderbook snapshot so the subject is active, keep buffering, then fetch GET /trade/v1/orderbook/{token_id}. Parse the REST hash as the baseline sequence, discard buffered frames with sequence <= hash, and process later frames in arrival order. A snapshot replaces the whole book; a delta changes it. The initial WebSocket snapshot alone is not a readiness fence. Neither this book fence nor the acknowledgement proves that the market is accepting orders.

4. Seed account state through REST. Fully page the complete order list, then read the portfolio open-order view, positions, summary, trades, and activities. Follow every cursor to the end. Positions and any trades page can be partial even with a 200. Retry the whole positions read while unavailable_exchanges includes AGARA. Check that field on every trades page; if a page includes AGARA, discard the entire accumulated walk and restart without a cursor. For a multi-exchange seed, require the array to be empty on every page.

5. Reconcile buffered account records. Use order_id and order_hash to correlate an order, but never discard a positive-sequence transition merely because either value repeats. Process those transitions in sequence order, deduplicate fills only by fill_id, and deduplicate a sequence: 0 order_rejected notice by order_id.

   A live frame can arrive before REST reflects it. For a fill, poll the order's trades until its fill_id appears, then poll the order until its state reflects all listed fills or a later terminal transition. A fully filled LIMIT or MARKET order is MATCHED. A partially filled GTC or GTD LIMIT order is PARTIALLY_FILLED with is_terminal: false. After a FAK_REMAINDER cancellation for a partially filled FAK LIMIT or MARKET order, wait for PARTIALLY_FILLED with is_terminal: true and retain its fills. The cancellation concerns the unfilled remainder. A FOK order never partially fills. For another order transition, poll the order until it reflects that event or a later state. Apply the same order-versus-fill check to the REST seed, then redo the fully paginated order list and refresh portfolio open orders. Every fill, position, or collateral event is also a signal to refetch positions and summary. Require an event-specific result. Compare summary with a collateral frame only when no later fill, mint, merge, redemption, deposit, or withdrawal could also have changed cash; otherwise reconcile the complete ordered set of effects or stay paused. Every positions response used as evidence must exclude AGARA from unavailable_exchanges; require the array to be empty for a complete multi-exchange view. For mint or merge, require the expected settled or unwound position state. For redemption, require a cleared resolved-market position. When collateral, mint, or merge carries batch_hash, also require final batch status and unwound_at after a failure. Two identical reads alone are not proof. Stay paused if REST does not catch up or the final condition cannot be verified. If a lower batch is FAILED_DIVERGENT, do not treat a higher failed row's unwound_at as proof of restoration; keep mutations paused and follow the stranded-sequence recovery rule.

6. Read current market state through REST while buffering lifecycle frames. Use the market UUID from discovery, not the stream's condition_id.

7. Apply the entire ordered lifecycle buffer after the REST read. Include frames received both before and during the read. Never let a stale REST response override a later halt, outcome proposal, or resolution. If an observed tradeability-changing frame and REST disagree, stay paused and poll until REST catches up.

8. Treat any lifecycle gap as unresolved. market_status is live-only and has no snapshot, replay, or subscription-time baseline marker. After startup, disconnect, or reset, seeing no lifecycle frame does not prove that none was missed, and a REST response can briefly lag a change. The current APIs cannot prove the latest lifecycle state automatically. A strict market maker must stay paused until a human or its documented risk policy accepts the current REST state.

9. Resume placements only after that lifecycle decision, when REST reports is_accepting_orders: true, no observed lifecycle frame vetoes quoting, and the public orderbook and private account view are current. Do not use best_quote alone for readiness: it reports the best prices and no depth.

10. On a reset, disconnect, or action: "reconnect", return to step 1.

The endpoints used during the REST seed are:

Ordinary orders

Immediate rejection

An order rejected before acceptance returns a complete Problem Details body:

{
  "type": "urn:agara:problem:insufficient-balance",
  "title": "Insufficient balance",
  "status": 422,
  "code": "insufficient_balance",
  "detail": "Available balance is lower than the order value.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "none"
  }
}

Accepted order

After 202, follow the durable order row and the account stream. A terminal FAILED or REJECTED row carries failure; other states carry null. order_rejected is emitted for both terminal states and contains the same typed failure, but REST remains authoritative.

No placement response

Ordinary order placement has no idempotency key. If the connection ends before you receive a response:

1. Pause new orders for the affected outcome.
2. Fully page the order list and compare token_id, side, type, and creation time with the ambiguous intent. For a LIMIT order compare price and original shares; for a MARKET BUY compare collateral_amount_micro; for a MARKET SELL compare original shares. Use the lifecycle history to follow candidate IDs; an active-order snapshot alone cannot tell you how a terminal candidate ended.
3. If you cannot identify the result safely, cancel the plausible candidates or use cancel-all.
4. Keep reconciling: cancel-all covers orders visible when it begins, but it does not fence a placement that is still completing. Cancel a late order or repeat cancel-all if one appears afterward.
5. Resume only after the ambiguous placement is resolved and a fully paginated order list shows no unwanted nonterminal order. Refresh portfolio open orders before sizing new commitments. If you cannot prove that, keep the outcome paused.

Never blindly resend an ordinary order after a timeout.

Signed orders and signed batches

One signed order

You know order_hash before sending, so a lost response is recoverable through GET /trade/v1/orders/by-hash/{order_hash}. If a replay reaches the existing identity check, one possible response is:

{
  "type": "urn:agara:problem:duplicate-order",
  "title": "Duplicate order",
  "status": 409,
  "code": "duplicate_order",
  "detail": "An order with this identity already exists.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "none"
  }
}

Look up the existing order. Do not treat duplicate_order as a new rejection of the trading intent. The server repeats current market, wallet, signature, and admission checks before checking for an existing identity, so a replay can instead return a current validation or trading failure. The hash lookup, not a particular replay response, is authoritative for whether the order exists.

The server deliberately returns generic 422 validation_failed for invalid signed envelopes. Before submission, validate chain ID, exchange address, maker, hash, low-s signature, and recovered owner locally; do not parse public text to identify the failed check.

Mixed signed batch

The HTTP request can succeed while individual entries are rejected:

{
  "results": [
    {
      "index": 0,
      "outcome": "accepted",
      "order_id": "9c4a3e7d-12b4-4f8e-9a3c-d2c7f0a45e1b",
      "source": "AGARA",
      "status": "PENDING",
      "pending_operation": "SUBMIT",
      "as_of": "2026-05-21T17:20:01.633Z"
    },
    {
      "index": 1,
      "outcome": "rejected",
      "failure": {
        "code": "duplicate_order",
        "title": "Duplicate order",
        "detail": "An order with this identity already exists.",
        "recovery": {
          "strategy": "none"
        }
      }
    }
  ],
  "as_of": "2026-05-21T17:20:01.633Z"
}

Valid per-item failure codes are validation_failed, order_value_below_minimum, order_value_above_maximum, order_price_out_of_range, order_expiration_too_soon, insufficient_balance, insufficient_shares, market_closed, order_not_fillable, post_only_would_cross, and duplicate_order.

If the whole batch response is lost, look up every submitted hash. Retry only the signed envelopes whose hashes are absent. Exact replay cannot create a second identity. An existing hash can return duplicate_order if replay reaches the identity check, or a current preflight failure before that check; reconcile it as the existing order found by the hash lookup.

Account batches

An account batch has a deterministic batch_hash. Keep the hash before sending. An exact replay cannot create a second batch identity. However, current wallet, market, deadline, value, and signature checks are reevaluated and can return another typed failure instead of 201.

An uncertain submission identifies the object to read:

{
  "type": "urn:agara:problem:dependency-outcome-unknown",
  "title": "Submission status is uncertain",
  "status": 502,
  "code": "dependency_outcome_unknown",
  "detail": "Your request may have succeeded. Check your positions before trying again.",
  "request_id": "8051f907-4eeb-4ca9-bd84-b447ea65a68c",
  "recovery": {
    "strategy": "check_status",
    "resource": {
      "kind": "batch",
      "batch_hash": "0x51cda675bcc49a94f2c4ace692aa542d182dcb43ffe000da5d343bb49a441574"
    }
  }
}

Terminal batch failure

Poll GET /trade/v1/batches/{batch_hash}. A failed batch exposes a typed failure, not private failure text:

{
  "batch_hash": "0x51cda675bcc49a94f2c4ace692aa542d182dcb43ffe000da5d343bb49a441574",
  "status": "FAILED",
  "seq": 4,
  "deadline_unix_seconds": 1784022000,
  "origin": "PRESIGNED",
  "tx_hash": null,
  "executed_at": null,
  "failure": {
    "code": "sequence_conflict",
    "title": "Sequence conflict",
    "detail": "Order submission failed, please retry.",
    "recovery": {
      "strategy": "none"
    }
  },
  "superseded_by_batch_hash": null,
  "heals_batch_hash": null,
  "unwound_at": "2026-07-14T09:14:41Z",
  "created_at": "2026-07-14T09:12:03Z"
}

Keep polling the durable batch even when account-stream events show position or collateral changes.

WebSocket recovery

WebSocket failures combine two instructions:

• failure.recovery.strategy describes the underlying failure.
• action requests resubscribe, reconnect, or no follow-up action. An action: "none" failure can still close the socket; treat the close code and actual socket state as authoritative. See the full WebSocket error matrix.

Honor these instructions only for a failure.code your client recognizes. For an unknown code, treat both as none, pause the affected work, log the frame, and update the client. A transport close still requires reconnect-and-reseed.

Rebuild one subject

{
  "op": "error",
  "failure": {
    "code": "stream_subject_unavailable",
    "title": "Stream subject temporarily unavailable",
    "detail": "This stream subject is temporarily unavailable.",
    "recovery": {
      "strategy": "none"
    }
  },
  "action": "resubscribe",
  "channel": "orderbook",
  "token_id": "21742633..."
}

Pause quotes for that subject, discard its local book, resubscribe, and begin buffering. After the initial orderbook snapshot proves the subject is active, fetch the REST book and apply the sequence-baseline procedure above. Do not resume from the WebSocket snapshot alone. Other subscriptions can keep buffering.

Rebuild both sides

{
  "op": "error",
  "failure": {
    "code": "server_restarting",
    "title": "Server restarting",
    "detail": "The server is restarting. Reconnect after a short delay.",
    "recovery": {
      "strategy": "retry"
    }
  },
  "action": "reconnect"
}

This frame is followed by close code 1012. Return to the full quote-readiness state machine.

On sequence_reset, the subscription remains active but messages were missed. For an orderbook, discard local state and wait for the first post-reset snapshot while buffering. Its arrival proves the live path is active again, but it is not a state fence. Continue buffering, fetch the REST book, parse hash as the baseline sequence, discard buffered frames with sequence <= hash, then process newer frames in arrival order. A frame with data.kind: "snapshot" replaces the entire baseline; a delta changes the current baseline. Restart recovery if another reset arrives. If you resubscribe instead, use the same snapshot-then-REST order. For account events, keep buffering and redo the complete REST seed. Correlate order transitions by order_id or order_hash; never discard positive-sequence transitions because those IDs repeat. Deduplicate fills only by fill_id, and deduplicate a sequence: 0 order_rejected notice by order_id. Before rebuilding open orders, wait for each transition to become durable and for the order state to reflect all its durable fills or a later terminal transition. Then redo the fully paginated order list and refresh portfolio open orders. Use position and collateral effects only to trigger fresh positions and summary reads. Require the event-specific collateral, batch, or redemption condition from the readiness procedure; repeated identical reads alone are not proof that REST includes the event.

Read the current market through REST while buffering lifecycle frames, then apply the entire ordered buffer. If REST and the latest lifecycle frame disagree, stay paused and poll until they agree; never let stale REST state override a later stop event. A transport close can arrive without an error frame; treat it as reconnect-and-reseed. Because market_status is live-only, this does not prove that no lifecycle event was missed during a gap. Do not resume automatically without the lifecycle decision in the readiness procedure.

Read and enqueue each frame promptly. Perform strategy calculations outside the socket-reading path; reconnecting without fixing a slow_consumer loop will repeat the failure.

Crash and restart runbook

1. Stop the strategy loop and block new placements.
2. Request cancel-all, but do not treat it as a fence for a placement that may still be completing.
3. Resolve each ambiguous ordinary order, signed order hash, and batch hash. Cancel a late order or repeat cancel-all if one appears after the first sweep.
4. Require a fully paginated order list to show no unwanted nonterminal order, then refresh portfolio open orders. A 202 cancel-all response is not completion; if an ambiguous placement remains unresolved, stay paused.
5. Open and buffer both WebSocket streams.
6. Seed every authoritative REST source in the table above.
7. Reconcile buffered records. Process every positive-sequence order transition; deduplicate only fills by fill_id and sequence: 0 rejections by order_id. For a fill, require both its durable trade row and the corresponding order state to reflect all listed fills before redoing the fully paginated order list and refreshing portfolio open orders. Refetch inventory after every fill, position, or collateral event and require its event-specific final condition. Rebuild the orderbook with the active-subscription + REST-watermark procedure; stay paused if any durable state remains behind.
8. Read current market state while buffering lifecycle frames, then apply the entire ordered buffer, including frames from before and during the read. Poll any disagreement until REST catches up, and never let a stale REST response override a later stop event. This does not reconstruct events lost during the crash, so keep quoting paused until the lifecycle risk decision from the readiness procedure is satisfied.
9. Re-enable quoting only when public and account state are both ready.

A WebSocket disconnect does not cancel resting orders. When local state is uncertain, cancelling and rebuilding is safer than assuming silence means the book is empty.