> ## Documentation Index
> Fetch the complete documentation index at: https://docs.stateset.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Replay one eval and grade it against the recorded answer

> Runs the real generation path, then grades the reply against preferred_output on substance rather than wording…

Runs the real generation path, then grades the reply against preferred\_output on substance rather than wording. `passed` is null — never false — when grading could not run, so an unavailable judge is not mistaken for a failing agent.

### Path parameters

<ParamField path="id" type="string (uuid)" required>
  The eval id.
</ParamField>

### Request body

`object`

<ParamField body="agent_id" type="string (uuid)">
  Agent to replay against. Defaults to the eval's own `agent_id`; 400 if neither is set.
</ParamField>

### Response

`object`

<ResponseField name="object" type="string" required />

<ResponseField name="eval_id" type="string (uuid)" required />

<ResponseField name="agent_id" type="string (uuid)" required>
  The agent the eval was replayed against (the request `agent_id`, else the eval's own).
</ResponseField>

<ResponseField name="user_message" type="string,null">
  The replayed customer message. Absent when the run itself failed (`reason: 'run_failed'`).
</ResponseField>

<ResponseField name="expected" type="string,null">
  The eval's `preferred_output`. Absent when the run itself failed.
</ResponseField>

<ResponseField name="actual" type="string,null">
  The agent's reply on this run; null when the agent returned no reply. Absent when the run itself failed.
</ResponseField>

<ResponseField name="passed" type="boolean,null" required>
  True when `verdict` is `pass`, false when it is `partial` or `fail`, and null when the reply was not graded — 'the judge was unavailable' and 'the agent got it wrong' are kept distinct.
</ResponseField>

<ResponseField name="graded" type="boolean" required>
  Whether the judge produced a verdict. When false, `reason` says why and the verdict fields are absent.
</ResponseField>

<ResponseField name="reason" type="string">
  Present only when `graded` is false.
</ResponseField>

<ResponseField name="error" type="string">
  Present only when `reason` is `run_failed` (a run inside a suite threw); the public error message.
</ResponseField>

<ResponseField name="duration_ms" type="integer">
  Generation plus judge time. Absent when the run itself failed.
</ResponseField>

<ResponseField name="verdict" type="string">
  Present only when `graded` is true. Judged on substance, not wording.
</ResponseField>

<ResponseField name="equivalence" type="number,null">
  Present only when `graded` is true. 0–5, how closely the substance matches (5 = equivalent); one decimal.
</ResponseField>

<ResponseField name="contradicts" type="boolean,null">
  Present only when `graded` is true.
</ResponseField>

<ResponseField name="omits" type="string[]">
  Present only when `graded` is true. Essential points from the reference that the reply is missing.
</ResponseField>

<ResponseField name="differences" type="string,null">
  Present only when `graded` is true.
</ResponseField>

<ResponseField name="explanation" type="string,null">
  Present only when `graded` is true. The judge's reasoning, so a caller can disagree with the grade.
</ResponseField>

### Status codes

| Code  | Meaning                                                                                                                                                                              |
| ----- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `200` | The replay result: the agent's reply on this run, graded against the eval's `preferred_output` by a reference-based judge. `passed` is null (not false) when grading did not happen. |
| `400` | Invalid request. `issues` names the offending field(s).                                                                                                                              |
| `401` | Missing, invalid, or expired API key.                                                                                                                                                |
| `403` | The key lacks a required scope.                                                                                                                                                      |
| `404` | Not found, or not in the organization that owns the key.                                                                                                                             |
| `429` | Rate limit exceeded.                                                                                                                                                                 |

### Access

|                |                        |
| -------------- | ---------------------- |
| Required scope | `responses:write`      |
| Rate limit     | 30 requests per minute |

<RequestExample>
  ```bash cURL theme={null}
  curl --request POST \
    --url 'https://response.stateset.com/api/v1/evals/{id}/run' \
    --header "Authorization: Bearer $RESPONSECX_API_KEY" \
    --header 'Content-Type: application/json' \
    --data '{
    "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"
  }'
  ```
</RequestExample>

<ResponseExample>
  ```json 200 theme={null}
  {
    "object": "eval.run",
    "eval_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "agent_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6",
    "user_message": "string",
    "expected": "string",
    "actual": "string",
    "passed": true,
    "graded": true,
    "reason": "judge_unavailable",
    "error": "string",
    "duration_ms": 8,
    "verdict": "pass",
    "equivalence": 1.5,
    "contradicts": true,
    "omits": [
      "string"
    ],
    "differences": "string",
    "explanation": "string"
  }
  ```
</ResponseExample>