# Python API reference

Reference for the `river_client` Python package, generated from river-client `0.6.1`. The [guide](/) covers the same API as a walkthrough; this page lists every public class, method, and type.

<!-- Generated by scripts/generate_python_api.py. Do not edit by hand; regenerate with `uv run services/river-docs/scripts/generate_python_api.py`. -->

---

```bash
pip install river-client
```

```python
import river_client as river
```

---

## Client

River API client.

Connects to the River API server over gRPC with automatic retry on
transient failures and connection keepalive.

```python
Client(
    api_key: str,
    endpoint: str = 'api.river.ai',
    port: int = 443,
    timeout: float = 86400.0,
    use_ssl: bool = True,
    enable_retries: bool = True,
)
```

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `api_key` | `str` |  | API key for authentication |
| `endpoint` | `str` | `'api.river.ai'` | API endpoint hostname |
| `port` | `int` | `443` | API port |
| `timeout` | `float` | `86400.0` | Default timeout for operations |
| `use_ssl` | `bool` | `True` | Whether to use SSL |
| `enable_retries` | `bool` | `True` | Whether submit RPCs and gRPC transport may retry transient failures. Disable this for fail-closed evaluation protocols that require one server submission per model turn. |

**Methods:** [`session`](#clientsession), [`sample`](#clientsample), [`health_check`](#clienthealth_check), [`get_capabilities`](#clientget_capabilities), [`get_streaming_replica`](#clientget_streaming_replica), [`promote_streaming_replica`](#clientpromote_streaming_replica), [`chat_complete_stream`](#clientchat_complete_stream), [`chat_complete`](#clientchat_complete), [`chat_complete_from_checkpoint`](#clientchat_complete_from_checkpoint), [`chat_complete_from_training`](#clientchat_complete_from_training), [`close`](#clientclose)

### Client.session

```python
Client.session(*, timeout: float = 86400.0, **tags: str) -> SessionContext
```

Create a session context manager.

**Returns:** `SessionContext` — Context manager that yields a Session

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `timeout` | `float` | `86400.0` | End-to-end session-creation timeout in seconds. |
| `tags` | `str` |  |  |

### Client.sample

```python
Client.sample(
    prompts: str | list[str] | None = None,
    *,
    base_model: str,
    num_samples: int = 1,
    max_tokens: int = 256,
    temperature: float = 1.0,
    top_p: float = 1.0,
    top_k: int = -1,
    stop: list[str] | None = None,
    seed: int | None = None,
    return_prompt_logprobs: bool = False,
    logprobs: int | None = None,
    images: list[bytes] | list[list[bytes]] | None = None,
    prompt_token_ids: list[int] | list[list[int]] | None = None,
    model_input: list[dict] | list[list[dict]] | None = None,
    tokenizer: Any | None = None,
    metrics_type: str = '',
    timeout: float | None = None,
) -> list[Sample]
```

Sample from a base model (no session required).

**Returns:** `list[Sample]` — Flat `list[Sample]` — all samples across all prompts. For a single prompt with `num_samples=1` (the default), this is a list with one element: `result[0].text`.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `prompts` | `str \| list[str] \| None` | `None` | Single prompt string or list of prompts. Mutually exclusive with `prompt_token_ids`. |
| `base_model` | `str` |  | Base model name (e.g. `"Qwen/Qwen3.6-35B-A3B-FP8"`). |
| `num_samples` | `int` | `1` | Number of independent samples per prompt. |
| `max_tokens` | `int` | `256` | Maximum tokens to generate per sample. |
| `temperature` | `float` | `1.0` | Sampling temperature. |
| `top_p` | `float` | `1.0` | Nucleus sampling threshold. |
| `top_k` | `int` | `-1` | Top-k sampling (-1 = disabled). |
| `stop` | `list[str] \| None` | `None` | Stop sequences. |
| `seed` | `int \| None` | `None` | Random seed (varied per sample automatically). |
| `return_prompt_logprobs` | `bool` | `False` | Whether to return prompt token logprobs. |
| `logprobs` | `int \| None` | `None` | If set to `K > 0`, request the top-K alternative logprobs at each position. Off by default — enabling it roughly halves server throughput. |
| `images` | `list[bytes] \| list[list[bytes]] \| None` | `None` | Optional raw image bytes for multimodal sampling. See `sample` for the per-prompt vs. broadcast semantics. |
| `prompt_token_ids` | `list[int] \| list[list[int]] \| None` | `None` | Pre-tokenized prompt(s); mutually exclusive with `prompts`. See `sample` for details. |
| `model_input` | `list[dict] \| list[list[dict]] \| None` | `None` | Training-style chunk list(s); mutually exclusive with `prompts` / `prompt_token_ids` / `images`. See `sample` for details. |
| `tokenizer` | `Any \| None` | `None` | Optional tokenizer name or already-loaded tokenizer. Defaults to `base_model` after applying River model-alias resolution. |
| `metrics_type` | `str` | `''` | Opaque server-interpreted token enabling extra scalar metrics on the response. Unrecognized values are silently ignored; when recognized, per-result metrics appear on `Sample.metrics`. |
| `timeout` | `float \| None` | `None` | Timeout in seconds. |

### Client.health_check

```python
Client.health_check() -> bool
```

Check API health.

**Returns:** `bool` — True if healthy

### Client.get_capabilities

```python
Client.get_capabilities() -> list[str]
```

Get supported models.

**Returns:** `list[str]` — List of supported model names

### Client.get_streaming_replica

```python
Client.get_streaming_replica(
    model: str,
    *,
    timeout: float | None = None,
) -> PromotedStreamingReplica | None
```

Return promoted streaming metadata for `model` when available.

This reads server-owned routing metadata from the control plane.
Missing metadata returns `None`; other HTTP/auth failures raise a
River client exception.

| Parameter | Type | Default |
| --- | --- | --- |
| `model` | `str` |  |
| `timeout` | `float \| None` | `None` |

### Client.promote_streaming_replica

```python
Client.promote_streaming_replica(
    checkpoint: str | Checkpoint,
    model: str,
    *,
    timeout: float | None = None,
) -> PromotedStreamingReplica
```

Request beta promotion of a checkpoint to a streaming model alias.

The request is asynchronous: the returned metadata is usually
`status="provisioning"`. Poll `get_streaming_replica` until the
status is `"ready"` or `"degraded"` before using
`chat_complete_stream`.

| Parameter | Type | Default |
| --- | --- | --- |
| `checkpoint` | `str \| Checkpoint` |  |
| `model` | `str` |  |
| `timeout` | `float \| None` | `None` |

### Client.chat_complete_stream

```python
Client.chat_complete_stream(
    messages: list[dict],
    *,
    model: str,
    timeout: float | None = None,
    on_not_ready: str = 'raise',
    **kwargs,
) -> Iterator[dict[str, Any]]
```

Stream OpenAI-compatible chat chunks from a promoted replica.

`timeout` is the socket read timeout for each blocking read, not an
end-to-end generation deadline. Let the iterator finish, or call
`close()` on it when breaking early, so the HTTP connection is closed.

**Returns:** `Iterator[dict[str, Any]]` — Iterator of decoded OpenAI streaming chunk dictionaries.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `messages` | `list[dict]` |  | OpenAI-format messages list. |
| `model` | `str` |  | Product-facing promoted model alias. |
| `timeout` | `float \| None` | `None` | Per-read HTTP timeout. Defaults to 60 seconds. |
| `on_not_ready` | `str` | `'raise'` | `"raise"` (default) or `"blocking"`. The blocking fallback returns one stream-shaped chunk converted from the existing blocking control-plane chat path when promoted metadata is missing, unreachable, or not ready. |
| `kwargs` |  |  |  |

### Client.chat_complete

```python
Client.chat_complete(
    messages: list[dict],
    *,
    base_model: str,
    timeout: float | None = None,
    **kwargs,
) -> ChatCompleteResult | Iterator[dict[str, Any]]
```

Chat completion from a base model (no LoRA).

Builds an OpenAI-format request body and sends it through the
gRPC `ChatCompleteFromBase` RPC.

**Returns:** `ChatCompleteResult | Iterator[dict[str, Any]]` — ChatCompleteResult with response_json and status_code.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `messages` | `list[dict]` |  | OpenAI-format messages list. |
| `base_model` | `str` |  | Base model name for routing. |
| `timeout` | `float \| None` | `None` | Timeout in seconds. |
| `kwargs` |  |  |  |

### Client.chat_complete_from_checkpoint

```python
Client.chat_complete_from_checkpoint(
    messages: list[dict],
    *,
    checkpoint_path: str,
    base_model: str = '',
    timeout: float | None = None,
    **kwargs,
) -> ChatCompleteResult
```

Chat completion from a saved checkpoint.

**Returns:** `ChatCompleteResult` — ChatCompleteResult with response_json and status_code.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `messages` | `list[dict]` |  | OpenAI-format messages list. |
| `checkpoint_path` | `str` |  | `river://` checkpoint path. |
| `base_model` | `str` | `''` | Base model name (optional; resolved from DB if empty). |
| `timeout` | `float \| None` | `None` | Timeout in seconds. |
| `kwargs` |  |  |  |

### Client.chat_complete_from_training

```python
Client.chat_complete_from_training(
    messages: list[dict],
    *,
    model_id: str,
    timeout: float | None = None,
    **kwargs,
) -> ChatCompleteResult
```

Chat completion from in-memory training weights.

**Returns:** `ChatCompleteResult` — ChatCompleteResult with response_json and status_code.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `messages` | `list[dict]` |  | OpenAI-format messages list. |
| `model_id` | `str` |  | Training model ID (e.g. `session_id:model:seq`). |
| `timeout` | `float \| None` | `None` | Timeout in seconds. |
| `kwargs` |  |  |  |

### Client.close

```python
Client.close() -> None
```

Close the client connection.

---

## Session

A training session with GPU allocation.

Entered through [`Client.session`](#clientsession); it owns the models you train.

**Methods:** [`attest_training_data`](#sessionattest_training_data), [`create_model`](#sessioncreate_model), [`sample`](#sessionsample), [`submit_sample`](#sessionsubmit_sample)

### Session.attest_training_data

```python
Session.attest_training_data(
    artifacts: list[TrainingDataArtifact],
    timeout: float = 86400.0,
) -> TrainingDataAttestation
```

Ask the API to hash source artifacts and retain their manifest.

The source bytes are discarded by the API after hashing. Pass the
result to `create_model` to make the server fail closed before
every forward/backward request if that manifest disappears or no longer
belongs to the model's session.

| Parameter | Type | Default |
| --- | --- | --- |
| `artifacts` | `list[TrainingDataArtifact]` |  |
| `timeout` | `float` | `86400.0` |

### Session.create_model

```python
Session.create_model(
    base_model: str,
    lora: LoraConfig | None = None,
    tokenizer: str | Any | None = None,
    checkpoint: str | Checkpoint | None = None,
    timeout: float = 86400.0,
    training_data_attestation: TrainingDataAttestation | str | None = None,
) -> Model
```

Create a new model for training.

**Returns:** `Model` — Model object for training

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `base_model` | `str` |  | Base model name (e.g., "Qwen/Qwen3.6-35B-A3B-FP8") |
| `lora` | `LoraConfig \| None` | `None` | Optional LoRA configuration |
| `tokenizer` | `str \| Any \| None` | `None` | Tokenizer name (defaults to base_model) or an already-loaded tokenizer object |
| `checkpoint` | `str \| Checkpoint \| None` | `None` | Optional checkpoint to load after creation. Can be a `river://` path string or a `Checkpoint` object. If a `Checkpoint` is passed, its step is restored and `load_optimizer` is set automatically based on checkpoint type. |
| `timeout` | `float` | `86400.0` | Timeout in seconds |
| `training_data_attestation` | `TrainingDataAttestation \| str \| None` | `None` | Optional server-verified source-artifact manifest. When supplied, the API rejects forward/backward requests if its manifest is missing or no longer belongs to this session. |

### Session.sample

```python
Session.sample(
    prompts: str | list[str] | None = None,
    *,
    base_model: str,
    checkpoint: str | Checkpoint | None = None,
    num_samples: int = 1,
    max_tokens: int = 256,
    temperature: float = 1.0,
    top_p: float = 1.0,
    top_k: int = -1,
    stop: list[str] | None = None,
    seed: int | None = None,
    return_prompt_logprobs: bool = False,
    logprobs: int | None = None,
    return_expert_routing: bool = False,
    images: list[bytes] | list[list[bytes]] | None = None,
    prompt_token_ids: list[int] | list[list[int]] | None = None,
    model_input: list[dict] | list[list[dict]] | None = None,
    tokenizer: Any | None = None,
    metrics_type: str = '',
    timeout: float = 86400.0,
) -> list[list[Sample]]
```

Sample from a base model or checkpoint.

When `checkpoint` is provided, the server loads the LoRA from the
saved checkpoint, generates text, then unloads.

**Returns:** `list[list[Sample]]` — `list[list[Sample]]` — outer list is per-prompt, inner list is per-sample.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `prompts` | `str \| list[str] \| None` | `None` | Single prompt string or list of prompts. Mutually exclusive with `prompt_token_ids`. |
| `base_model` | `str` |  | Base model name (e.g. `"Qwen/Qwen3.6-35B-A3B-FP8"`). |
| `checkpoint` | `str \| Checkpoint \| None` | `None` | Optional `river://` path or `Checkpoint` object. If provided, samples from that checkpoint's LoRA weights. |
| `num_samples` | `int` | `1` | Number of independent samples per prompt. |
| `max_tokens` | `int` | `256` | Maximum tokens to generate per sample. |
| `temperature` | `float` | `1.0` | Sampling temperature. |
| `top_p` | `float` | `1.0` | Nucleus sampling threshold. |
| `top_k` | `int` | `-1` | Top-k sampling (-1 = disabled). |
| `stop` | `list[str] \| None` | `None` | Stop sequences. |
| `seed` | `int \| None` | `None` | Random seed (varied per sample automatically). |
| `return_prompt_logprobs` | `bool` | `False` | Whether to return prompt token logprobs. |
| `logprobs` | `int \| None` | `None` | If set to `K > 0`, request the top-K alternative logprobs at each position. Off by default — enabling it roughly halves server throughput. |
| `return_expert_routing` | `bool` | `False` |  |
| `images` | `list[bytes] \| list[list[bytes]] \| None` | `None` | Optional raw image bytes for multimodal sampling. See `sample` for the per-prompt vs. broadcast semantics. |
| `prompt_token_ids` | `list[int] \| list[list[int]] \| None` | `None` | Pre-tokenized prompt(s); mutually exclusive with `prompts`. See `sample` for details. |
| `model_input` | `list[dict] \| list[list[dict]] \| None` | `None` | Training-style chunk list(s); mutually exclusive with `prompts` / `prompt_token_ids` / `images`. See `sample` for details. |
| `tokenizer` | `Any \| None` | `None` | Optional already-loaded tokenizer. Passing this avoids repeated Hugging Face cache/network resolution in tight loops. |
| `metrics_type` | `str` | `''` | Opaque server-interpreted token enabling extra scalar metrics on the response. Unrecognized values are silently ignored; when recognized, per-result metrics appear on `Sample.metrics`. |
| `timeout` | `float` | `86400.0` | Timeout in seconds. |

### Session.submit_sample

```python
Session.submit_sample(
    prompts: str | list[str] | None = None,
    *,
    base_model: str,
    checkpoint: str | Checkpoint | None = None,
    num_samples: int = 1,
    max_tokens: int = 256,
    temperature: float = 1.0,
    top_p: float = 1.0,
    top_k: int = -1,
    stop: list[str] | None = None,
    seed: int | None = None,
    return_prompt_logprobs: bool = False,
    logprobs: int | None = None,
    return_expert_routing: bool = False,
    images: list[bytes] | list[list[bytes]] | None = None,
    prompt_token_ids: list[int] | list[list[int]] | None = None,
    model_input: list[dict] | list[list[dict]] | None = None,
    tokenizer: Any | None = None,
    metrics_type: str = '',
    timeout: float = 86400.0,
) -> PendingSample
```

Submit sampling from a base model or checkpoint without waiting.

See `sample` for the full kwarg reference.

| Parameter | Type | Default |
| --- | --- | --- |
| `prompts` | `str \| list[str] \| None` | `None` |
| `base_model` | `str` |  |
| `checkpoint` | `str \| Checkpoint \| None` | `None` |
| `num_samples` | `int` | `1` |
| `max_tokens` | `int` | `256` |
| `temperature` | `float` | `1.0` |
| `top_p` | `float` | `1.0` |
| `top_k` | `int` | `-1` |
| `stop` | `list[str] \| None` | `None` |
| `seed` | `int \| None` | `None` |
| `return_prompt_logprobs` | `bool` | `False` |
| `logprobs` | `int \| None` | `None` |
| `return_expert_routing` | `bool` | `False` |
| `images` | `list[bytes] \| list[list[bytes]] \| None` | `None` |
| `prompt_token_ids` | `list[int] \| list[list[int]] \| None` | `None` |
| `model_input` | `list[dict] \| list[list[dict]] \| None` | `None` |
| `tokenizer` | `Any \| None` | `None` |
| `metrics_type` | `str` | `''` |
| `timeout` | `float` | `86400.0` |

### Session.session_id

```python
Session.session_id: str
```

---

## SessionContext

Context manager for Session with auto-heartbeat.

The context manager returned by [`Client.session`](#clientsession).

---

## Model

A training model with mutable in-memory weights.

Created by [`Session.create_model`](#sessioncreate_model).

**Methods:** [`forward`](#modelforward), [`forward_backward`](#modelforward_backward), [`optim_step`](#modeloptim_step), [`train_step`](#modeltrain_step), [`submit_forward_backward`](#modelsubmit_forward_backward), [`submit_optim_step`](#modelsubmit_optim_step), [`submit_train_step`](#modelsubmit_train_step), [`sample`](#modelsample), [`submit_sample`](#modelsubmit_sample), [`chat_complete`](#modelchat_complete), [`save_weights`](#modelsave_weights), [`promote_to_streaming`](#modelpromote_to_streaming), [`load_weights`](#modelload_weights)

### Model.forward

```python
Model.forward(
    data: list[dict],
    loss_fn: str = 'cross_entropy',
    timeout: float = 86400.0,
    **loss_config: float,
) -> ForwardResult
```

Forward pass only (compute loss, no gradients).

**Returns:** `ForwardResult` — ForwardResult with metrics

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `data` | `list[dict]` |  | List of training samples, each with "input_ids" and "labels" |
| `loss_fn` | `str` | `'cross_entropy'` | Loss function name |
| `timeout` | `float` | `86400.0` | Timeout in seconds |
| `loss_config` | `float` |  |  |

### Model.forward_backward

```python
Model.forward_backward(
    data: list[dict],
    loss_fn: str = 'cross_entropy',
    timeout: float = 86400.0,
    return_logprobs: bool = False,
    zero_out: bool = True,
    compute_expert_flip_metric: bool = False,
    force_routing_replay: bool = False,
    **loss_config: float,
) -> ForwardResult
```

Forward + backward pass (compute gradients).

**Returns:** `ForwardResult` — ForwardResult with metrics and logprobs when returned by the worker.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `data` | `list[dict]` |  | List of training samples, each with "input_ids" and "labels" |
| `loss_fn` | `str` | `'cross_entropy'` | Loss function name |
| `timeout` | `float` | `86400.0` | Timeout in seconds |
| `return_logprobs` | `bool` | `False` | Deprecated no-op. Training losses return per-token logprobs when the worker includes them in the result; this argument is accepted for older callers but is not sent to the server as a loss configuration key. |
| `zero_out` | `bool` | `True` | When `gradient_accumulation` is enabled, clear existing gradients before this call. Use `True` for the first micro-batch and `False` for subsequent micro-batches. |
| `compute_expert_flip_metric` | `bool` | `False` | When True, compares sampled expert routing against the training-time routing per token and MoE layer, then emits one scalar into `ForwardResult.metrics`: * `expert_flip/per_token_expert_rate` ∈ [0, 1] — the fraction of individual top-k expert slots that differ, (top_k − \|intersection\|) / top_k over (token, layer). Independent of `force_routing_replay`. Requires per-datum routing keys from `Sample.routing_datum_keys(required=True)`. |
| `force_routing_replay` | `bool` | `False` | When true, replay the sampled expert selection while recomputing routing weights at those experts with the trainer's live gate. Every datum in `data` must include the keys returned by `Sample.routing_datum_keys(required=True)`. |
| `loss_config` | `float` |  |  |

### Model.optim_step

```python
Model.optim_step(
    lr: float,
    beta1: float = 0.9,
    beta2: float = 0.999,
    eps: float = 1e-08,
    weight_decay: float = 0.0,
    grad_clip_norm: float | None = None,
    timeout: float = 86400.0,
) -> OptimStepResult
```

Apply gradients with Adam optimizer.

**Returns:** `OptimStepResult` — OptimStepResult with metrics (step, lr, grad_norm, grad_norm_finite)

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `lr` | `float` |  | Learning rate |
| `beta1` | `float` | `0.9` | Adam beta1 |
| `beta2` | `float` | `0.999` | Adam beta2 |
| `eps` | `float` | `1e-08` | Adam epsilon |
| `weight_decay` | `float` | `0.0` | Weight decay |
| `grad_clip_norm` | `float \| None` | `None` | Gradient clipping norm (None to disable) |
| `timeout` | `float` | `86400.0` | Timeout in seconds |

### Model.train_step

```python
Model.train_step(
    data: list[dict],
    lr: float,
    *,
    loss_fn: str = 'cross_entropy',
    beta1: float = 0.9,
    beta2: float = 0.999,
    eps: float = 1e-08,
    weight_decay: float = 0.0,
    grad_clip_norm: float | None = None,
    compute_expert_flip_metric: bool = False,
    force_routing_replay: bool = False,
    timeout: float = 86400.0,
    **loss_config: float,
) -> tuple[ForwardResult, OptimStepResult]
```

Complete training step: forward+backward plus optimizer update.

Submits forward+backward and the optimizer step back-to-back —
the server runs them in order as one pipelined unit, without a
client round trip in between — then waits for both results. See
`forward_backward` and `optim_step` for the full
parameter reference.

The error path differs from calling `forward_backward()` then
`optim_step()`: the optimizer step is already submitted when
forward-backward resolves, so if forward-backward fails, this
call raises its error while the optimizer step still runs
server-side and `Model.step` has already advanced. Callers
that need to inspect both outcomes should use
`submit_train_step`.

A train step is a complete step: gradients are always cleared
first. For micro-batch gradient accumulation, use
`submit_forward_backward(zero_out=...)` and
`submit_optim_step` directly.

**Returns:** `tuple[ForwardResult, OptimStepResult]` — Tuple of (ForwardResult, OptimStepResult).

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `data` | `list[dict]` |  | List of training samples, each with "input_ids" and "labels" |
| `lr` | `float` |  | Learning rate |
| `loss_fn` | `str` | `'cross_entropy'` |  |
| `beta1` | `float` | `0.9` |  |
| `beta2` | `float` | `0.999` |  |
| `eps` | `float` | `1e-08` |  |
| `weight_decay` | `float` | `0.0` |  |
| `grad_clip_norm` | `float \| None` | `None` |  |
| `compute_expert_flip_metric` | `bool` | `False` |  |
| `force_routing_replay` | `bool` | `False` |  |
| `timeout` | `float` | `86400.0` |  |
| `loss_config` | `float` |  |  |

### Model.submit_forward_backward

```python
Model.submit_forward_backward(
    data: list[dict],
    loss_fn: str = 'cross_entropy',
    timeout: float = 86400.0,
    return_logprobs: bool = False,
    zero_out: bool = True,
    compute_expert_flip_metric: bool = False,
    force_routing_replay: bool = False,
    **loss_config: float,
) -> PendingOp
```

Submit forward+backward without blocking. Returns a PendingOp.

Call `pending.result()` later to get the ForwardResult.
This enables pipelining: submit step N+1 while step N is still running.
See `forward_backward` for the full kwarg reference.

When submitting multiple micro-batches before `submit_optim_step`, use
`zero_out=True` for the first one and `False` for later submissions.

`timeout` bounds the submit RPC and, separately, the wait inside
`pending.result()`.

| Parameter | Type | Default |
| --- | --- | --- |
| `data` | `list[dict]` |  |
| `loss_fn` | `str` | `'cross_entropy'` |
| `timeout` | `float` | `86400.0` |
| `return_logprobs` | `bool` | `False` |
| `zero_out` | `bool` | `True` |
| `compute_expert_flip_metric` | `bool` | `False` |
| `force_routing_replay` | `bool` | `False` |
| `loss_config` | `float` |  |

### Model.submit_optim_step

```python
Model.submit_optim_step(
    lr: float,
    beta1: float = 0.9,
    beta2: float = 0.999,
    eps: float = 1e-08,
    weight_decay: float = 0.0,
    grad_clip_norm: float | None = None,
    timeout: float = 86400.0,
) -> PendingOp
```

Submit optimizer step without blocking. Returns a PendingOp.

Call `pending.result()` later to get the OptimStepResult.
`Model.step` advances at submit time, even if the operation
later fails.

| Parameter | Type | Default |
| --- | --- | --- |
| `lr` | `float` |  |
| `beta1` | `float` | `0.9` |
| `beta2` | `float` | `0.999` |
| `eps` | `float` | `1e-08` |
| `weight_decay` | `float` | `0.0` |
| `grad_clip_norm` | `float \| None` | `None` |
| `timeout` | `float` | `86400.0` |

### Model.submit_train_step

```python
Model.submit_train_step(
    data: list[dict],
    lr: float,
    *,
    loss_fn: str = 'cross_entropy',
    beta1: float = 0.9,
    beta2: float = 0.999,
    eps: float = 1e-08,
    weight_decay: float = 0.0,
    grad_clip_norm: float | None = None,
    compute_expert_flip_metric: bool = False,
    force_routing_replay: bool = False,
    timeout: float = 86400.0,
    **loss_config: float,
) -> tuple[PendingOp, PendingOp]
```

Submit a complete training step without blocking.

Fires forward+backward and the optimizer step back-to-back; the
server runs them in submission order per model. Returns the two
PendingOps as `(forward_backward, optim_step)`.

Because both are submitted up front, a failed forward-backward
does not cancel the already-submitted optimizer step.
`Model.step` advances at submit time. See
`train_step` for the full kwarg reference.

| Parameter | Type | Default |
| --- | --- | --- |
| `data` | `list[dict]` |  |
| `lr` | `float` |  |
| `loss_fn` | `str` | `'cross_entropy'` |
| `beta1` | `float` | `0.9` |
| `beta2` | `float` | `0.999` |
| `eps` | `float` | `1e-08` |
| `weight_decay` | `float` | `0.0` |
| `grad_clip_norm` | `float \| None` | `None` |
| `compute_expert_flip_metric` | `bool` | `False` |
| `force_routing_replay` | `bool` | `False` |
| `timeout` | `float` | `86400.0` |
| `loss_config` | `float` |  |

### Model.sample

```python
Model.sample(
    prompts: str | list[str] | None = None,
    *,
    num_samples: int = 1,
    max_tokens: int = 256,
    temperature: float = 1.0,
    top_p: float = 1.0,
    top_k: int = -1,
    stop: list[str] | None = None,
    seed: int | None = None,
    return_prompt_logprobs: bool = False,
    logprobs: int | None = None,
    images: list[bytes] | list[list[bytes]] | None = None,
    return_expert_routing: bool = False,
    prompt_token_ids: list[int] | list[list[int]] | None = None,
    model_input: list[dict] | list[list[dict]] | None = None,
    metrics_type: str = '',
    timeout: float = 86400.0,
    poll_interval: float = 1.0,
) -> list[list[Sample]]
```

Sample from the current in-memory training weights.

Generates text using the model's current weights, without
needing to save a checkpoint first. Per-token logprobs are
always returned.

**Returns:** `list[list[Sample]]` — `list[list[Sample]]` — outer list is per-prompt, inner list is per-sample. Each `Sample` has `.tokens`, `.text`, `.logprobs`, and `.stop_reason`.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `prompts` | `str \| list[str] \| None` | `None` | Single prompt string or list of prompts. Mutually exclusive with `prompt_token_ids`. |
| `num_samples` | `int` | `1` | Number of independent samples per prompt. |
| `max_tokens` | `int` | `256` | Maximum tokens to generate per sample. |
| `temperature` | `float` | `1.0` | Sampling temperature. |
| `top_p` | `float` | `1.0` | Nucleus sampling threshold. |
| `top_k` | `int` | `-1` | Top-k sampling (-1 = disabled). |
| `stop` | `list[str] \| None` | `None` | Stop sequences. |
| `seed` | `int \| None` | `None` | Random seed (varied per sample automatically). |
| `return_prompt_logprobs` | `bool` | `False` | Whether to return prompt token logprobs. |
| `logprobs` | `int \| None` | `None` | If set to `K > 0`, request the top-K alternative logprobs at each output position (and, when `return_prompt_logprobs=True`, at each prompt position). Off by default — enabling it roughly halves server throughput for small serialization gain, so it's opt-in. |
| `images` | `list[bytes] \| list[list[bytes]] \| None` | `None` | Optional raw image bytes (PNG / JPEG) for multimodal sampling. Accepts `list[bytes]` (broadcast the same image set to every prompt) or `list[list[bytes]]` (per-prompt explicit). Bytes are sent to the inference backend as image data. Most ergonomic source: `**Qwen35VLRenderer.build_sample_prompt(messages).to_kwargs()`, which emits `{"prompt", "images"}`. The image format is inferred from the bytes' magic header, so no separate format hint is sent over the wire. |
| `return_expert_routing` | `bool` | `False` | Capture per-token MoE expert routing during this sampling call. When available, each `Sample` exposes an `.expert_routing` object that can be round-tripped into training data with `sample.routing_datum_keys(required=True)` before calling `forward_backward(force_routing_replay=...)` or `forward_backward(compute_expert_flip_metric=True)`. |
| `prompt_token_ids` | `list[int] \| list[list[int]] \| None` | `None` | Pre-tokenized prompt(s) — a flat `list[int]` (one prompt) or `list[list[int]]` (one entry per prompt). Mutually exclusive with `prompts`. Ids are forwarded verbatim for sampling, bypassing server-side tokenization, so the sampled continuation is conditioned on exactly these ids (no training/sampling tokenization skew). Ids must be valid for this model's vocabulary. May be combined with `images` using the same single-placeholder convention as text prompts: one un-expanded `<\|image_pad\|>`-style token id per image, in `images` order — the placeholder count must match the image count exactly (a surplus of images is silently dropped by the backend otherwise). |
| `model_input` | `list[dict] \| list[list[dict]] \| None` | `None` | Training-style chunk list(s) — the same `[{"type": "text", "tokens": [...]}, {"type": "image", "data": bytes, ...}, ...]` shape `forward_backward` accepts, for one prompt (`list[dict]`) or a batch (`list[list[dict]]`). Lowered client-side to `prompt_token_ids` + `images` (each image chunk becomes one un-expanded placeholder token). Mutually exclusive with `prompts` / `prompt_token_ids` / `images`. `expected_tokens` and `format` on image chunks are accepted and ignored; to validate the backend's image expansion against `expected_tokens`, pass `return_prompt_logprobs=True` and count placeholder ids in the echoed `Sample.prompt_token_ids`. |
| `metrics_type` | `str` | `''` | Opaque server-interpreted token enabling extra scalar metrics on the response. Unrecognized values are silently ignored; when recognized, per-result metrics appear on `Sample.metrics`. |
| `timeout` | `float` | `86400.0` | Timeout in seconds for the entire operation (includes server-side wait for LoRA slot availability). |
| `poll_interval` | `float` | `1.0` | Seconds between completion polls once the request is in flight. The default (1s) suits ad-hoc sampling; tight RL loops that immediately consume the results can lower it to shave the post-completion notice lag off every step. |

### Model.submit_sample

```python
Model.submit_sample(
    prompts: str | list[str] | None = None,
    *,
    num_samples: int = 1,
    max_tokens: int = 256,
    temperature: float = 1.0,
    top_p: float = 1.0,
    top_k: int = -1,
    stop: list[str] | None = None,
    seed: int | None = None,
    return_prompt_logprobs: bool = False,
    logprobs: int | None = None,
    images: list[bytes] | list[list[bytes]] | None = None,
    return_expert_routing: bool = False,
    prompt_token_ids: list[int] | list[list[int]] | None = None,
    model_input: list[dict] | list[list[dict]] | None = None,
    metrics_type: str = '',
    timeout: float = 86400.0,
    poll_interval: float = 1.0,
) -> PendingSample
```

Submit sampling from current training weights without waiting.

See `sample` for the full kwarg reference.

| Parameter | Type | Default |
| --- | --- | --- |
| `prompts` | `str \| list[str] \| None` | `None` |
| `num_samples` | `int` | `1` |
| `max_tokens` | `int` | `256` |
| `temperature` | `float` | `1.0` |
| `top_p` | `float` | `1.0` |
| `top_k` | `int` | `-1` |
| `stop` | `list[str] \| None` | `None` |
| `seed` | `int \| None` | `None` |
| `return_prompt_logprobs` | `bool` | `False` |
| `logprobs` | `int \| None` | `None` |
| `images` | `list[bytes] \| list[list[bytes]] \| None` | `None` |
| `return_expert_routing` | `bool` | `False` |
| `prompt_token_ids` | `list[int] \| list[list[int]] \| None` | `None` |
| `model_input` | `list[dict] \| list[list[dict]] \| None` | `None` |
| `metrics_type` | `str` | `''` |
| `timeout` | `float` | `86400.0` |
| `poll_interval` | `float` | `1.0` |

### Model.chat_complete

```python
Model.chat_complete(
    messages: list[dict],
    *,
    timeout: float | None = None,
    **kwargs,
) -> ChatCompleteResult
```

OpenAI-compatible chat completion from current training weights.

Like `model.sample()` but using the OpenAI chat-completions format
instead of raw prompts.

**Returns:** `ChatCompleteResult` — ChatCompleteResult with `response_json` (full OpenAI-format JSON string) and `status_code`.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `messages` | `list[dict]` |  | OpenAI-format messages list (e.g. `[{"role": "user", "content": "Hello"}]`). |
| `timeout` | `float \| None` | `None` | Timeout in seconds. |
| `kwargs` |  |  |  |

### Model.save_weights

```python
Model.save_weights(
    name: str,
    mode: str = 'training',
    timeout: float = 86400.0,
    ttl: datetime.timedelta | None = None,
) -> Checkpoint
```

Save a checkpoint of the current model weights.

**Returns:** `Checkpoint` — Checkpoint object with `river://` path.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `name` | `str` |  | Checkpoint name (e.g. `"final"` or `"step_000100"`). |
| `mode` | `str` | `'training'` | `"training"` saves optimizer state (for training continuation), `"inference"` saves PEFT format only (for sampling/inference). |
| `timeout` | `float` | `86400.0` | Timeout in seconds. |
| `ttl` | `datetime.timedelta \| None` | `None` | Lifetime before the checkpoint is reaped. Applies to explicit user-saved checkpoints in both modes; when omitted, the server default is 1 year. 1 year is also the maximum — the server rejects a longer `ttl`. |

### Model.promote_to_streaming

```python
Model.promote_to_streaming(
    model: str,
    checkpoint: str | Checkpoint | None = None,
    checkpoint_name: str | None = None,
    timeout: float | None = 86400.0,
) -> PromotedStreamingReplica
```

Promote this model's current or saved checkpoint to a stream alias.

When `checkpoint` is omitted, this saves the current weights with
`mode="inference"` using `checkpoint_name` or a generated name,
then asks the control plane to promote that checkpoint asynchronously.
The `timeout` is applied separately to the save and promotion calls,
not as one end-to-end deadline. Omitted `checkpoint_name` values
create a new server-side inference checkpoint for each call.

| Parameter | Type | Default |
| --- | --- | --- |
| `model` | `str` |  |
| `checkpoint` | `str \| Checkpoint \| None` | `None` |
| `checkpoint_name` | `str \| None` | `None` |
| `timeout` | `float \| None` | `86400.0` |

### Model.load_weights

```python
Model.load_weights(
    checkpoint: str | Checkpoint,
    load_optimizer: bool = True,
    timeout: float = 86400.0,
) -> None
```

Load weights from a checkpoint.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `checkpoint` | `str \| Checkpoint` |  | A `river://` path string or a `Checkpoint` object. If a `Checkpoint` is passed, its step is restored on the model. |
| `load_optimizer` | `bool` | `True` | Whether to load optimizer state. |
| `timeout` | `float` | `86400.0` | Timeout in seconds. |

### Model.model_id

```python
Model.model_id: str
```

### Model.training_run_id

```python
Model.training_run_id: str
```

### Model.step

```python
Model.step: int
```

Current training step.

Advances when an optimizer step is submitted (so pipelined
submissions observe a consistent value), not when it resolves —
a failed or timed-out optimizer step leaves it advanced.

---

## Data types

Values returned by the client and the configuration objects you pass to it.

### LoraConfig

LoRA adapter configuration.

| Field | Type | Default |
| --- | --- | --- |
| `rank` | `int` | `16` |
| `train_attn` | `bool` | `True` |
| `train_mlp` | `bool` | `True` |
| `train_unembed` | `bool` | `False` |
| `seed` | `int \| None` | `None` |

### Sample

A generated sample.

`tokens` contains generated token IDs when the response includes them;
older responses may fall back to retokenizing `text` client-side.

`prompt_token_ids` / `top_logprobs` / `prompt_top_logprobs` are
`None` when the server did not provide them or the feature was not
requested.

`expert_routing` is populated when `return_expert_routing=True`
was passed on the request.

`metrics` carries per-result scalar metrics; empty unless the
originating request passed a recognized `metrics_type` token.

| Field | Type | Default |
| --- | --- | --- |
| `tokens` | `list[int]` |  |
| `text` | `str` |  |
| `logprobs` | `list[float]` |  |
| `stop_reason` | `str` |  |
| `model_step` | `int` |  |
| `prompt_logprobs` | `list[float] \| None` | `None` |
| `request_id` | `str` | `''` |
| `prompt_token_ids` | `list[int] \| None` | `None` |
| `top_logprobs` | `list[list[TopLogprob]] \| None` | `None` |
| `prompt_top_logprobs` | `list[list[TopLogprob]] \| None` | `None` |
| `expert_routing` | `ExpertRouting \| None` | `None` |
| `metrics` | `dict[str, float]` | `{}` |

#### Sample.routing_datum_keys

```python
Sample.routing_datum_keys(*, required: bool = False) -> dict[str, bytes | str]
```

Return the per-datum keys to splat into a `forward_backward`
datum when enabling `force_routing_replay` or
`compute_expert_flip_metric`.

Returns an empty dict when this sample carries no routing capture and
`required` is false. Raises `ValueError` when `required` is true
and no routing handle is available.

Example:

```python
data = []
for sample in samples:
    datum = {
        "input_ids": ...,
        "advantages": ...,
        **sample.routing_datum_keys(required=True),
    }
    data.append(datum)
```

| Parameter | Type | Default |
| --- | --- | --- |
| `required` | `bool` | `False` |

### ChatCompleteResult

Result of a chat completion request.

| Field | Type |
| --- | --- |
| `response_json` | `str` |
| `status_code` | `int` |

### ForwardResult

Result of forward or forward_backward pass.

| Field | Type | Default |
| --- | --- | --- |
| `metrics` | `dict[str, float]` |  |
| `logprobs` | `list \| None` | `None` |

### OptimStepResult

Result of an optimizer step.

| Field | Type |
| --- | --- |
| `metrics` | `dict[str, float]` |

### Checkpoint

A saved model checkpoint.

| Field | Type |
| --- | --- |
| `path` | `str` |
| `step` | `int` |
| `checkpoint_type` | `str` |

### TopLogprob

One top-K candidate token at a single position.

| Field | Type | Default |
| --- | --- | --- |
| `logprob` | `float` |  |
| `token_id` | `int` |  |
| `token` | `str` | `''` |

### PendingOp

A submitted but not-yet-resolved async operation.

Returned by `Model.submit_forward_backward()` and
`Model.submit_optim_step()`. Call `.result()` to block until complete.

| Field | Type |
| --- | --- |
| `request_id` | `str` |

#### PendingOp.result

```python
PendingOp.result() -> ForwardResult | OptimStepResult
```

Block until the operation completes and return the result.

### PendingSample

A submitted but not-yet-resolved sampling operation.

| Field | Type |
| --- | --- |
| `request_id` | `str` |

#### PendingSample.result

```python
PendingSample.result() -> list[list[Sample]]
```

Block until sampling completes and return grouped samples.

### ExpertRouting

Captured MoE expert routing for one sample.

Populated when the caller passed `return_expert_routing=True` on
the sample request and routing capture was available.

To enable router replay / flip-rate metrics, splat the canonical
per-datum keys via `Sample.routing_datum_keys()` rather than
building them by hand:

```python
datum = {..., **sample.routing_datum_keys(required=True)}
```
Current servers return routing captures as an opaque `handle`; the
splat yields `expert_routing_handle`. `topk_ids` is retained for
inspection and proto round-trip compatibility only; replay always
recomputes the routing weights in the trainer.

The shape header fields (`num_decoder_layers` / `top_k` /
`layer_indices`) are *informational only* and let the user inspect
capture metadata.

Byte layout (when ids are present): `[num_tokens, num_decoder_layers,
top_k]` row-major int16 little-endian. `num_tokens = seqlen - 1`
because routing capture omits the trailing position.

| Field | Type | Default |
| --- | --- | --- |
| `topk_ids` | `bytes` | `b''` |
| `num_tokens` | `int` | `0` |
| `num_decoder_layers` | `int` | `0` |
| `top_k` | `int` | `0` |
| `layer_indices` | `list[int]` | `[]` |
| `handle` | `str` | `''` |

### PromotedStreamingReplica

Server-owned routing metadata for a promoted streaming replica.

| Field | Type | Default |
| --- | --- | --- |
| `checkpoint` | `str` |  |
| `status` | `str` |  |
| `base_url` | `str \| None` |  |
| `replica_id` | `str \| None` |  |
| `model` | `str` |  |
| `base_model` | `str` |  |
| `updated_at` | `str` |  |
| `status_reason` | `str \| None` | `None` |

### TrainingDataArtifact

Source bytes and expected digest for server-side integrity verification.

The API hashes `content` itself and retains only the resulting manifest.
Callers should verify their source files locally before constructing these
objects, then bind the returned `TrainingDataAttestation` to the
model they create.

| Field | Type |
| --- | --- |
| `name` | `str` |
| `expected_sha256` | `str` |
| `content` | `bytes` |

### AttestedTrainingDataArtifact

Server-observed source-artifact digest and size.

| Field | Type |
| --- | --- |
| `name` | `str` |
| `sha256` | `str` |
| `size_bytes` | `int` |

### TrainingDataAttestation

A server-verified source-artifact manifest bound to one session.

| Field | Type |
| --- | --- |
| `training_data_attestation_id` | `str` |
| `artifacts` | `list[AttestedTrainingDataArtifact]` |

---

## Errors

Every server and transport failure derives from `RiverError`, so a single `except river.RiverError` catches all of them. Invalid arguments still raise the standard `ValueError` and `TypeError`.

### RiverError

Inherits `Exception`.

Base exception for River client errors.

### AuthenticationError

Inherits `RiverError`.

Authentication failed.

### CapacityError

Inherits `RiverError`.

No capacity available for the operation.

### ModelNotFoundError

Inherits `RiverError`.

Model not found.

### RiverConnectionError

Inherits `RiverError`.

Connection or communication error with the River API server.

This wraps gRPC errors with a more user-friendly message while preserving
the original error details for debugging.

| Field | Description |
| --- | --- |
| `message` | Human-readable error message |
| `status_code` | gRPC status code name (e.g., "UNAVAILABLE", "DEADLINE_EXCEEDED") |
| `details` | Additional error details from the server |
| `original_error` | The original gRPC RpcError for debugging |

#### RiverConnectionError.from_grpc_error

```python
RiverConnectionError.from_grpc_error(
    error: Exception,
    context: str = 'API call',
) -> 'RiverConnectionError'
```

Create a RiverConnectionError from a gRPC RpcError.

| Parameter | Type | Default |
| --- | --- | --- |
| `error` | `Exception` |  |
| `context` | `str` | `'API call'` |

### RiverTimeoutError

Inherits `RiverError`.

Operation timed out while retaining its recoverable future ID.

| Field |
| --- |
| `request_id` |

### SessionHeartbeatError

Inherits `RiverConnectionError`.

Session heartbeat was lost or rejected while a session was active.

---

## Tokenizers

Helpers for tokenizing prompts yourself, for example when you build `prompt_token_ids` for training data.

### load_tokenizer

```python
load_tokenizer(
    tokenizer: str | Any | None = None,
    *,
    base_model: str | None = None,
    revision: str | None = None,
    local_files_only: bool = False,
    resolve_aliases: bool = True,
)
```

Load or return a tokenizer for River client result parsing.

`base_model` remains the public River model name used for routing. When
it is a hyphen-suffixed deployment alias of a known canonical model, this
helper resolves it to the underlying Hugging Face tokenizer id before
loading. `local_files_only` keeps sealed jobs from reaching Hugging Face
after their tokenizer revision has been frozen. `resolve_aliases=False`
retains a deployment's own tokenizer source for unpinned compatibility
paths.

| Parameter | Type | Default |
| --- | --- | --- |
| `tokenizer` | `str \| Any \| None` | `None` |
| `base_model` | `str \| None` | `None` |
| `revision` | `str \| None` | `None` |
| `local_files_only` | `bool` | `False` |
| `resolve_aliases` | `bool` | `True` |

### resolve_tokenizer_name

```python
resolve_tokenizer_name(model_name: str) -> str
```

Return the tokenizer for a canonical River model or one of its aliases.

Deployment aliases must append a hyphen-delimited suffix to a canonical
model name. Longest-root matching keeps the result deterministic if a
future canonical model name extends another root.

| Parameter | Type |
| --- | --- |
| `model_name` | `str` |

### MODEL_TOKENIZER_ALIASES

```python
MODEL_TOKENIZER_ALIASES: dict[str, str]
```

| Key | Value |
| --- | --- |
| `Qwen/Qwen3.6-35B-A3B-FP8` | `Qwen/Qwen3.6-35B-A3B` |
| `Qwen/Qwen3.5-397B-A17B-FP8` | `Qwen/Qwen3.5-397B-A17B-FP8` |
| `nvidia/Kimi-K2.6-NVFP4` | `nvidia/Kimi-K2.6-NVFP4` |
| `nvidia/GLM-5.1-NVFP4` | `nvidia/GLM-5.1-NVFP4` |
| `nvidia/GLM-5.2-NVFP4` | `nvidia/GLM-5.2-NVFP4` |