Throughput and the Console

Throughput and batch size

Training and inference run on an autoscaling worker pool — River scales capacity up and down automatically to match your load. Because there's fixed per-step overhead, bigger batches generally give higher throughput: packing more prompts/sequences into a single forward_backward or sample call amortizes that overhead.

Increase batch size gradually and measure tokens per second alongside training quality. The useful limit depends on the model, sequence lengths, and capacity available to your account or cluster. Dedicated production deployments use the replica targets you configure; scale those explicitly as serving demand changes.

River Console

The River Console gives your team a shared view of training runs, checkpoints, deployments, usage, and spend. Use it alongside the Python client to check run status, inspect saved checkpoints, and manage API keys.

To report your own loss, reward, or evaluation metrics and compare experiments, see Metrics and charts. It covers model.log(), chart axes, and where to find run and project charts in Console.

Export checkpoints to Hugging Face

River's internal LoRA names and fused projections can differ from the base model's Hugging Face layout. The converter in river-client 0.13.0 and later writes an HF/PEFT adapter with matching target names, tensor shapes, and scaling. It exports the adapter, not merged base weights or training/optimizer state.

Download and convert from Console

Open Checkpoints, select an inference checkpoint, and choose Download HF export script. Save the script as download.py, then run:

pip install -U "river-client[hf-export]>=0.13.0" requests
export RIVER_API_KEY="your-api-key"
python download.py

The script downloads the checkpoint and converts it locally into a directory named after the checkpoint ID. Use that directory with the corresponding base model. A river:// path identifies a checkpoint inside River; it is not a public download URL or a filesystem path accepted by the converter.

Convert an existing local checkpoint

The source directory must contain adapter_config.json and either adapter_model.safetensors or its sharded index and referenced files.

pip install -U "river-client[hf-export]>=0.13.0"
python -m river_client.hf_export ./river-checkpoint ./hf-adapter

The output contains adapter_config.json and adapter_model.safetensors in HF/PEFT adapter format. The source is preserved, and an existing output directory is never overwritten. The converter validates every adapter target against the base model before publishing the output directory. It runs on CPU and needs memory proportional to the adapter size.

By default, it reads the base model named in the adapter configuration. Set HF_TOKEN for a gated base model. Only safetensors headers are fetched, not the base weights. Override the model and pin its revision when needed:

python -m river_client.hf_export ./river-checkpoint ./hf-adapter \
  --base-model "<base-model-id>" --revision "<base-model-commit>"

Use --base-model /path/to/local/base to read local base-model headers. The same operation is available from Python:

from river_client.hf_export import export_hf_adapter

output = export_hf_adapter("./river-checkpoint", "./hf-adapter")
# Optional keyword arguments: base_model="...", revision="...".
print(output)

Conversion preserves adapter values and dtype; inference compatibility also depends on the loader. In our tests with Transformers 5.8.1 and PEFT 0.18.1, loading BF16 adapters onto native FP8 layers cast the adapter weights to FP8, changing their values. Use a loader that preserves BF16 adapters alongside the FP8 base. The workaround verified in that test was to load the base with FineGrainedFP8Config(dequantize=True) before attaching the adapter. This requires more memory and does not quantize the adapter.

Download unsupported layouts

For layouts the converter does not support, including NVFP4 expert adapters, choose Download raw script in Console. Install requests, then run RIVER_API_KEY=... python download-raw.py. It saves the original checkpoint files unchanged under <checkpoint-id>-raw, without requiring the HF converter or access to the base model. Raw files are not converted to HF format.