How the API works

River lets your Python program train and use an LLM running on River's GPUs. You create a model instance, ask it to generate responses, and update it with training data. Your code decides what to teach; River runs the model and computes the updates. You do not need a local GPU.

The core distinction is simple: sampling uses the current weights; training changes them. Sampling by itself does not teach the model anything.

Create a model instance

There are three objects to recognize:

After the quickstart, you can create an instance like this:

import os
from contextlib import closing

import river_client as river

with closing(river.Client(api_key=os.environ["RIVER_API_KEY"])) as client:
    with client.session() as session:
        model = session.create_model(
            base_model="Qwen/Qwen3.5-9B",
            lora=river.LoraConfig(rank=8),
        )
        samples = model.sample("What is 2 + 2?", max_tokens=256)
        print(samples[0][0].text)

create_model starts from an existing model's pretrained weights. It creates your trainable instance; it does not train a new LLM from scratch. River uses LoRA adapters, a small set of additional weights that your training updates. The rank of 8 is a starting configuration; you can learn about adapter choices later.

The Python model object is a handle to that remote instance. Keep subsequent sampling and training calls inside the session. Exiting the with block releases its live models.

Sample to see what the model does

model.sample(prompt, ...) generates responses using that instance's current weights. A sample includes the generated .text, along with token data for training algorithms that need it.

The result is grouped by prompt and then by response. In the example, samples[0][0] means the first response to the first prompt. max_tokens limits how many tokens the model generates.

The quickstart uses client.sample(..., base_model=...) to generate from a base model without creating a trainable instance. Once you are training, use model.sample(...) to try your instance's updated weights. You do not need to save or deploy it between training and sampling.

Train with one call

Once you have a prepared training batch, one call applies an update:

model.train_step(batch, loss_fn="cross_entropy", lr=1e-4)

Run this inside the same session as model. A batch is a list of training examples. For SFT, those examples contain tokenized text and weights selecting which response tokens to learn from. The SFT walkthrough shows how to prepare one; you do not need to understand the token arrays yet.

loss_fn chooses the learning objective. Here, cross_entropy teaches the model to imitate the examples. lr is the learning rate, which controls the size of the update. Use the settings in the walkthrough for your first run.

train_step computes the training loss, calculates gradients—the direction in which to adjust the trainable weights—and applies an optimizer update. The call waits for the results. You can then call model.sample(...) again to inspect the updated model. One update is one step in learning, not a guarantee that the response will already improve.

SFT, reinforcement learning, and distillation use these same operations. What changes is how you build the batch and choose the learning signal. In RL, your program samples attempts and scores them before constructing the update; River does not decide the task's reward for you.

Optional: split the training step

Most introductory code can use train_step. When you need more control, its two parts are available separately:

model.forward_backward(batch, loss_fn="cross_entropy")
model.optim_step(lr=1e-4)

forward_backward computes the loss and gradients, but does not change the weights. optim_step applies those gradients to change the weights. Separating them lets you accumulate several batches or inspect results before updating. The custom RL chapter covers those controls and how failure handling differs from train_step.

Keep the result and continue

Save the weights before leaving the session if you want to use them later:

checkpoint = model.save_weights("my-first-model", mode="training")
print(checkpoint.path)

A checkpoint preserves the trained result beyond the live instance's lifetime. Later chapters explain how to resume it or deploy it.

You now have the API's core loop: create an instance, prepare a batch, train, and sample. Continue to Learning from examples to understand the simplest learning signal, or use the first SFT run to put the whole loop together.