> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/NVIDIA/OpenShell/llms.txt
> Use this file to discover all available pages before exploring further.

# openshell inference

> CLI reference for inference commands: set, update, and get. Configure the gateway-scoped LLM inference route used by sandbox workloads.

# openshell inference

Manage the gateway-level inference configuration. The inference route determines which provider and model are used when sandbox workloads call `inference.local` — the managed LLM endpoint injected by the policy engine.

Inference configuration is scoped to the active gateway, not to individual sandboxes.

***

## openshell inference set

Set the gateway-level inference provider and model. Overwrites the current configuration.

```bash theme={null}
openshell inference set --provider NAME --model MODEL [OPTIONS]
```

<ParamField body="--provider" type="string" required>
  Provider name (must already exist — see `openshell provider create`). The provider's credentials are used for all inference calls routed through `inference.local`.
</ParamField>

<ParamField body="--model" type="string" required>
  Model identifier to use for generation calls (e.g., `gpt-4o`, `claude-opus-4-5`, `meta/llama-3.1-70b-instruct`).
</ParamField>

<ParamField body="--system" type="boolean">
  Configure the system inference route instead of the user-facing route. The system route is used by platform functions (e.g., the agent harness) and is not accessible to user code.
</ParamField>

<ParamField body="--no-verify" type="boolean">
  Skip endpoint verification before saving the route. By default, the CLI makes a test request to confirm the provider and model are reachable.
</ParamField>

### Examples

```bash theme={null}
# Set the inference route to GPT-4o via an OpenAI provider
openshell inference set --provider openai --model gpt-4o

# Set to Claude via an Anthropic provider
openshell inference set --provider anthropic --model claude-opus-4-5

# Set the system inference route
openshell inference set --provider openai --model gpt-4o --system

# Set without verifying the endpoint (useful in offline/staging setups)
openshell inference set --provider openai --model gpt-4o --no-verify
```

***

## openshell inference update

Partially update the gateway-level inference configuration. Only the flags you provide are changed; omitted flags leave the current values in place.

```bash theme={null}
openshell inference update [OPTIONS]
```

<ParamField body="--provider" type="string">
  Provider name. Unchanged if omitted.
</ParamField>

<ParamField body="--model" type="string">
  Model identifier. Unchanged if omitted.
</ParamField>

<ParamField body="--system" type="boolean">
  Target the system inference route instead of the user-facing route.
</ParamField>

<ParamField body="--no-verify" type="boolean">
  Skip endpoint verification before saving.
</ParamField>

### Examples

```bash theme={null}
# Swap to a different model, keeping the same provider
openshell inference update --model gpt-4-turbo

# Change provider and model together
openshell inference update --provider anthropic --model claude-sonnet-4-5
```

***

## openshell inference get

Show the current gateway-level inference configuration.

```bash theme={null}
openshell inference get [OPTIONS]
```

<ParamField body="--system" type="boolean">
  Show the system inference route instead of the user-facing route. When omitted, both routes are displayed.
</ParamField>

### Examples

```bash theme={null}
# Show all inference routes
openshell inference get

# Show only the system route
openshell inference get --system
```

***

## How inference routing works

When a sandbox workload calls `inference.local`, the policy engine:

1. Intercepts the outbound request.
2. Strips the caller's credentials.
3. Injects the configured provider's credentials.
4. Forwards the request to the provider's API.

This keeps model API credentials out of sandbox environments entirely — agents call a stable local endpoint and never see the underlying keys.

<Note>
  The inference route must reference a provider that exists on the gateway. Create one first with `openshell provider create` if you have not already.
</Note>

## Supported provider types

Any provider type registered with `openshell provider create` can be used as an inference provider, provided the underlying API is compatible with the OpenAI chat completions interface. The following types are tested and known to work:

| Provider type | Example model values                   |
| ------------- | -------------------------------------- |
| `openai`      | `gpt-4o`, `gpt-4-turbo`, `o1`          |
| `anthropic`   | `claude-opus-4-5`, `claude-sonnet-4-5` |
| `nvidia`      | `meta/llama-3.1-70b-instruct`          |
| `opencode`    | Depends on backing service             |