The agent can think with a model you run yourself, or one behind your own gateway, instead of a hosted provider. Anything that serves the OpenAI chat completions API works.

​
Pick the provider

Open the Persona and go to Model (LLM). Temperature and Max Tokens are under More settings, as for every provider. Max Tokens reaches Ollama and Custom servers as max_tokens, the name they all read.

​
Keys

Ollama and Custom models need no key. In Settings their row says Key optional, and the readiness check never lists them as missing. Paste a key only if your server checks one: vLLM started with --api-key, a LiteLLM master or virtual key, or a gateway token. The agent sends it as a bearer token. It is kept like any key you paste (see Keys): in this browser tab, and with any version you publish. Save never stores it for the workspace, and the workspace has no key for these providers. OpenAI with a Base URL uses only a key pasted in Settings. Your workspace’s own OpenAI key is never sent to another server, so the readiness check warns when the agent would otherwise rely on it.

​
The address must be reachable from the agent

The model is called by the agent worker, not by your browser, so the Base URL has to work from where the worker runs.
  • Hosted workspace. The worker runs on the workspace’s server, where localhost points at nothing of yours. Use an address on the internet, over HTTPS, and protect it with a key.
  • Self-hosted with Docker. Inside the agent container, localhost is the container itself. For a server on the same machine, use http://host.docker.internal:11434/v1 (Docker Desktop), or on Linux add host.docker.internal:host-gateway under extra_hosts in the agent service. A server in the same Compose project is reached by its service name, for example http://vllm:8000/v1.
  • Self-hosted without Docker. localhost is the machine that runs the worker.

​
Servers

​
Ollama

Pull a model with ollama pull llama3.1, then pick Ollama and type llama3.1 as the Model. Ollama listens only on the machine it runs on by default; to reach it from a container or another machine, make it listen on all interfaces (Ollama’s FAQ explains how) and set Base URL to that address.

​
vLLM

Pick Custom (OpenAI-compatible), set Base URL to http://your-host:8000/v1, Model to meta-llama/Llama-3.1-8B-Instruct, and paste the token in Settings. Agents call tools (to end the call, to move between Skills, and every tool you connect), and vLLM rejects a request that offers tools unless it was started with --enable-auto-tool-choice and the --tool-call-parser for your model family, for example llama3_json or hermes.

​
LiteLLM

Run the LiteLLM proxy with your config (it listens on port 4000), pick Custom (OpenAI-compatible), set Base URL to http://your-host:4000/v1 and Model to a model_name from the config. If the proxy has a master key, paste it or a virtual key in Settings. LiteLLM in turn reaches any provider it supports.

​
LM Studio

Start the local server in LM Studio’s Developer tab (port 1234), and turn on serving on the local network if the agent runs on another machine or in a container. Pick Custom (OpenAI-compatible), set Base URL to http://your-host:1234/v1, and copy the model’s identifier into Model.

​
Gateways

  • In front of OpenAI only: pick OpenAI, set Base URL under More settings, and paste the key the gateway expects.
  • Many providers behind one OpenAI-style API: pick Custom (OpenAI-compatible) and use the model names the gateway lists.

​
What to expect

  • The model has to call tools. The agent ends calls and moves between Skills through tool calls in the OpenAI format. A model or server without tool calling stalls or never hangs up.
  • The first word has to come quickly. A caller waits for the model’s first token. When the server is silent for about 5 seconds, including before its first token, the worker treats the request as failed, so a slow model or a cold start costs the turn.
  • After the call. Structured Output uses the same server, through its Responses API (/v1/responses). A server without that API fails the extraction; the call itself is unaffected.
  • Not covered yet. Model comparisons and evals do not run Ollama or Custom models.