The OpenAI-wire provider — Ollama, vLLM and more

A ModelProvider for any endpoint speaking the OpenAI chat-completions wire format. No SDK: one POST {baseUrl}/chat/completions and an event-stream parser. A local, air-gapped install is a configuration choice.

Install

npm install @orthacms/copilot-provider-openai

Configure

# .env — local Ollama
COPILOT_ENABLED=true
# setting it registers the backend
COPILOT_OPENAI_BASE_URL=http://localhost:11434/v1
# comma-separated, first is the default
COPILOT_OPENAI_MODELS=llama3.1
# empty for no auth
COPILOT_OPENAI_API_KEY=

Variable	Default	Notes
`COPILOT_OPENAI_BASE_URL`	unset	Include the version segment: `…/v1`
`COPILOT_OPENAI_MODELS`	`llama3.1`	Comma-separated
`COPILOT_OPENAI_API_KEY`	empty	Sent as a bearer token when set

Runtime	`COPILOT_OPENAI_BASE_URL`
Ollama	`http://localhost:11434/v1`
vLLM	`http://localhost:8000/v1`
LM Studio	`http://localhost:1234/v1`
OpenAI	`https://api.openai.com/v1`
LiteLLM, OpenRouter, Azure	the URL that deployment publishes

The reference server, apps/server/config/copilot.ts:

const openAiBaseUrl = readEnv('COPILOT_OPENAI_BASE_URL');

ollama: when(openAiBaseUrl, () => ({
    baseUrl: openAiBaseUrl as string,
    models: readList('COPILOT_OPENAI_MODELS', 'llama3.1'),
    apiKey: readEnv('COPILOT_OPENAI_API_KEY') ?? ''
})),

A generated app reads the same three variables in config/copilot-openai.ts and returns them under providers.openai.

All options

OpenAiProviderConfig:

Option	Default	Notes
`baseUrl`	required	Trailing slashes trimmed, `/chat/completions` appended
`models`	required	Non-empty
`apiKey`	none
`headers`	none	Extra request headers for a gateway
`capabilities`	see below	Partial; merged over the defaults
`timeoutMs`	`120000`	One request, retries and backoff included
`maxRetries`	`2`	Pre-stream only
`maxTokensField`	`max_tokens`	`max_completion_tokens` for OpenAI reasoning models

Default capabilities: tool calling and streaming on, vision off, 32,768-token context, 4,096 output tokens.

Register

// apps/server/src/plugins.ts
import { createOpenAiProvider } from '@orthacms/copilot-provider-openai';

const { ollama } = config.plugins.copilot.providers;

CopilotPlugin({
    providers: ollama
        ? [{ name: 'ollama', provider: createOpenAiProvider(ollama) }]
        : [],
    config: config.plugins.copilot
});

Two endpoints side by side are two registrations:

providers: [
    { name: 'ollama-fast', provider: createOpenAiProvider({
        baseUrl: 'http://localhost:11434/v1',
        models: ['llama3.1'],
        capabilities: { toolCalling: false, contextWindow: 8_192 }
    }) },
    { name: 'openai', provider: createOpenAiProvider({
        baseUrl: 'https://api.openai.com/v1',
        apiKey: process.env.OPENAI_API_KEY,
        models: ['<model-id>'],
        maxTokensField: 'max_completion_tokens'
    }) }
]

Verify

Check the endpoint first, from the server’s host:

curl -s http://localhost:11434/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{"model":"llama3.1","messages":[{"role":"user","content":"ping"}]}'

Then the catalogue:

curl -s -b cookies.txt http://localhost:3000/api/copilot/models
# {"items":[{"provider":"ollama","model":"llama3.1"}, ...]}

Sign-in and a streamed run: the copilot page.

Declare capabilities

There is no probe: no endpoint reliably reports its own. The default is the optimistic case.

Small models: declare toolCalling: false

A model that cannot call tools must be declared { toolCalling: false }. Otherwise runs fail confusingly instead of dropping into degraded mode. Anything below tool calling, streaming and a 32,000-token context runs degraded, and the admin says so.

Behaviour worth knowing

Retries run before the stream only. Retried: refused connection, 408, 429, 5xx. A Retry-After over 30 seconds ends the retries.
Tool-call fragments are reassembled by index, so parallel calls arrive as whole calls.
Unknown model = error, no request.