Adding a provider

A "provider" is an LLM upstream Pluma can talk to. Today's set: OpenAIProvider (handles every OpenAI-compatible backend) and AnthropicProvider (talks the Messages API natively, translates back to OAI shape).

This page walks through adding a new one. The future plugin-axis story keeps the same interface at the call site, so what you write here ports forward unchanged.

The interface

// server/providers.go
type Provider interface {
    ID() string
    Upstream() string
    ListModels(ctx context.Context) ([]ModelInfo, error)
    StreamChat(ctx context.Context, req ChatRequest, onLine func(string) error) error
}

Method	Returns
ID()	Short stable string. Matches the compat field in connections.json. Used in logs.
Upstream()	The base URL the provider was constructed with. For diagnostics + the health endpoint.
ListModels()	Models the upstream advertises through whatever its model-list endpoint is.
StreamChat()	Sends the request, calls onLine with each SSE line from the upstream (or synthesises SSE-shaped lines from non-streaming responses).

Skeleton

package main

import (
    "context"
    "net/http"
)

type MyProvider struct {
    baseURL string
    apiKey  string
    client  *http.Client
}

func NewMyProvider(profile ConnectionProfile, client *http.Client) Provider {
    return &MyProvider{
        baseURL: profile.BaseURL,
        apiKey:  profile.APIKey,  // resolved from keyring by the caller
        client:  client,
    }
}

func (p *MyProvider) ID() string       { return "mybackend" }
func (p *MyProvider) Upstream() string { return p.baseURL }

func (p *MyProvider) ListModels(ctx context.Context) ([]ModelInfo, error) {
    // Hit your upstream's model-list endpoint; map to ModelInfo.
    // ModelInfo = { id, display, family }.
    return nil, nil
}

func (p *MyProvider) StreamChat(ctx context.Context, req ChatRequest, onLine func(string) error) error {
    // POST to your upstream.
    // For each chunk in the response, call onLine("data: <json>") with an
    // OAI-shaped delta. End with onLine("data: [DONE]").
    // Returning an error stops the stream + emits an error event to the FE.
    return nil
}

Registration

server/providers.go keeps a factory map. Add yours in init():

func init() {
    RegisterProvider("mybackend", NewMyProvider)
}

The connection profile's compat field picks the factory. Update the UI's compat dropdown in web/src/components/SettingsConnections.svelte to include the new option.

Translating non-OAI shapes

If your upstream doesn't speak OpenAI streaming SSE, translate inside StreamChat. The pattern from AnthropicProvider:

1. Build the upstream-native request body.
2. Stream the response.
3. For each chunk, convert to an OAI-shape chat.completion.chunk JSON: {"choices":[{"delta":{"content":"..."}}]}.
4. Emit data: <json>\n via onLine.
5. After the last chunk: onLine("data: [DONE]").

The FE doesn't know the upstream isn't OAI.

Testing

A test file at server/myprovider_test.go. The pattern:

func TestMyProviderStreamChat(t *testing.T) {
    srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        // emit your upstream's SSE shape
    }))
    defer srv.Close()

    p := NewMyProvider(ConnectionProfile{BaseURL: srv.URL}, http.DefaultClient)
    var got []string
    err := p.StreamChat(context.Background(), ChatRequest{ /* ... */ }, func(line string) error {
        got = append(got, line)
        return nil
    })
    // assert err is nil, got contains the OAI-shape lines you expected
}

make test-go picks it up.

Authentication

API keys live in the OS keyring under connection:<profile-id>. By the time NewMyProvider runs, the caller (ConnectionManager.Provider()) has already resolved the key into profile.APIKey. Use it as a normal string; don't reach back into the keyring yourself.

Streaming SSE format

OpenAI's chat.completion.chunk shape Pluma's FE expects:

data: {"choices":[{"delta":{"role":"assistant"}}]}
data: {"choices":[{"delta":{"content":"Hello"}}]}
data: {"choices":[{"delta":{"content":" world"}}]}
data: [DONE]

Empty lines between data: lines are fine; the FE parses one event per data: line. The [DONE] sentinel signals end-of-stream.

Errors: throw an error from StreamChat; the chat handler emits an event: error\ndata: <message> SSE event the FE renders inline.

Plugin axis (future)

When the plugin axis (smelt-7bq) lands, the Provider interface stays at the call site. Built-in providers (like yours) keep registering at init() against an in-process bypass. Community providers register over the gRPC bus by implementing the matching protobuf service. Same interface either way.