or is a modular Go toolkit for building applications with language models and
higher-level agents. A provider-neutral LLM package keeps conversations, tools,
reasoning, and streaming events stable while models and wire protocols change
underneath, and an agent package builds the tool-call loop, state, and streaming
events on top.
- Use one conversation model across OpenAI-compatible and Anthropic-compatible providers.
- Stream text, reasoning, tool calls, usage, and errors through typed events.
- Define tools from Go structs and validate model-generated arguments.
- Preserve provider metadata needed for multi-turn reasoning and tool use.
- Switch models between turns without rebuilding conversation history.
- Add custom model protocols without expanding the shared request API.
- Run autonomous multi-step tool loops with streaming events, mid-run steering, and per-turn model switching.
- Layer transcript persistence, context compaction, per-turn system prompts, and skills on top with the harness.
| Package | Status | Description |
|---|---|---|
or/llm |
Available | Unified model access, streaming, tools, reasoning, images, and conversation history |
or/agent |
Available | Stateful agent loop with tools, streaming events, steering, follow-ups, and abort |
or/agent/harness |
Available | Orchestration over the agent: transcript persistence, context compaction, per-turn system prompt, skills, and prompt templates |
Future packages can build higher-level orchestration on the same foundations without turning the root package into a single large API.
- Go 1.25 or later
- An API key for the selected hosted provider, or a compatible local endpoint
Install the LLM package:
go get github.com/ktsoator/or/llm@latestSet the API key expected by the selected provider. For example:
export DEEPSEEK_API_KEY=your-deepseek-api-keySee Providers and models for supported provider IDs, environment variables, catalog discovery, and custom endpoints.
package main
import (
"context"
"fmt"
"log"
"github.com/ktsoator/or/llm"
_ "github.com/ktsoator/or/llm/openai" // registers the OpenAI-compatible protocol (DeepSeek, Groq, xAI, ...)
)
func main() {
model := llm.GetModel("deepseek", "deepseek-v4-flash")
response, err := llm.Complete(
context.Background(),
model,
llm.Prompt("Explain Go channels briefly."),
llm.StreamOptions{},
)
if err != nil {
log.Fatal(err)
}
fmt.Println(response.Text())
}Each protocol lives in a provider package that registers itself on import. Pull
in the protocols you use — and only their vendor SDKs — by importing the matching
provider package for its side effects (llm/openai, llm/anthropic), or import
llm/all for every built-in protocol at once.
Use llm.Stream instead of llm.Complete to consume deltas while the model is
generating:
events, err := llm.Stream(ctx, model, input, llm.StreamOptions{})
if err != nil {
log.Fatal(err)
}
for event := range events {
switch event.Type {
case llm.EventTextDelta:
fmt.Print(event.Delta)
case llm.EventError:
log.Fatal(event.Err)
}
}Guides for both packages live at ktsoator.github.io/or.
API reference: or/llm ·
or/agent
The built-in adapters implement:
- OpenAI-compatible Chat Completions
- Anthropic-compatible Messages
The model catalog includes explicit compatibility metadata for DeepSeek, MiniMax, Xiaomi MiMo, Z.AI, Moonshot AI, Kimi, Anthropic, OpenRouter, and other compatible providers. Catalog presence is not a guarantee that every model has been live-tested; both wire adapters are covered by automated mock-server tests.
v0.5.x builds on the or/agent package with or/agent/harness, a stateful
orchestration layer (transcript persistence, context compaction, per-turn system
prompt, and skills), and is the recommended baseline for new integrations. The
project remains pre-1.0, so APIs may continue to evolve between minor versions.
Breaking changes will be called out in release notes.
This project is inspired by and partially adapted from earendil-works/pi, created by Mario Zechner.
Released under the MIT License.