llm-council

Several language models, one conversation, in turn.

You pick a roster of participants. On each message they answer one after another, and every later participant sees what the earlier ones said — so they actually build on, agree with, or push back against each other, instead of you fanning the same prompt out N times and stitching the replies together yourself.

• Roundtable cross-talk — participant k sees participants 1..k-1's replies from this round.
• Three modes — roundtable (default), conductor (a lead drafts, workers refine, the lead synthesizes one final answer), and debate (rounds repeat until the positions measurably converge — by a deterministic φ⁻¹ similarity metric, not a model's opinion).
• Any provider — any OpenAI-compatible endpoint (OpenAI, Ollama, vLLM, LM Studio, Together, Groq, …) and Anthropic, configured by environment.
• Cost-aware by default — participants are free or metered. A roster with a metered participant won't run until you explicitly confirm, so a large roster can't surprise-bill you. A local model is free.
• Streaming — Server-Sent Events; every token is tagged with the participant that produced it, so the UI renders per-participant blocks live.
• Small — one async process. No database, no daemon, no auth.

The design is in SPEC.md.

Quick start

pip install -r requirements.txt

# at least one provider — e.g. local Ollama (free) or a hosted key
export OLLAMA_BASE_URL=http://127.0.0.1:11434/v1     # default; free
# export OPENAI_API_KEY=sk-...
# export ANTHROPIC_API_KEY=sk-ant-...

python -m council.server          # http://127.0.0.1:8800

Open http://127.0.0.1:8800, add a couple of participants, send a message.

API

curl -N http://127.0.0.1:8800/api/council/stream \
  -H 'Content-Type: application/json' \
  -d '{
        "messages":[{"role":"user","content":"Name a primary color, briefly."}],
        "roster":[
          {"provider":"ollama","model":"llama3.2"},
          {"provider":"ollama","model":"qwen2.5"}
        ],
        "mode":"debate"
      }'

mode is optional (roundtable | conductor | debate, default roundtable); an unknown value is rejected before anything runs.

SSE events: council_started ({mode,participants}), participant_turn_started, token ({pid,text}), participant_turn_done, participant_error (non-fatal — the round continues), cost_gate, round_done ({mode,...}), done. Debate adds debate_round_started / debate_converged; conductor tags turns with a phase (lead/worker/synthesis). Full table in SPEC.md §4.

A roster with a metered provider returns a cost_gate event and stops; re-send with "confirm_metered": true to proceed.

Configuration

Env	Effect
OPENAI_API_KEY (+ OPENAI_BASE_URL, OPENAI_PRICE_IN/OUT)	provider openai
ANTHROPIC_API_KEY (+ ANTHROPIC_BASE_URL, ANTHROPIC_PRICE_IN/OUT)	provider anthropic
OLLAMA_BASE_URL	provider ollama (default http://127.0.0.1:11434/v1, free)
OPENAI_COMPAT_BASE_URL (+ OPENAI_COMPAT_API_KEY, ..._PRICE_IN/OUT)	provider compat — any other OpenAI-compatible host
HOST / PORT	bind (default 127.0.0.1:8800)

A localhost endpoint is treated as free. A remote provider with no price configured is treated as free too — set *_PRICE_IN/OUT if it bills, and it will move behind the cost gate.

Tests

pip install -r requirements.txt
pytest          # no network — a fake provider streams canned text

License

MIT.