Local Model Support via Ollama $400

[rparcus]

Overview

Implement local model support via Ollama, enabling self-hosted LLM capabilities with optimized performance.

Core Features

• Ollama integration
• Local model management
• Model download and caching
• Performance optimization
• Resource management

Technical Details

• Implement Ollama client
• Add model management system
• Set up caching mechanisms
• Implement resource controls
• Add performance monitoring

Acceptance Criteria

• Implement Ollama API client
• Add model management functionality
• Create caching system for models
• Implement resource usage controls
• Add performance monitoring
• Include documentation and examples
• Add integration tests

Budget: $400

[BHARATJHAWAR52]

Hi @rparcus
I am interested in contributing to this project where we can run local models via Ollama.
I have an experience in building models, and analyzing data, for various purposes. Also, I know about integrating Ollama with different projects.

[1914Jegx]

/attempt #96

Implementation plan:

1. Implement Lux.LLM.Ollama stack (client, resource pool, model manager, supervisor) with telemetry and configuration hooks.
2. Wire runtime/application config and env vars so Ollama defaults work locally and in tests.
3. Add documentation plus unit/integration coverage validating model caching, auto-pull, and request flow.

[IshGadol]

Hi team,

I’d like to take on this bounty and implement Ollama support for Lux. My plan is straightforward:

• Build a dedicated Ollama provider that plugs into Lux’s existing LLM integration layer.
• Add an HTTP client for Ollama’s API (/api/chat, /api/generate, /api/list, /api/pull, /api/show, /api/delete, etc.).
• Provide model management helpers (list, pull, show, delete) with light caching so models are pulled on-demand and tracked.
• Map Lux runtime options to Ollama’s resource controls (num_ctx, temperature, top_p, keep_alive, etc.).
• Support both streaming and non-streaming responses, plus JSON/Schema outputs using Ollama’s format parameter.
• Emit performance telemetry (load duration, tokens/sec, eval times) so metrics can feed into Lux’s monitoring.
• Ship a minimal working example (Beam) and documentation showing setup, config (OLLAMA_BASE_URL, OLLAMA_MODEL, etc.), and usage.
• Add tests: unit tests with mocked responses, and optional integration tests gated on a local Ollama install.

A couple of quick clarifications to align with your expectations:

• Should the default interface be /api/chat, or do you also want /api/generate supported in v1?
• Are embeddings (/api/embeddings) in scope, or should I focus only on chat/generate for now?
• Any preference on the default model (e.g., llama3.2:latest) for the demo/example Beam?

If this scope looks good to you, I’ll get started. Thanks for putting up the bounty—I’m confident I can deliver this cleanly and keep it aligned with Lux’s direction.

—David

[lmee]

I went through "Local Model Support via Ollama $400" plus the discussion thread and mapped the failing path.

Problem I see: Implement local model support via Ollama, enabling self-hosted LLM capabilities with optimized performance.
Root-cause hypothesis: a stale-state race between optimistic updates and confirmed data.
First patch plan: trace policy creation gating and billing-plan conditions, patch the branch creating invalid records, and add tests for control/annual plan combinations.
If this aligns with maintainer expectations, I will post repro evidence first and then open the scoped patch.

[sushi-levanter]

Hi! I've submitted PR #374 implementing Ollama LLM provider support for local models. Includes a full OllamaProvider module with model management, chat/generate/embeddings APIs, streaming support, and comprehensive tests. Happy to iterate based on feedback.

[sushi-levanter]

Hi! I've submitted PR #374 for Ollama LLM provider support. Apologies for not commenting first per the bounty guidelines. The implementation includes full Ollama API integration with model management, streaming, embeddings support, and comprehensive tests. Open to feedback and scope discussion.