Skip to content

Performance: Move blocking inventory and hardware work off async request paths #140

Description

@BerryUIKI

Summary

Manager/model endpoints call synchronous directory scanning and hardware evaluation; hardware inspection uses subprocesses. Lifecycle operations use synchronous process calls, while async asset methods perform synchronous file writes/hashing. main.py:236 rescans inventory for status.

Environment and evidence

  • Review finding: F40 (2026-10-05 product/technical review).
  • Baseline: Windows, dev at c1c5dbe. The remote dev matched this commit when filing.
  • Evidence: Code.
  • Priority recommendation: P2 - material improvement or integration validation. This is review triage, not a production-incident severity declaration.
  • No real credentials, paid inference, engine installation, or unrelated process termination were used for review probes. Mocked observations establish the stated code behavior, not live-provider/GPU acceptance.

Reproduction or validation

Trace manager/model routes into synchronous scans and hardware subprocess calls; async asset methods also perform synchronous file writes/hashing. Status requests rescan inventory. No load/latency benchmark was run, so this issue identifies an event-loop blocking risk rather than claiming a measured latency number.

User impact

Large libraries or slow disks/processes can delay unrelated health, task, and UI requests. No load benchmark was run.

Proposed approach

Move blocking work to bounded worker threads/processes, maintain an incremental model index, cache hardware/status snapshots, and measure request latency during large imports/scans.

Acceptance criteria

  • Run blocking operations in bounded workers.
  • Maintain an incremental model index and cached status/hardware snapshots.
  • Avoid full inventory scans on routine status polling.
  • Measure health/task latency during representative large scans and imports.

Additional source references

Verification scope

Real GPU inference, paid-provider compatibility, and a clean-machine packaged desktop journey remain unverified. Any follow-up implementation should target dev under the repository's contribution/branching rules.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    backendBackend Python / FastAPI / Engine issuesperformancePerformance, memory leaks, and socket/resource management

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions