fix: make ParserManager and parse caches thread-safe

[r0h1tb]

Summary

I picked up #12 (parallelise file parsing), built it, benchmarked it — and it doesn't pay off on this binding. So this PR ships the thread-safety work that came out of it, which turned out to be justified for an unrelated and more immediate reason, and drops the parallel driver.

The thread-safety bugs are live today

WorkspaceWatcher builds its ParserManager on the main thread but parses from threading.Timer callbacks — a new thread per debounce cycle (ast_rag/services/watcher_service.py). So a single tree_sitter.Parser, which holds mutable state across a parse, is already shared between threads during any watch session. Three concrete defects:

1. Shared parser across threads. Parsers are now thread-local, created on first use per thread. Language and compiled Query objects stay shared — safe, because the extractors allocate a fresh QueryCursor per call and never mutate the Query. The constructing thread keeps the parsers built in _init_languages, so single-threaded behaviour is byte-identical.

2. LazyTree._ensure was racy. Its docstring promises the loader runs "exactly once", but two threads could both see _tree is None and both parse. Now double-checked locked, so the already-resolved fast path stays lock-free.

3. BoundedParseCache._hashes grew without bound. The LRU evicted trees but never the corresponding hashes, so a bounded cache leaked over a long watch session. Hashes are now pruned to the live entries on put().

Cache backends (ParseCache, BoundedParseCache, SQLiteParseCache) are guarded by locks. Worth noting check_same_thread=False on the SQLite connection only silenced sqlite3's ownership assertion — it never serialised access.

On #12: parallel parsing isn't the lever

Measured on 10 cores, CPython 3.12, tree-sitter 0.24 (benchmarks/parallel_parsing_benchmark.py, included so this can be re-checked rather than re-argued):

workload	8 workers
parse only (tree-sitter C)	0.99x
extract only (Python)	1.17x
end to end	0.96x
end to end, ProcessPoolExecutor	0.22x

py-tree-sitter does not release the GIL around Parser.parse, so the C parsing that should have been the whole win is fully serialised. Extraction is Python (walking query matches, building ASTNode/ASTEdge) and is GIL-bound by construction. What remains is lock and scheduling overhead, which is why end-to-end lands slightly below 1.0.

Processes are worse: per-file work is only a couple of milliseconds, Tree objects aren't picklable so workers can't share the parse cache and must re-parse, and shipping extracted nodes/edges back over a pipe dominates.

I had a working ThreadPoolExecutor driver and a --jobs flag, and removed both rather than add complexity and a config surface for a measurable slowdown. The direction in #12's own note — moving parsing to Rust, or otherwise getting the work out from under the GIL — is the one that would actually pay.

Tests

9 new tests in tests/test_parser_thread_safety.py covering parser isolation per thread, extraction equivalence between sequential and concurrent runs, cache counter coherence under contention, the SQLite single-connection path, the load-once guarantee, and the hash leak.

Full suite: 182 passed. The 4 failures on this branch (test_summarizer, test_unsupported_language, test_update_project_dry_run x2) also fail on a clean main — missing optional deps such as mcp, unrelated to this change.

Closes nothing outright; leaves #12 open with data attached.

[r0h1tb]

Rebased onto current main. CI here was red only at the unpinned-ruff step that #51 has since fixed — this branch predated it, so the rebase is the whole fix; no source changes.

Verified locally against main (6f71a9b):

	main	this branch
ruff check ast_rag/	pass	pass
ruff format --check ast_rag/ tests/	pass	pass
pytest tests/	209 passed, 1 skipped, 1 xfailed	218 passed, 1 skipped, 1 xfailed

The +9 are this PR's own tests/test_parser_thread_safety.py. No pre-existing test changed state.