LCORE-4313: (design) PII redaction strategy

Signed-off-by: Anik Bhattacharjee <anbhatta@redhat.com>

Author: anik120

docs/design/observability-opentelemetry/observability-opentelemetry-design.md

@@ -374,12 +374,93 @@ No **required** change to JSON requests/responses. The `/config` response gains
 - Which `OTEL_*` variables are included in the `/config` scrape?
 
 
+## Addendum: PII redaction strategy
+
+|                          |                                                                                   |
+|--------------------------|-----------------------------------------------------------------------------------|
+| **Date**                 | 2026-09-22                                                                        |
+| **Authors**              | Anik Bhattacharjee                                                                |
+| **Feature / Initiative** | [UIESTRAT-229: Enable collection & upload of Observability OTel data stripped of PII/sensitive data](https://redhat.atlassian.net/browse/UIESTRAT-229)                      |
+
+The three core inference spans (`/v1/query`, `/v1/streaming_query`, `/v1/responses`) carry
+raw, un-hashed request/response content for evaluation, relaxing the original metadata-only rule
+(**R7**; §Why "Safe observability by design"). This addendum covers how that content is
+redacted of PII.
+
+### The problem
+
+Raw span text (prompts, responses, RAG chunks, tool I/O) can contain PII, so it must be scrubbed
+before the span leaves LCORE — before OTLP export to a hosted backend such as LangFuse, and before
+any cross-org sharing. This is a portfolio-wide obligation from Red Hat's AI Assessment (AIA/PIA)
+process, not specific to LCORE.
+
+### The approach — Presidio from PyPI
+
+Use the upstream Presidio library directly. Add `presidio-analyzer` and `presidio-anonymizer`
+(both on PyPI) to `pyproject.toml`, and wrap them in a small `redact(text) -> str` helper that
+detects and replaces email, hostname, IP (v4/v6), location, organization, person, phone, and URL.
+No new infrastructure — a dependency plus one helper, applied wherever content is set on a span
+(next section).
+
+Note: Presidio needs a spaCy model for named-entity detection, which is a sizable download.
+
+### Where redaction runs — redact at emission, through one helper
+
+Content is redacted at the point it's set on the span. All content fields will go on spans
+through a **single centralized helper** — e.g. `set_content_attribute(span, key, text)` — that runs
+each value through the Presidio `redact()` helper before calling `set_attribute`. For any content
+serialized as a JSON string (e.g. RAG chunks or tool I/O), the helper redacts each leaf string
+(`content`/`args`/`source`) before `json.dumps`, so the serialized attribute never contains raw
+PII. Properties of this approach:
+
+- **The span never holds raw PII.** R11 ("redact before content leaves the process") is satisfied
+  by construction — there is no in-memory window where a raw value sits on a span or in the export
+  queue.
+- **One code path to audit.** Because content can only reach a span through the helper, there is a
+  single place to review and test; individual emission sites cannot forget to redact (there is no
+  raw path to `set_attribute` for content keys). A review/lint rule reinforces "content goes
+  through the helper."
+- **Fail-closed.** If the redactor raises, the helper omits the field rather than emitting raw
+  text — consistent with R9 (a tracing failure must never leak or break the request).
+- **Cost.** Redaction runs on the request path (synchronous), not in the export background thread.
+  This is minor. It's a text pass over already-generated output, dwarfed by the LLM call, and only
+  incurred when raw-content capture is enabled (scoped to the three core spans). The helper is the
+  single place to make it async/best-effort if it ever becomes a bottleneck.
+
+### Future: adopting the shared `data-anonymizer` library
+
+A shared, Presidio-based redactor already exists: `data-anonymizer` (used by Ask Red Hat et al.;
+see the [shared Presidio library strategy doc](https://docs.google.com/document/d/1BNBIDUz-lLmUjT9N9cEMqgaQv9EqfLKwZrA8gtbtlY8/edit?tab=t.0#heading=h.9qfphopokr7u)).
+LCORE may move to it eventually, so redaction is maintained once across teams rather than
+per-project. The only obstacle is distribution. `data-anonymizer` lives on Red Hat's internal GitLab
+(`gitlab.cee.redhat.com/uxe-data-ai-solutions/data-anonymizer`), not PyPI, and a GitHub build/CI
+can't install from internal GitLab without internal credentials — so it can't go in
+`pyproject.toml`.
+
+The swap stays small because redaction is already funneled through the single `redact()` helper
+(§ Where redaction runs). To adopt the shared library without breaking the GitHub build, turn that
+helper into a **redaction slot** — one interface the telemetry path calls without knowing the
+backend:
+
+- **GitHub build (default):** the slot keeps using public Presidio from PyPI.
+- **Red Hat's internal build:** a downstream build fills the slot with `data-anonymizer` via an
+  adapter, added at the stage where internal GitLab is reachable.
+
+Until such a move is deemed required, the PyPI Presidio helper above is the whole
+implementation.
+
+### Requirements addendum
+
+- **R11 — PII redaction before export.** Raw content on the three core inference spans shall pass
+  through PII detection/redaction before leaving the process (OTLP export or cross-org sharing).
+
 ## Appendix A: Jira epics and related tracking
 
 **Epics**
 
 - [LCORE-1791](https://redhat.atlassian.net/browse/LCORE-1791)
 - [LCORE-1799](https://redhat.atlassian.net/browse/LCORE-1799)
+- [UIESTRAT-229](https://redhat.atlassian.net/browse/UIESTRAT-229)
 
 ## Appendix B: External references