|
11 | 11 |
|
12 | 12 | <!-- vim-markdown-toc GFM --> |
13 | 13 |
|
14 | | - * [What](#what) |
15 | | - * [Why](#why) |
16 | | - * [Requirements](#requirements) |
| 14 | +* [Overview](#overview) |
| 15 | + * [The problem](#the-problem) |
17 | 16 | * [Use Cases](#use-cases) |
18 | | - * [Architecture](#architecture) |
19 | | - * [Trigger mechanism](#trigger-mechanism) |
20 | | - * [Storage / data model changes](#storage--data-model-changes) |
21 | | -* [References](#references) |
| 17 | + * [Configurable idle TTL enforcement](#configurable-idle-ttl-enforcement) |
| 18 | + * [Active session usability preserved](#active-session-usability-preserved) |
| 19 | + * [Predictable compliance behavior for high-security tenants](#predictable-compliance-behavior-for-high-security-tenants) |
| 20 | + * [Automated deletion across multiple persistence layers](#automated-deletion-across-multiple-persistence-layers) |
| 21 | + * [Stale conversation cleanup during low traffic](#stale-conversation-cleanup-during-low-traffic) |
| 22 | + * [Idempotent handling of concurrent activity](#idempotent-handling-of-concurrent-activity) |
| 23 | + * [Tenant-specific retention profiles](#tenant-specific-retention-profiles) |
| 24 | + * [Programmatic manual deletion still supported](#programmatic-manual-deletion-still-supported) |
| 25 | + * [Deletion when users stop mid-session](#deletion-when-users-stop-mid-session) |
| 26 | + * [Bulk/periodic expiration windows](#bulkperiodic-expiration-windows) |
| 27 | + * [Retention policy changes take effect predictably](#retention-policy-changes-take-effect-predictably) |
| 28 | + * [Audit/logging for compliance evidence*](#auditlogging-for-compliance-evidence) |
| 29 | + * [The recommendation](#the-recommendation) |
| 30 | + * [PoC validation](#poc-validation) |
| 31 | +* [Proposed solution](#proposed-solution) |
| 32 | + * [Basic idea](#basic-idea) |
| 33 | + * [Conversation cleaner variants](#conversation-cleaner-variants) |
| 34 | + * [Solution S1](#solution-s1) |
| 35 | + * [Pros](#pros) |
| 36 | + * [Cons](#cons) |
| 37 | + * [Solution S2](#solution-s2) |
| 38 | + * [Pros](#pros-1) |
| 39 | + * [Cons](#cons-1) |
22 | 40 | * [Changelog](#changelog) |
23 | 41 |
|
24 | 42 | <!-- vim-markdown-toc --> |
25 | 43 |
|
26 | | -## What |
| 44 | +# Overview |
| 45 | + |
| 46 | +This document is the formal deliverable for feature request |
| 47 | +[LCORE-1475](https://issues.redhat.com/browse/LCORE-1475). In its final form it |
| 48 | +presents a detailed design proposal for adding the new end-to-end support to |
| 49 | +clean older conversations in a controlled and consistent manner. The proposal |
| 50 | +covers the key mechanisms needed to identify which conversations are eligible |
| 51 | +for cleaning, how the cleanup process should be recommended or initiated, and |
| 52 | +how the system should ensure safe and predictable behavior across different |
| 53 | +conversation states. In addition, it outlines a proof-of-concept validation |
| 54 | +plan to confirm that the proposed approach works as intended, demonstrates the |
| 55 | +expected outcomes, and provides early evidence for operational feasibility and |
| 56 | +effectiveness of the recommendation and cleanup workflow. |
| 57 | + |
| 58 | +## The problem |
| 59 | + |
| 60 | +Conversation history can already be removed manually via REST API endpoints, |
| 61 | +but customers in high-security environments need stronger, more predictable |
| 62 | +controls over how long conversation data persists. Because conversations by |
| 63 | +nature require some degree of persistence for usability (for example, to |
| 64 | +maintain context across an active session), those customers have accepted that |
| 65 | +conversation data will remain available for a limited time that is fully |
| 66 | +configurable. |
| 67 | + |
| 68 | +To meet their security expectations, they want this persistence to be governed |
| 69 | +by a configurable idle timer or TTL mechanism: when a conversation has not been |
| 70 | +used for a specified period, it should be automatically deleted from every |
| 71 | +applicable persistence layer. This aligns the retention behavior with their |
| 72 | +operational policies, reducing the chance that stale conversation data remains |
| 73 | +accessible longer than intended. |
| 74 | + |
| 75 | +The requirement for configurable, idle-based automatic cleanup was communicated |
| 76 | +to us by our stakeholders in the RHOKP team. |
27 | 77 |
|
28 | | -## Why |
29 | | - |
30 | | -## Requirements |
31 | 78 |
|
32 | 79 | ## Use Cases |
33 | 80 |
|
34 | | -## Architecture |
| 81 | +Several use cases have been identified: |
| 82 | + |
| 83 | +### Configurable idle TTL enforcement |
| 84 | + |
| 85 | +- An admin sets an idle timeout (e.g., 30 minutes) for a tenant. |
| 86 | +- When a conversation is inactive longer than the TTL, the system automatically deletes it from every configured persistence layer. |
| 87 | + |
| 88 | +### Active session usability preserved |
| 89 | + |
| 90 | +- A user keeps interacting within the idle window. |
| 91 | +- The conversation remains available to maintain context, and is deleted only after the user stops for longer than the TTL. |
| 92 | + |
| 93 | +### Predictable compliance behavior for high-security tenants |
| 94 | + |
| 95 | +- A security team defines an operational policy: “No conversation data older than X idle time.” |
| 96 | +- The platform maps that policy to the configurable TTL so deletion timing is deterministic and auditable. |
| 97 | + |
| 98 | +### Automated deletion across multiple persistence layers |
| 99 | + |
| 100 | +- Conversations are stored in more than one place (e.g., conversation store + indexing/cache layer). |
| 101 | +- When idle TTL expires, the system ensures deletion propagates to all applicable layers so no stale copies remain. |
| 102 | + |
| 103 | +### Stale conversation cleanup during low traffic |
| 104 | + |
| 105 | +- TTL-based cleanup runs even when there are few or no API calls. |
| 106 | +- Conversations that expire are deleted automatically without requiring users to trigger deletion. |
| 107 | + |
| 108 | +### Idempotent handling of concurrent activity |
| 109 | + |
| 110 | +- A conversation approaches TTL expiration while a user submits a new message. |
| 111 | +- The system prevents premature deletion by resetting the “last used” timestamp on activity. |
| 112 | + |
| 113 | +### Tenant-specific retention profiles |
| 114 | + |
| 115 | +- Different customers (or environments like staging vs production) require different retention idle timers. |
| 116 | +- The platform enforces TTL per tenant/config profile. |
| 117 | + |
| 118 | +### Programmatic manual deletion still supported |
| 119 | + |
| 120 | +- A customer calls a REST endpoint to delete a conversation immediately. |
| 121 | +- The TTL mechanism remains intact for other conversations; the system handles “manual delete then TTL” safely (no errors, no resurrected data). |
| 122 | + |
| 123 | +### Deletion when users stop mid-session |
| 124 | + |
| 125 | +- A user starts a conversation, stops using it, and leaves. |
| 126 | +- After the TTL expires, the conversation is automatically removed without requiring any explicit user action. |
| 127 | + |
| 128 | +### Bulk/periodic expiration windows |
| 129 | + |
| 130 | +- The system performs background sweeps (e.g., every N minutes). |
| 131 | +- Any conversation whose idle time exceeds TTL by the sweep window is deleted, aligning retention with policy. |
| 132 | + |
| 133 | +### Retention policy changes take effect predictably |
| 134 | + |
| 135 | +- An admin updates TTL configuration (e.g., from 60 minutes to 15 minutes). |
| 136 | +- Existing conversations follow the updated policy according to defined rules (e.g., next evaluation vs immediate recalculation). |
| 137 | + |
| 138 | +### Audit/logging for compliance evidence* |
| 139 | + |
| 140 | +- For each deleted conversation (triggered by idle TTL), the system records metadata useful for audits (e.g., conversation ID, deletion reason=idle TTL, timestamp). |
| 141 | +- This supports demonstrating that stale data wasn’t retained longer than intended. |
| 142 | + |
| 143 | +## The recommendation |
| 144 | + |
| 145 | +This problem can be addressed by introducing a new background runner/service |
| 146 | +responsible for identifying idle conversations and enforcing retention based on |
| 147 | +a configured policy. The runner would periodically scan for conversations that |
| 148 | +have been inactive for longer than the allowed idle window and, for each |
| 149 | +candidate, compare its calculated remaining time-to-live (TTL) against the |
| 150 | +maximum duration defined by configuration. If a conversation’s TTL exceeds the |
| 151 | +allowed duration—meaning it has effectively remained in persistence longer than |
| 152 | +the policy permits—the runner would automatically delete it. |
| 153 | + |
| 154 | +To ensure consistent behavior and reduce duplication of logic, the deletion |
| 155 | +action should reuse the exact implementation used by the existing “delete |
| 156 | +conversation” REST API endpoints (please note that there are two such |
| 157 | +endpoints: for conversation v1 and conversation v2). By leveraging the same |
| 158 | +deletion code path, the solution maintains a single source of truth for cleanup |
| 159 | +semantics (such as what records are removed, what related data is handled, and |
| 160 | +how deletion is validated). |
| 161 | + |
| 162 | +## PoC validation |
| 163 | + |
| 164 | +The proof-of-concept (PoC) implementation is not included in this deliverable |
| 165 | +because the document is focused on proposing and validating the overall design |
| 166 | +and requirements (how idle conversations are identified, how TTL/retention |
| 167 | +policy is enforced, and how deletion should be triggered). Providing a full PoC |
| 168 | +implementation would require additional scope such as wiring the new runner |
| 169 | +into the running services, selecting and configuring persistence-layer queries, |
| 170 | +setting up test environments, and implementing end-to-end integration details |
| 171 | +that are inherently dependent on the target architecture and operational setup. |
| 172 | + |
| 173 | +Instead, this deliverable defines the approach and expected behavior so the PoC |
| 174 | +can be implemented in a follow-up phase once the design is agreed upon. That |
| 175 | +sequencing ensures the PoC effort validates the right thing—confirming that the |
| 176 | +proposed design works—rather than committing to implementation choices before |
| 177 | +the requirements, boundaries, and deletion semantics are finalized. |
| 178 | + |
| 179 | +# Proposed solution |
| 180 | + |
| 181 | +## Basic idea |
| 182 | + |
| 183 | +Lightspeed Core uses mechanism based on so called _runners_. Runners implementation are located under `src/runners`. Currently, two runners are implemented: |
| 184 | + |
| 185 | +1. `uvicorn` |
| 186 | +1. `quota_scheduler` |
| 187 | + |
| 188 | +These runners are started from the main package: `lightspeed_stack.py` |
| 189 | + |
| 190 | +In order to solve the old conversation cleanup mechanism We can implement a new runner called _conversation_cleaner_ that will run in it's own thread and that will need to be configured. This runner will perform the cleanup. |
| 191 | + |
| 192 | +## Conversation cleaner variants |
| 193 | + |
| 194 | +## Solution S1 |
| 195 | + |
| 196 | +Runner will regularly perform the following operations in a loop: |
| 197 | + |
| 198 | +1. select all conversations older than given duration from database tables |
| 199 | +1. call the cleaning deletion code, one conversation by one |
| 200 | + |
| 201 | +### Pros |
| 202 | +### Cons |
| 203 | + |
| 204 | +## Solution S2 |
35 | 205 |
|
36 | | -### Trigger mechanism |
| 206 | +Runner will regularly perform the following operations in a loop: |
37 | 207 |
|
38 | | -### Storage / data model changes |
| 208 | +1. select and delete all conversations older than given duration from database tables |
39 | 209 |
|
40 | | -# References |
| 210 | +### Pros |
| 211 | +### Cons |
41 | 212 |
|
42 | 213 | # Changelog |
43 | 214 |
|
44 | 215 | TODO: Record significant changes after initial creation. |
45 | 216 |
|
46 | | -| Date | Change | Reason | |
47 | | -|------------|-----------------|-----------------| |
48 | | -| 2026-06-29 | Initial version | feature request | |
| 217 | +| Date | Change | Reason | |
| 218 | +|------------|---------------------|-----------------| |
| 219 | +| 2026-06-29 | Initial version | feature request | |
| 220 | +| 2026-07-01 | Problem description | initial spike | |
49 | 221 |
|
0 commit comments