Skip to content

Commit bfd4e4f

Browse files
committed
LCORE-2661: Use cases + proposed solution
1 parent 56ae5fd commit bfd4e4f

2 files changed

Lines changed: 190 additions & 35 deletions

File tree

‎docs/design/automatic-conversation-expiration/automatic-conversation-expiration-spike.md‎

Lines changed: 0 additions & 17 deletions
This file was deleted.

‎docs/design/automatic-conversation-expiration/automatic-conversation-expiration.md‎

Lines changed: 190 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -11,39 +11,211 @@
1111

1212
<!-- vim-markdown-toc GFM -->
1313

14-
* [What](#what)
15-
* [Why](#why)
16-
* [Requirements](#requirements)
14+
* [Overview](#overview)
15+
* [The problem](#the-problem)
1716
* [Use Cases](#use-cases)
18-
* [Architecture](#architecture)
19-
* [Trigger mechanism](#trigger-mechanism)
20-
* [Storage / data model changes](#storage--data-model-changes)
21-
* [References](#references)
17+
* [Configurable idle TTL enforcement](#configurable-idle-ttl-enforcement)
18+
* [Active session usability preserved](#active-session-usability-preserved)
19+
* [Predictable compliance behavior for high-security tenants](#predictable-compliance-behavior-for-high-security-tenants)
20+
* [Automated deletion across multiple persistence layers](#automated-deletion-across-multiple-persistence-layers)
21+
* [Stale conversation cleanup during low traffic](#stale-conversation-cleanup-during-low-traffic)
22+
* [Idempotent handling of concurrent activity](#idempotent-handling-of-concurrent-activity)
23+
* [Tenant-specific retention profiles](#tenant-specific-retention-profiles)
24+
* [Programmatic manual deletion still supported](#programmatic-manual-deletion-still-supported)
25+
* [Deletion when users stop mid-session](#deletion-when-users-stop-mid-session)
26+
* [Bulk/periodic expiration windows](#bulkperiodic-expiration-windows)
27+
* [Retention policy changes take effect predictably](#retention-policy-changes-take-effect-predictably)
28+
* [Audit/logging for compliance evidence*](#auditlogging-for-compliance-evidence)
29+
* [The recommendation](#the-recommendation)
30+
* [PoC validation](#poc-validation)
31+
* [Proposed solution](#proposed-solution)
32+
* [Basic idea](#basic-idea)
33+
* [Conversation cleaner variants](#conversation-cleaner-variants)
34+
* [Solution S1](#solution-s1)
35+
* [Pros](#pros)
36+
* [Cons](#cons)
37+
* [Solution S2](#solution-s2)
38+
* [Pros](#pros-1)
39+
* [Cons](#cons-1)
2240
* [Changelog](#changelog)
2341

2442
<!-- vim-markdown-toc -->
2543

26-
## What
44+
# Overview
45+
46+
This document is the formal deliverable for feature request
47+
[LCORE-1475](https://issues.redhat.com/browse/LCORE-1475). In its final form it
48+
presents a detailed design proposal for adding the new end-to-end support to
49+
clean older conversations in a controlled and consistent manner. The proposal
50+
covers the key mechanisms needed to identify which conversations are eligible
51+
for cleaning, how the cleanup process should be recommended or initiated, and
52+
how the system should ensure safe and predictable behavior across different
53+
conversation states. In addition, it outlines a proof-of-concept validation
54+
plan to confirm that the proposed approach works as intended, demonstrates the
55+
expected outcomes, and provides early evidence for operational feasibility and
56+
effectiveness of the recommendation and cleanup workflow.
57+
58+
## The problem
59+
60+
Conversation history can already be removed manually via REST API endpoints,
61+
but customers in high-security environments need stronger, more predictable
62+
controls over how long conversation data persists. Because conversations by
63+
nature require some degree of persistence for usability (for example, to
64+
maintain context across an active session), those customers have accepted that
65+
conversation data will remain available for a limited time that is fully
66+
configurable.
67+
68+
To meet their security expectations, they want this persistence to be governed
69+
by a configurable idle timer or TTL mechanism: when a conversation has not been
70+
used for a specified period, it should be automatically deleted from every
71+
applicable persistence layer. This aligns the retention behavior with their
72+
operational policies, reducing the chance that stale conversation data remains
73+
accessible longer than intended.
74+
75+
The requirement for configurable, idle-based automatic cleanup was communicated
76+
to us by our stakeholders in the RHOKP team.
2777

28-
## Why
29-
30-
## Requirements
3178

3279
## Use Cases
3380

34-
## Architecture
81+
Several use cases have been identified:
82+
83+
### Configurable idle TTL enforcement
84+
85+
- An admin sets an idle timeout (e.g., 30 minutes) for a tenant.
86+
- When a conversation is inactive longer than the TTL, the system automatically deletes it from every configured persistence layer.
87+
88+
### Active session usability preserved
89+
90+
- A user keeps interacting within the idle window.
91+
- The conversation remains available to maintain context, and is deleted only after the user stops for longer than the TTL.
92+
93+
### Predictable compliance behavior for high-security tenants
94+
95+
- A security team defines an operational policy: “No conversation data older than X idle time.”
96+
- The platform maps that policy to the configurable TTL so deletion timing is deterministic and auditable.
97+
98+
### Automated deletion across multiple persistence layers
99+
100+
- Conversations are stored in more than one place (e.g., conversation store + indexing/cache layer).
101+
- When idle TTL expires, the system ensures deletion propagates to all applicable layers so no stale copies remain.
102+
103+
### Stale conversation cleanup during low traffic
104+
105+
- TTL-based cleanup runs even when there are few or no API calls.
106+
- Conversations that expire are deleted automatically without requiring users to trigger deletion.
107+
108+
### Idempotent handling of concurrent activity
109+
110+
- A conversation approaches TTL expiration while a user submits a new message.
111+
- The system prevents premature deletion by resetting the “last used” timestamp on activity.
112+
113+
### Tenant-specific retention profiles
114+
115+
- Different customers (or environments like staging vs production) require different retention idle timers.
116+
- The platform enforces TTL per tenant/config profile.
117+
118+
### Programmatic manual deletion still supported
119+
120+
- A customer calls a REST endpoint to delete a conversation immediately.
121+
- The TTL mechanism remains intact for other conversations; the system handles “manual delete then TTL” safely (no errors, no resurrected data).
122+
123+
### Deletion when users stop mid-session
124+
125+
- A user starts a conversation, stops using it, and leaves.
126+
- After the TTL expires, the conversation is automatically removed without requiring any explicit user action.
127+
128+
### Bulk/periodic expiration windows
129+
130+
- The system performs background sweeps (e.g., every N minutes).
131+
- Any conversation whose idle time exceeds TTL by the sweep window is deleted, aligning retention with policy.
132+
133+
### Retention policy changes take effect predictably
134+
135+
- An admin updates TTL configuration (e.g., from 60 minutes to 15 minutes).
136+
- Existing conversations follow the updated policy according to defined rules (e.g., next evaluation vs immediate recalculation).
137+
138+
### Audit/logging for compliance evidence*
139+
140+
- For each deleted conversation (triggered by idle TTL), the system records metadata useful for audits (e.g., conversation ID, deletion reason=idle TTL, timestamp).
141+
- This supports demonstrating that stale data wasn’t retained longer than intended.
142+
143+
## The recommendation
144+
145+
This problem can be addressed by introducing a new background runner/service
146+
responsible for identifying idle conversations and enforcing retention based on
147+
a configured policy. The runner would periodically scan for conversations that
148+
have been inactive for longer than the allowed idle window and, for each
149+
candidate, compare its calculated remaining time-to-live (TTL) against the
150+
maximum duration defined by configuration. If a conversation’s TTL exceeds the
151+
allowed duration—meaning it has effectively remained in persistence longer than
152+
the policy permits—the runner would automatically delete it.
153+
154+
To ensure consistent behavior and reduce duplication of logic, the deletion
155+
action should reuse the exact implementation used by the existing “delete
156+
conversation” REST API endpoints (please note that there are two such
157+
endpoints: for conversation v1 and conversation v2). By leveraging the same
158+
deletion code path, the solution maintains a single source of truth for cleanup
159+
semantics (such as what records are removed, what related data is handled, and
160+
how deletion is validated).
161+
162+
## PoC validation
163+
164+
The proof-of-concept (PoC) implementation is not included in this deliverable
165+
because the document is focused on proposing and validating the overall design
166+
and requirements (how idle conversations are identified, how TTL/retention
167+
policy is enforced, and how deletion should be triggered). Providing a full PoC
168+
implementation would require additional scope such as wiring the new runner
169+
into the running services, selecting and configuring persistence-layer queries,
170+
setting up test environments, and implementing end-to-end integration details
171+
that are inherently dependent on the target architecture and operational setup.
172+
173+
Instead, this deliverable defines the approach and expected behavior so the PoC
174+
can be implemented in a follow-up phase once the design is agreed upon. That
175+
sequencing ensures the PoC effort validates the right thing—confirming that the
176+
proposed design works—rather than committing to implementation choices before
177+
the requirements, boundaries, and deletion semantics are finalized.
178+
179+
# Proposed solution
180+
181+
## Basic idea
182+
183+
Lightspeed Core uses mechanism based on so called _runners_. Runners implementation are located under `src/runners`. Currently, two runners are implemented:
184+
185+
1. `uvicorn`
186+
1. `quota_scheduler`
187+
188+
These runners are started from the main package: `lightspeed_stack.py`
189+
190+
In order to solve the old conversation cleanup mechanism We can implement a new runner called _conversation_cleaner_ that will run in it's own thread and that will need to be configured. This runner will perform the cleanup.
191+
192+
## Conversation cleaner variants
193+
194+
## Solution S1
195+
196+
Runner will regularly perform the following operations in a loop:
197+
198+
1. select all conversations older than given duration from database tables
199+
1. call the cleaning deletion code, one conversation by one
200+
201+
### Pros
202+
### Cons
203+
204+
## Solution S2
35205

36-
### Trigger mechanism
206+
Runner will regularly perform the following operations in a loop:
37207

38-
### Storage / data model changes
208+
1. select and delete all conversations older than given duration from database tables
39209

40-
# References
210+
### Pros
211+
### Cons
41212

42213
# Changelog
43214

44215
TODO: Record significant changes after initial creation.
45216

46-
| Date | Change | Reason |
47-
|------------|-----------------|-----------------|
48-
| 2026-06-29 | Initial version | feature request |
217+
| Date | Change | Reason |
218+
|------------|---------------------|-----------------|
219+
| 2026-06-29 | Initial version | feature request |
220+
| 2026-07-01 | Problem description | initial spike |
49221

0 commit comments

Comments
 (0)