LCORE-4606: reject quota limiter periods SQLite cannot use

docs/devel_doc/openapi.json

@@ -19416,7 +19416,7 @@
                     "period"
                 ],
                 "title": "QuotaLimiterConfiguration",
-                "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``."
+                "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased. With the SQLite storage the period must be\n   a modifier of the SQLite function ``datetime()`` that moves the time\n   forward, for example ``7 days``; ``1 week`` is rejected.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``."
             },
             "QuotaSchedulerConfiguration": {
                 "properties": {

docs/devel_doc/openapi.md

@@ -8056,7 +8056,9 @@ There are three configuration options for each limiter:
 1. ``period`` is specified in a human-readable form, see
    https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT
    for all possible options. When the end of the period is reached, the
-   quota is reset or increased.
+   quota is reset or increased. With the SQLite storage the period must be
+   a modifier of the SQLite function ``datetime()`` that moves the time
+   forward, for example ``7 days``; ``1 week`` is rejected.
 2. ``initial_quota`` is the value set at the beginning of the period.
 3. ``quota_increase`` is the value (if specified) used to increase the
    quota when the period is reached.

docs/models/successful_responses.json

@@ -4756,7 +4756,7 @@
             },
             "QuotaLimiterConfiguration": {
                 "additionalProperties": false,
-                "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``.",
+                "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased. With the SQLite storage the period must be\n   a modifier of the SQLite function ``datetime()`` that moves the time\n   forward, for example ``7 days``; ``1 week`` is rejected.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``.",
                 "properties": {
                     "type": {
                         "description": "Quota limiter type, either user_limiter or cluster_limiter",

docs/models/successful_responses.md

@@ -2059,7 +2059,9 @@ There are three configuration options for each limiter:
 1. ``period`` is specified in a human-readable form, see
    https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT
    for all possible options. When the end of the period is reached, the
-   quota is reset or increased.
+   quota is reset or increased. With the SQLite storage the period must be
+   a modifier of the SQLite function ``datetime()`` that moves the time
+   forward, for example ``7 days``; ``1 week`` is rejected.
 2. ``initial_quota`` is the value set at the beginning of the period.
 3. ``quota_increase`` is the value (if specified) used to increase the
    quota when the period is reached.

docs/user_doc/config.html

@@ -1886,7 +1886,10 @@ <h2 id="quotalimiterconfiguration">QuotaLimiterConfiguration</h2>
       <li><code>period</code> is specified in a human-readable form, see
 https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT
 for all possible options. When the end of the period is reached, the
-quota is reset or increased.</li>
+quota is reset or increased. With the SQLite storage the period must be
+a modifier of the SQLite function <code>datetime()</code> that moves the
+time forward, for example <code>7 days</code>; <code>1 week</code> is
+rejected.</li>
       <li><code>initial_quota</code> is the value set at the beginning of the
 period.</li>
       <li><code>quota_increase</code> is the value (if specified) used to

docs/user_doc/config.json

@@ -1645,7 +1645,7 @@
       },
       "QuotaLimiterConfiguration": {
         "additionalProperties": false,
-        "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``.",
+        "description": "Configuration for one quota limiter.\n\nThere are three configuration options for each limiter:\n\n1. ``period`` is specified in a human-readable form, see\n   https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT\n   for all possible options. When the end of the period is reached, the\n   quota is reset or increased. With the SQLite storage the period must be\n   a modifier of the SQLite function ``datetime()`` that moves the time\n   forward, for example ``7 days``; ``1 week`` is rejected.\n2. ``initial_quota`` is the value set at the beginning of the period.\n3. ``quota_increase`` is the value (if specified) used to increase the\n   quota when the period is reached.\n\nThere are two basic use cases:\n\n1. When the quota needs to be reset to a specific value periodically (for\n   example on a weekly or monthly basis), set ``initial_quota`` to the\n   required value.\n2. When the quota needs to be increased by a specific value periodically\n   (for example on a daily basis), set ``quota_increase``.",
         "properties": {
           "type": {
             "description": "Quota limiter type, either user_limiter or cluster_limiter",

docs/user_doc/config.md

@@ -723,7 +723,9 @@ There are three configuration options for each limiter:
 1. ``period`` is specified in a human-readable form, see
    https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT
    for all possible options. When the end of the period is reached, the
-   quota is reset or increased.
+   quota is reset or increased. With the SQLite storage the period must be
+   a modifier of the SQLite function ``datetime()`` that moves the time
+   forward, for example ``7 days``; ``1 week`` is rejected.
 2. ``initial_quota`` is the value set at the beginning of the period.
 3. ``quota_increase`` is the value (if specified) used to increase the
    quota when the period is reached.

src/models/config.py

@@ -2488,7 +2488,9 @@ class QuotaLimiterConfiguration(ConfigurationBase):
     1. ``period`` is specified in a human-readable form, see
        https://www.postgresql.org/docs/current/datatype-datetime.html#DATATYPE-INTERVAL-INPUT
        for all possible options. When the end of the period is reached, the
-       quota is reset or increased.
+       quota is reset or increased. With the SQLite storage the period must be
+       a modifier of the SQLite function ``datetime()`` that moves the time
+       forward, for example ``7 days``; ``1 week`` is rejected.
     2. ``initial_quota`` is the value set at the beginning of the period.
     3. ``quota_increase`` is the value (if specified) used to increase the
        quota when the period is reached.
@@ -2600,6 +2602,37 @@ class QuotaHandlersConfiguration(ConfigurationBase):
         description="Enables storing information about token usage history",
     )
 
+    @model_validator(mode="after")
+    def check_limiter_periods(self) -> Self:
+        """
+        Ensure that the SQLite storage can use the period of every limiter.
+
+        The quota scheduler passes the period to the SQLite function datetime()
+        as a modifier. For a period SQLite can not parse datetime() returns
+        NULL and the quota is never renewed, without any error. Such a period
+        is rejected, and so is a period that does not move the time forward.
+        Nothing is checked when SQLite is not configured: PostgreSQL accepts a
+        wider syntax.
+
+        Returns:
+            Self: The validated configuration instance.
+
+        Raises:
+            ValueError: If SQLite is configured and it can not use the period
+            of a limiter.
+        """
+        if self.sqlite is None:
+            return self
+        for limiter in self.limiters:
+            if not checks.is_valid_sqlite_period(limiter.period):
+                raise ValueError(
+                    f"Quota limiter '{limiter.name}': period '{limiter.period}' "
+                    "can not be used with the SQLite storage. Use one positive "
+                    "number followed by seconds, minutes, hours, days, months "
+                    "or years, for example '7 days'."
+                )
+        return self
+
 
 class RerankerConfiguration(ConfigurationBase):
     """Reranker configuration for RAG chunk reranking."""

src/utils/checks.py

@@ -3,11 +3,16 @@
 import importlib
 import importlib.util
 import os
+import sqlite3
+from contextlib import closing
 from types import ModuleType
 from typing import Optional
 
 from pydantic import FilePath
 
+# fixed point in time the quota limiter periods are tried on
+SQLITE_PERIOD_REFERENCE = "2000-01-01 00:00:00"
+
 
 class InvalidConfigurationError(Exception):
     """Lightspeed configuration is invalid."""
@@ -153,3 +158,32 @@ def is_valid_profile(profile_module: ModuleType) -> bool:
         return False
 
     return isinstance(profile_config.get("system_prompts"), dict)
+
+
+def is_valid_sqlite_period(period: str) -> bool:
+    """
+    Check whether SQLite can use the period to move a timestamp forward.
+
+    The quota scheduler passes the period of a quota limiter to the SQLite
+    function datetime() as a modifier. SQLite itself is asked what it makes of
+    the period, so the check can not drift from what SQLite accepts: datetime()
+    returns NULL for a modifier it can not parse, and a zero or negative period
+    does not give a later time.
+
+    Parameters:
+    ----------
+        period (str): Period as specified in the quota limiter configuration.
+
+    Returns:
+    -------
+        bool: True if datetime() applied to a fixed timestamp with the period
+        as the modifier returns a later time, False otherwise.
+    """
+    # julianday() makes it a comparison of times: as text, the result of the
+    # modifier 'subsec' would be greater than the timestamp it does not move
+    with closing(sqlite3.connect(":memory:")) as connection:
+        (is_later,) = connection.execute(
+            "SELECT julianday(datetime(?, ?)) > julianday(?)",
+            (SQLITE_PERIOD_REFERENCE, period, SQLITE_PERIOD_REFERENCE),
+        ).fetchone()
+    return bool(is_later)

tests/unit/models/config/test_quota_handlers_config.py

@@ -265,7 +265,7 @@ def test_quota_handlers_configuration_improper_values(subtests: SubTests) -> Non
                 "name": "Christina Hall",
                 "initial_quota": 723,
                 "quota_increase": 112,
-                "period": "Media pretty recently push gas.",
+                "period": "1 day",
             }
         ],
         "scheduler": {
@@ -284,14 +284,14 @@ def test_quota_handlers_configuration_improper_values(subtests: SubTests) -> Non
                 "name": "Xavier Anthony",
                 "initial_quota": 536,
                 "quota_increase": 80,
-                "period": "Wrong class strategy.",
+                "period": "30 seconds",
             },
             {
                 "type": "user_limiter",
                 "name": "Caroline Weaver",
                 "initial_quota": 31,
                 "quota_increase": 57,
-                "period": "Lead boy least base.",
+                "period": "2 hours",
             },
         ],
         "scheduler": {
@@ -388,7 +388,7 @@ def test_quota_handlers_configuration_improper_values(subtests: SubTests) -> Non
                 "name": "Bill Boyd",
                 "initial_quota": 582,
                 "quota_increase": 84,
-                "period": "Side no born set. Different weight speak why daugh",
+                "period": "7 days",
             }
         ],
         "scheduler": {
@@ -416,21 +416,21 @@ def test_quota_handlers_configuration_improper_values(subtests: SubTests) -> Non
                 "name": "William Armstrong",
                 "initial_quota": 305,
                 "quota_increase": 648,
-                "period": "Relate couple song way wind rule model.",
+                "period": "5 minutes",
             },
             {
                 "type": "user_limiter",
                 "name": "Terry Mitchell",
                 "initial_quota": 206,
                 "quota_increase": 316,
-                "period": "Onto within arrive type group. Black none human re",
+                "period": "1 month",
             },
             {
                 "type": "user_limiter",
                 "name": "Deborah Vazquez",
                 "initial_quota": 24,
                 "quota_increase": 24,
-                "period": "Call close table.",
+                "period": "1 year",
             },
         ],
         "scheduler": {
@@ -458,21 +458,21 @@ def test_quota_handlers_configuration_improper_values(subtests: SubTests) -> Non
                 "name": "James Burgess",
                 "initial_quota": 698,
                 "quota_increase": 945,
-                "period": "Employee stage activity total.",
+                "period": "3 seconds",
             },
             {
                 "type": "user_limiter",
                 "name": "Brenda Richard",
                 "initial_quota": 583,
                 "quota_increase": 582,
-                "period": "Among or only nearly.",
+                "period": "12 hours",
             },
             {
                 "type": "user_limiter",
                 "name": "Carol Colon",
                 "initial_quota": 26,
                 "quota_increase": 527,
-                "period": "Machine pull finally stock without us him.",
+                "period": "30 days",
             },
         ],
         "scheduler": {
@@ -748,3 +748,82 @@ def test_quota_handlers_from_dict_negative_cases(config_dict: dict[str, Any]) ->
         # try to initialize the app config and load configuration from a Python
         # dictionary
         QuotaHandlersConfiguration(**config_dict)
+
+
+sqlite_storage = {"sqlite": {"db_path": ":memory:"}}
+
+postgres_storage = {"postgres": {"db": "quota", "user": "quota", "password": "secret"}}
+
+no_storage: dict[str, Any] = {}
+
+
+def configuration_with_period(storage: dict[str, Any], period: str) -> dict[str, Any]:
+    """Return quota handlers configuration whose second limiter has the given period."""
+    return {
+        **storage,
+        "limiters": [
+            {
+                "type": "cluster_limiter",
+                "name": "cluster_daily_limits",
+                "initial_quota": 1000,
+                "quota_increase": 0,
+                "period": "1 day",
+            },
+            {
+                "type": "user_limiter",
+                "name": "user_weekly_limits",
+                "initial_quota": 100,
+                "quota_increase": 0,
+                "period": period,
+            },
+        ],
+    }
+
+
+periods_usable_by_sqlite = [
+    "3 seconds",
+    "30 minutes",
+    "2 hours",
+    "7 days",
+    "1 month",
+    "1 year",
+]
+
+
+@pytest.mark.parametrize("period", periods_usable_by_sqlite)
+def test_quota_handlers_sqlite_usable_period(period: str) -> None:
+    """Test that a period SQLite can use is accepted with the SQLite storage."""
+    cfg = QuotaHandlersConfiguration(
+        **configuration_with_period(sqlite_storage, period)
+    )
+    assert cfg.limiters[1].period == period
+
+
+periods_not_usable_by_sqlite = [
+    "1 week",
+    "1 day 2 hours",
+    "0 days",
+    "-1 day",
+    "",
+    "forever",
+]
+
+
+@pytest.mark.parametrize("period", periods_not_usable_by_sqlite)
+def test_quota_handlers_sqlite_unusable_period(period: str) -> None:
+    """Test that a period SQLite can not use is rejected with the SQLite storage."""
+    with pytest.raises(
+        ValidationError,
+        match=f"Quota limiter 'user_weekly_limits': period '{period}' can not be "
+        "used with the SQLite storage.*for example '7 days'",
+    ):
+        QuotaHandlersConfiguration(**configuration_with_period(sqlite_storage, period))
+
+
+@pytest.mark.parametrize("storage", [postgres_storage, no_storage])
+def test_quota_handlers_period_not_checked_without_sqlite(
+    storage: dict[str, Any],
+) -> None:
+    """Test that the period is not checked when the SQLite storage is not configured."""
+    cfg = QuotaHandlersConfiguration(**configuration_with_period(storage, "1 week"))
+    assert cfg.limiters[1].period == "1 week"