Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 1 addition & 7 deletions deploy/kube/configmap.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -710,13 +710,7 @@ data:
# # (X-Trickster-Result: phit) rather than truncating the merged response silently. default is 268435456 (256 MiB)
# max_capture_bytes: 268435456

# # These next 7 settings only apply to Time Series backends

# # dpc_fallback_warning, when true (default), logs a warning when a query cannot be parsed
# # as a time range query and falls back from DPC (Delta Proxy Cache) to OPC (Object Proxy Cache).
# # Set to false to suppress these warnings (they will still appear at debug level).
# # Applies to SQL-based TSDB backends like ClickHouse that may receive non-time-series queries.
# dpc_fallback_warning: true
# # These next 6 settings only apply to Time Series backends

# # backfill_tolerance prevents new datapoints that fall within the tolerance window (relative to time.Now) from being permanently
# # cached. Think of it as "the newest N milliseconds of real-time data are preliminary and subject to updates, so refresh them periodically"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1031,7 +1031,7 @@
]
},
"time": {
"from": "now-7d",
"from": "now-2d",
"to": "now"
},
"timepicker": {},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1035,7 +1035,7 @@
]
},
"time": {
"from": "now-7d",
"from": "now-2d",
"to": "now"
},
"timepicker": {},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -925,7 +925,7 @@
]
},
"time": {
"from": "now-7d",
"from": "now-2d",
"to": "now"
},
"timepicker": {},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -1141,7 +1141,7 @@
]
},
"time": {
"from": "now-7d",
"from": "now-2d",
"to": "now"
},
"timepicker": {},
Expand Down
19 changes: 9 additions & 10 deletions docs/new-changed-2.1.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,18 +5,19 @@ Trickster 2.1 includes a number of new features to give it even more uses in a n
## Features

* **Kubernetes Gateway API and Ingress Controller** - Trickster now runs as a Kubernetes controller, serving `gateway.networking.k8s.io` GatewayClass, Gateway, HTTPRoute, GRPCRoute, TCPRoute, TLSRoute, UDPRoute, ReferenceGrant and BackendTLSPolicy objects, and `networking.k8s.io/v1` Ingress objects, translating them into its own configuration and reloading onto it in-process. Routes may be served through the Service's cluster IP or load balanced across discovered endpoints, TLS certificates arrive from Kubernetes Secrets without a reload, and status, conditions and Events are written back to every claimed object. Enable it with the new top-level `kubernetes` section. See [kubernetes-gateway.md](./kubernetes-gateway.md), [kubernetes-ingress.md](./kubernetes-ingress.md), [kubernetes-rbac.md](./kubernetes-rbac.md) and [kubernetes-deploy.md](./kubernetes-deploy.md).

* **TricksterCachePolicy** - A custom resource that attaches caching behavior — a cache, TTLs, cache key components, CORS, header updates and time series acceleration — to Gateways, HTTPRoutes, Ingresses and Services, so a Prometheus or ClickHouse Service behind a route is accelerated rather than merely proxied. See [kubernetes-cache-policy.md](./kubernetes-cache-policy.md).
* **TricksterCachePolicy Custom Resource** - A custom resource that attaches caching behavior — a cache, TTLs, cache key components, CORS, header updates and time series acceleration — to Gateways, HTTPRoutes, Ingresses and Services, so a Prometheus or ClickHouse Service behind a route is accelerated rather than merely proxied. See [kubernetes-cache-policy.md](./kubernetes-cache-policy.md).

* Auto-discovery - the ALB can now manage pool members through common [auto-discovery](./alb-autodiscovery.md) mechanisms such as Kubernetes APIs, DNS A and SRV records, etc.

* Config Management - We now support loading multiple config files in the same subdirectory below the main config. We've also added automatic config reloading when the file contents change - including automatic detection and reloading when a certificate is swapped out.
* Config - We now support loading multiple config files in the same subdirectory below the main config. We've also added automatic config reloading when the file contents change - including automatic detection and reloading when a TLS certificate is swapped out. See the [Configuring Documentation](./configuring.md#automatic-config-reload) for more info.

* Logging - We've added support for customizable access logging and error logging per backend in NCSA format, a top-level `access_log` that captures every request no backend handled, and logging to stdout or stderr for containerized deployments. See [access-logs.md](./access-logs.md).

* Regex Path Matching - You can now define path routes with regexes to match incoming requests, and expose their capture groups to request rewriters.
* Regex Path Matching - You can now [define path routes with regexes](./paths.md#regex-paths) to match incoming requests, and expose their capture groups to request rewriters.

* Wildcard Host Routing - A backend's `hosts` may name a single-label wildcard (`*.example.com`) or an any-depth wildcard (`**.example.com`), resolved by specificity ahead of global routes. See [Path Configuration Documentation](./paths.md#host-resolution).

* Wildcard Host Routing - A backend's `hosts` may name a single-label wildcard (`*.example.com`) or an any-depth wildcard (`**.example.com`), resolved by specificity ahead of global routes. See [paths.md](./paths.md).
* We now support accelerating [InfluxDB 3.x](./influxdb.md#influxdb-3x-support), including over Flight SQL (gRPC). Support for InfluxDB 1.x and 2.x remains and is unchanged.

* We now support accelerating [Graphite](./graphite.md)

Expand All @@ -42,10 +43,8 @@ Trickster 2.1 includes a number of new features to give it even more uses in a n

* **Real Client Addresses** - Listeners now accept the PROXY protocol, and the new `trusted_proxies` listener option resolves the real client address from `Forwarded`, `X-Forwarded-For` or `X-Real-IP` only when the connection comes from a trusted address. The resolved address is what the access log records and what `max_query_range` rejections are logged against. See [Trusted Proxies](./configuring.md#trusted-proxies).

## Breaking Changes

* **Reserved object-name prefixes** - Object names beginning with a reserved prefix are now refused in every named configuration section (`backends`, `caches`, `listeners`, `discovery`, `rules`, `request_rewriters`, `negative_caches`, `tracing` and `authenticators`), because Trickster generates configuration under them at runtime. Currently, the only reserved prefix is `kgw--`, used by the Kubernetes controller. A configuration that defines such a name fails to load. See [Reserved Names](./configuring.md#reserved-names).
## Developer Environment

## Installing
* All of the new supported backend time series providers are included in the Developer Env Docker Compose

You can build the 2.1 binary from the `main` branch, download binaries from the [Releases](http://github.com/trickstercache/trickster/releases) page, or use the `trickstercache/trickster` Docker image tag in containerized environments. Kubernetes deployments can start from the manifests in [deploy/kube](../deploy/kube) or the chart at <https://github.com/trickstercache/helm-charts>.
* We've refactored the Developer Environment Data Seeder job to use 100% locally-generated data. No more big S3 file downloads. This also speeds up the GitHub CI/CD integration tests that depend on the dev env. Seed data is generated once as a TSV, and all seedable databases (MySQL, Druid, ClickHouse) mount the same file to load the same data to enable cross-provider testing and verification.
8 changes: 1 addition & 7 deletions examples/conf/example.full.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -699,13 +699,7 @@ backends:
# # (X-Trickster-Result: phit) rather than truncating the merged response silently. default is 268435456 (256 MiB)
# max_capture_bytes: 268435456

# # These next 7 settings only apply to Time Series backends

# # dpc_fallback_warning, when true (default), logs a warning when a query cannot be parsed
# # as a time range query and falls back from DPC (Delta Proxy Cache) to OPC (Object Proxy Cache).
# # Set to false to suppress these warnings (they will still appear at debug level).
# # Applies to SQL-based TSDB backends like ClickHouse that may receive non-time-series queries.
# dpc_fallback_warning: true
# # These next 6 settings only apply to Time Series backends

# # backfill_tolerance prevents new datapoints that fall within the tolerance window (relative to time.Now) from being permanently
# # cached. Think of it as "the newest N milliseconds of real-time data are preliminary and subject to updates, so refresh them periodically"
Expand Down
9 changes: 0 additions & 9 deletions pkg/backends/clickhouse/observability.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,6 @@ func (c *Client) observeAnalysis(analysis sqlanalyzer.Analysis) {
}
mode := analysis.Mode.String()
metrics.SQLQueryAnalysis.WithLabelValues(backendName, clickHouseDialect, mode, reason).Inc()
if c != nil && c.TimeseriesBackend != nil && analysis.Mode == sqlanalyzer.CacheModeObject &&
analysis.Reason != sqlanalyzer.ReasonUnsupportedBucket && analysis.Reason != sqlanalyzer.ReasonNotTimeRange {
options := c.Configuration()
if options != nil && (options.DPCFallbackWarning == nil || *options.DPCFallbackWarning) {
logger.Warn("query fell back from DPC to OPC", logging.Pairs{
keys.BackendName: backendName, keys.Dialect: clickHouseDialect, keys.Reason: reason,
})
}
}
if logger.Level() == level.Debug {
logger.Debug("sql query cache eligibility analyzed", logging.Pairs{
keys.BackendName: backendName,
Expand Down
3 changes: 2 additions & 1 deletion pkg/backends/clickhouse/parsing_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,8 @@ func TestUnsafeAndUnsupportedQueriesUseObjectCache(t *testing.T) {
}{
{"or predicate", `SELECT toStartOfMinute(ts) AS t, count() FROM events WHERE ts >= 100 OR ts < 200 GROUP BY t`, sqlanalyzer.ReasonUnsafePredicate},
{"not predicate", `SELECT toStartOfMinute(ts) AS t, count() FROM events WHERE NOT (ts >= 100) GROUP BY t`, sqlanalyzer.ReasonUnsafePredicate},
{"not time series", `SELECT count() FROM events WHERE ts >= 100 GROUP BY service`, sqlanalyzer.ReasonUnsupportedBucket},
{"not time series", `SELECT count() FROM events WHERE ts >= 100 GROUP BY service`, sqlanalyzer.ReasonNotTimeRange},
{"metadata probe", `SELECT displayName(), version(), revision(), timezone()`, sqlanalyzer.ReasonNotTimeRange},
{"limit", `SELECT toStartOfMinute(ts) AS t, count() FROM events WHERE ts >= 100 GROUP BY t LIMIT 10`, sqlanalyzer.ReasonUnsupportedLimit},
{"unsupported interval", `SELECT toStartOfInterval(ts, INTERVAL 1 year) AS t, count() FROM events WHERE ts >= 100 AND ts < 200 GROUP BY t FORMAT JSON`, sqlanalyzer.ReasonUnsupportedBucket},
{"unsupported week mode", `SELECT toStartOfWeek(ts, 1) AS t, count() FROM events WHERE ts >= 100 AND ts < 200 GROUP BY t`, sqlanalyzer.ReasonUnsupportedBucket},
Expand Down
1 change: 1 addition & 0 deletions pkg/backends/druid/observability.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ const (
reasonSQLEligible analysisReason = "sql_eligible"
reasonSQLUnsupported analysisReason = "sql_unsupported"
reasonSQLUnsupportedFormat analysisReason = "sql_unsupported_format"
reasonSQLUnrenderable analysisReason = "sql_unrenderable"
reasonSQLInvalidQuery analysisReason = "sql_invalid_query"
)

Expand Down
Loading
Loading