Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
781dec3
fix: [makefile] use spaces in GO_TEST_PATH shell expansion
crandles Apr 19, 2026
cd889cb
feat: [influxdb] add v3 http api support (sql + influxql, jsonl/csv)
crandles Apr 19, 2026
9c1cfda
feat: [influxdb] add flight sql proxy with metadata, prepared stateme…
crandles Apr 19, 2026
bc11e90
fix: [influxdb/flux] cache queries with now() and nested paren in ran…
crandles Apr 19, 2026
90324a0
test: [integration] add influxdb v1/v3 http + flight sql sdk coverage
crandles Apr 19, 2026
f5b49c5
docs+dev: [influxdb] document v3 + flight sql; add dev env service, d…
crandles Apr 19, 2026
13ef3a8
docs: [influxdb] mark v3 done on roadmap, document flight_port, updat…
crandles Apr 20, 2026
5c80188
fix: [influxdb/sql] coerceValue panic on non-string value in String c…
crandles Apr 23, 2026
13d3a29
fix: [influxdb/sql] SetExtent uses base time column, not SELECT alias
crandles Apr 23, 2026
5c8797c
fix: [influxdb] v3 POST JSON+form bodies engage caching path
crandles Apr 23, 2026
17bf723
fix: [influxdb/flight] wire cache and scope keys per tenant
crandles Apr 23, 2026
6bde7e7
Merge branch 'main' into influx-3
crandles May 7, 2026
c805631
Merge branch 'main' into influx-3
jranson Aug 28, 2026
91a9856
linter nits
jranson Aug 28, 2026
5ee0094
migrate influx3 sql to cockroachdb parser
jranson Aug 28, 2026
5ebd7a4
bifurcate vitess parser from mysql backend
jranson Aug 28, 2026
4325a22
Merge branch 'main' into influx-3
jranson Aug 28, 2026
a6bb12d
linter
jranson Aug 28, 2026
3c35cd7
fix integration tests
jranson Aug 28, 2026
febc0fe
Merge branch 'main' into influx-3
jranson Aug 29, 2026
67c5c63
move FlightUpstreamAddress to InfluxDB-specific options
jranson Aug 29, 2026
d0108f2
support tls and fix defects
jranson Aug 29, 2026
f45bc1b
correctness fixes
jranson Aug 29, 2026
d869305
use 'flight-sql' for protocol id
jranson Aug 29, 2026
db952b6
bifurcate arrow flight sql from influxdb
jranson Aug 29, 2026
3f0e223
add arrow batch <-> DataSet interop
jranson Aug 29, 2026
ed7d049
add native delta (derived from mysql/cache.go)
jranson Aug 29, 2026
d996273
migrate mysql to native delta
jranson Aug 29, 2026
ec1a449
implement nativedelta on flight-sql
jranson Aug 29, 2026
a4a1589
support SchemaRPCs and parameterless Prepared Statements
jranson Aug 29, 2026
fd8fb8e
add docs, bench and metrics
jranson Aug 29, 2026
9244618
fix hanging integration test
jranson Aug 29, 2026
7286949
address flight sql defects/completeness
jranson Aug 29, 2026
df8ece1
fix test func scope
jranson Aug 29, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -171,6 +171,14 @@ benchmark-mysql:
@go test ./pkg/backends/mysql -run '^$$' -bench '^BenchmarkMySQL' -benchmem -count=5
@go test ./pkg/backends/alb/mech/ur -run '^$$' -bench '^BenchmarkResolveRouteProtocolNeutral$$' -benchmem -count=5

.PHONY: benchmark-flightsql-smoke
benchmark-flightsql-smoke:
@go test ./pkg/proxy/flightsql -run '^$$' -bench '^BenchmarkFlightSQLSmoke$$' -benchtime=1x -benchmem

.PHONY: benchmark-flightsql
benchmark-flightsql:
@go test ./pkg/proxy/flightsql -run '^$$' -bench '^BenchmarkFlightSQL' -benchmem -count=5

.PHONY: benchmark-graphite
benchmark-graphite:
@go test ./pkg/backends/graphite/resolution -run '^$$' -bench '^BenchmarkResolver' \
Expand All @@ -192,13 +200,14 @@ GO_TEST_FLAGS ?= -coverprofile=.coverprofile
.PHONY: test
test: check-license-headers check-codegen gotest check-fmtprints check-todos

GO_TEST_PATH ?= $(shell $(GO) list ./... | grep -v v2/integration)
GO_TEST_PATH ?= $(shell $(GO) list ./... | grep -v v2/integration | tr '\n' ' ')
.PHONY: gotest
gotest:
$(GO) test -timeout=5m -v ${GO_TEST_FLAGS} $(GO_TEST_PATH)
@./hack/filter-coverprofile.sh .coverprofile
@echo
@./hack/coverprofile-summary.sh
@echo "All tests passed successfully."

.PHONY: data-race-test
data-race-test:
Expand Down
23 changes: 21 additions & 2 deletions docs/developer/environment/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,26 @@ expected and is that phase's acceptance criterion — do not "fix" it by editing
the dashboard.** The metric names and label values those panels query are
frozen in todo item 3.4 and must be implemented unchanged in Phase 9.

## ClickHouse
## Included backends

The Compose file brings up Prometheus, InfluxDB 2.x, InfluxDB 3.x and ClickHouse
alongside Grafana. Trickster's dev config registers a matching backend for each,
so Grafana can query the upstream directly or via Trickster for a side-by-side
comparison.

## InfluxDB Details

For InfluxDB 3.x, Trickster also exposes an Apache Arrow Flight SQL (gRPC)
proxy on `:8485` (the `influx3-flight` listener with `protocol: flight-sql`,
mapped from the influx3 backend's `listener_names`). The
`Trickster InfluxDB 3` Grafana dashboard exercises both the HTTP InfluxQL
endpoint and Flight SQL via the InfluxDB datasource in SQL mode, with direct
and Trickster-cached targets on the same panel. An equivalent
`ClickHouse (Grafana Plugin)` dashboard is included for ClickHouse. The
`verify-influxdb3.sh` script runs a quick end-to-end check of the v3 HTTP
and Flight SQL paths through Trickster.

## ClickHouse Details

The environment runs ClickHouse on direct HTTP port `8123` and Native port
`9000`. Trickster's single `click1` backend uses the HTTP origin and is bound to
Expand Down Expand Up @@ -175,7 +194,7 @@ Native clients can use either origin transport. See the
[ClickHouse Support Guide](../../clickhouse.md) for TLS configuration,
delta-cacheable SQL, supported formats and types, and Native limitations.

## MySQL
## MySQL Details

The developer environment includes a pinned MySQL 8.4 (LTS) container seeded
with the same auto-phased NYC taxi `trips` dataset used by ClickHouse. The
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,266 @@
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": { "type": "grafana", "uid": "-- Grafana --" },
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"description": "Exercises both HTTP InfluxQL (cached via /query) and Flight SQL (cached via gRPC passthrough). HTTP SQL (/api/v3/query_sql) is tested via curl — Grafana does not speak HTTP for SQL mode.",
"editable": true,
"fiscalYearStartMonth": 0,
"graphTooltip": 0,
"links": [],
"panels": [
{
"type": "row",
"title": "$measurement",
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 0 },
"panels": []
},
{
"title": "HTTP InfluxQL — ${measurement} row count (Direct vs Trickster)",
"type": "timeseries",
"datasource": { "type": "datasource", "uid": "-- Mixed --" },
"gridPos": { "h": 10, "w": 12, "x": 0, "y": 1 },
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line", "lineWidth": 2, "fillOpacity": 10,
"pointSize": 5, "showPoints": "auto", "spanNulls": false,
"stacking": { "group": "A", "mode": "none" }
},
"unit": "short"
},
"overrides": [
{
"matcher": { "id": "byFrameRefID", "options": "Direct" },
"properties": [
{ "id": "displayName", "value": "Direct (InfluxDB 3)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "red" } }
]
},
{
"matcher": { "id": "byFrameRefID", "options": "Trickster" },
"properties": [
{ "id": "displayName", "value": "Trickster (cached)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "blue" } },
{ "id": "custom.lineStyle", "value": { "fill": "dash", "dash": [10, 10] } },
{ "id": "custom.lineWidth", "value": 3 }
]
}
]
},
"options": {
"legend": { "displayMode": "table", "placement": "bottom", "calcs": ["lastNotNull", "mean"] }
},
"targets": [
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_direct_ql" },
"query": "SELECT count(*) FROM \"${measurement}\" WHERE $timeFilter GROUP BY time($__interval) fill(0)",
"rawQuery": true, "resultFormat": "time_series", "refId": "Direct"
},
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_ql" },
"query": "SELECT count(*) FROM \"${measurement}\" WHERE $timeFilter GROUP BY time($__interval) fill(0)",
"rawQuery": true, "resultFormat": "time_series", "refId": "Trickster"
}
]
},
{
"title": "Flight SQL — ${measurement} row count (Direct vs Trickster)",
"type": "timeseries",
"datasource": { "type": "datasource", "uid": "-- Mixed --" },
"gridPos": { "h": 10, "w": 12, "x": 12, "y": 1 },
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line", "lineWidth": 2, "fillOpacity": 10,
"pointSize": 5, "showPoints": "auto", "spanNulls": false,
"stacking": { "group": "A", "mode": "none" }
},
"unit": "short"
},
"overrides": [
{
"matcher": { "id": "byFrameRefID", "options": "Direct" },
"properties": [
{ "id": "displayName", "value": "Direct (InfluxDB 3)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "red" } }
]
},
{
"matcher": { "id": "byFrameRefID", "options": "Trickster" },
"properties": [
{ "id": "displayName", "value": "Trickster (cached)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "blue" } },
{ "id": "custom.lineStyle", "value": { "fill": "dash", "dash": [10, 10] } },
{ "id": "custom.lineWidth", "value": 3 }
]
}
]
},
"options": {
"legend": { "displayMode": "table", "placement": "bottom", "calcs": ["lastNotNull", "mean"] }
},
"targets": [
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_direct_flight" },
"rawSql": "SELECT date_bin(INTERVAL '${__interval_ms} milliseconds', time) AS time, count(*) AS count FROM ${measurement} WHERE time >= $__timeFrom AND time < $__timeTo GROUP BY 1 ORDER BY 1",
"resultFormat": "time_series", "refId": "Direct"
},
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"rawSql": "SELECT date_bin(INTERVAL '${__interval_ms} milliseconds', time) AS time, count(*) AS count FROM ${measurement} WHERE time >= $__timeFrom AND time < $__timeTo GROUP BY 1 ORDER BY 1",
"resultFormat": "time_series", "refId": "Trickster"
}
]
},
{
"title": "${measurement}.${field} over time (Direct vs Trickster)",
"type": "timeseries",
"datasource": { "type": "datasource", "uid": "-- Mixed --" },
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 11 },
"fieldConfig": {
"defaults": {
"color": { "mode": "palette-classic" },
"custom": {
"drawStyle": "line", "lineWidth": 2, "fillOpacity": 10,
"pointSize": 5, "showPoints": "auto", "spanNulls": false,
"stacking": { "group": "A", "mode": "none" }
}
},
"overrides": [
{
"matcher": { "id": "byFrameRefID", "options": "Direct" },
"properties": [
{ "id": "displayName", "value": "Direct (InfluxDB 3)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "red" } }
]
},
{
"matcher": { "id": "byFrameRefID", "options": "Trickster" },
"properties": [
{ "id": "displayName", "value": "Trickster (cached)" },
{ "id": "color", "value": { "mode": "fixed", "fixedColor": "blue" } },
{ "id": "custom.lineStyle", "value": { "fill": "dash", "dash": [10, 10] } },
{ "id": "custom.lineWidth", "value": 3 }
]
}
]
},
"options": {
"legend": { "displayMode": "table", "placement": "bottom", "calcs": ["lastNotNull", "mean"] }
},
"targets": [
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_direct_flight" },
"rawSql": "SELECT date_bin(INTERVAL '${__interval_ms} milliseconds', time) AS time, avg(\"${field}\") AS \"${field}\" FROM ${measurement} WHERE time >= $__timeFrom AND time < $__timeTo GROUP BY 1 ORDER BY 1",
"resultFormat": "time_series", "refId": "Direct"
},
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"rawSql": "SELECT date_bin(INTERVAL '${__interval_ms} milliseconds', time) AS time, avg(\"${field}\") AS \"${field}\" FROM ${measurement} WHERE time >= $__timeFrom AND time < $__timeTo GROUP BY 1 ORDER BY 1",
"resultFormat": "time_series", "refId": "Trickster"
}
]
},
{
"title": "Flight SQL — Latest ${measurement} rows (via Trickster)",
"type": "table",
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 21 },
"description": "Latest 20 rows of the selected measurement. Exercises a non-aggregated Flight SQL query through Trickster's cache.",
"fieldConfig": {
"defaults": { "custom": { "align": "left" } },
"overrides": []
},
"options": { "showHeader": true, "cellHeight": "sm" },
"targets": [
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"rawSql": "SELECT * FROM (SELECT * FROM ${measurement} WHERE time >= $__timeFrom AND time < $__timeTo ORDER BY time DESC LIMIT 20) ORDER BY time ASC",
"resultFormat": "table",
"refId": "Latest"
}
]
},
{
"type": "row",
"title": "metadata",
"collapsed": false,
"gridPos": { "h": 1, "w": 24, "x": 0, "y": 31 },
"panels": []
},
{
"title": "Flight SQL — Tables in trickster DB (information_schema)",
"type": "table",
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"gridPos": { "h": 10, "w": 24, "x": 0, "y": 32 },
"description": "Exercises InfluxDB 3's information_schema via Trickster's Flight SQL proxy. Proves the Flight SQL query path end-to-end with a non-time-series result set.",
"fieldConfig": {
"defaults": { "custom": { "align": "left" } },
"overrides": [
{
"matcher": { "id": "byName", "options": "time" },
"properties": [{ "id": "custom.hidden", "value": true }]
}
]
},
"options": {
"showHeader": true,
"cellHeight": "sm"
},
"targets": [
{
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"rawSql": "SELECT now() AS time, table_catalog, table_schema, table_name, table_type FROM information_schema.tables WHERE table_schema IN ('iox', 'system') ORDER BY table_schema, table_name",
"resultFormat": "table",
"refId": "Tables"
}
]
}
],
"schemaVersion": 39,
"tags": ["influxdb", "influxdb3", "trickster"],
"templating": {
"list": [
{
"name": "measurement",
"label": "Measurement (via Flight SQL)",
"type": "query",
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"query": "SELECT DISTINCT table_name FROM information_schema.tables WHERE table_schema = 'iox' ORDER BY table_name",
"refresh": 1,
"multi": false,
"includeAll": false,
"current": { "value": "cpu", "text": "cpu" }
},
{
"name": "field",
"label": "Field (numeric columns of selected measurement)",
"type": "query",
"datasource": { "type": "influxdb", "uid": "ds_infl3_trk_flight" },
"query": "SELECT column_name FROM information_schema.columns WHERE table_schema = 'iox' AND table_name = '${measurement}' AND data_type IN ('Float64', 'Int64', 'UInt64') ORDER BY column_name",
"refresh": 1,
"multi": false,
"includeAll": false
}
]
},
"time": { "from": "now-1h", "to": "now" },
"timepicker": {},
"timezone": "",
"title": "Trickster - InfluxDB 3.x",
"uid": "trickster-influxdb3",
"version": 4
}
Loading
Loading