Skip to content

Commit 28a37c2

Browse files
author
Agent
committed
feat(cli): report anonymous install telemetry
After a successful pgflow install, send one strict allowlisted contribution with the fixed package version, fresh/update/no-op result, and bucketed copied-migration count. CI, tests, DO_NOT_TRACK, and PGFLOW_TELEMETRY_DISABLED suppress the event; request failures never affect installation. Reuse the existing telemetry worker and retention policy, extend its allowlist and tests, and expand the single telemetry reference page with an upfront no-identifiers guarantee, exact CLI payload, and both CLI and database opt-out controls. Raw migration timestamps, filenames, paths, errors, and project values never leave the machine.
1 parent 56bef34 commit 28a37c2

10 files changed

Lines changed: 234 additions & 21 deletions

File tree

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
"pgflow": patch
3+
---
4+
5+
Report anonymous fresh, update, or no-op install completion telemetry with the pgflow version and a bucketed migration count. CI, tests, `DO_NOT_TRACK`, and `PGFLOW_TELEMETRY_DISABLED` disable the event.

‎apps/telemetry-worker/src/index.test.ts‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,6 +50,24 @@ describe('telemetry ingest', () => {
5050
expect(points).toHaveLength(3);
5151
});
5252

53+
it.each([
54+
'cli_install_fresh',
55+
'cli_install_update',
56+
'cli_install_noop',
57+
])('accepts the %s event with version and migration-count bucket', async (metric) => {
58+
const { env, points } = makeEnv();
59+
const res = await post({
60+
schema: 1,
61+
contributions: [{ metric, bucket: '0.18.0', count: '4-7' }],
62+
}, env);
63+
expect(res.status).toBe(204);
64+
expect(points).toEqual([{
65+
indexes: [metric],
66+
blobs: ['0.18.0', '4-7'],
67+
doubles: [1],
68+
}]);
69+
});
70+
5371
it('stores the count bucket as a second blob', async () => {
5472
const { env, points } = makeEnv();
5573
await post(valid, env);

‎apps/telemetry-worker/src/index.ts‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ type BucketKind =
2929
| 'yes' | 'semver';
3030

3131
const METRICS: Record<string, BucketKind> = {
32+
cli_install_fresh: 'semver',
33+
cli_install_update: 'semver',
34+
cli_install_noop: 'semver',
3235
active_db_day: 'yes',
3336
workers_by_version: 'semver',
3437
version_changed: 'yes',
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
import { describe, expect, it, vi } from 'vitest';
2+
import { reportInstallTelemetry } from '../../../src/commands/install/report-install-telemetry';
3+
4+
const result = { kind: 'update' as const, copied: 5 };
5+
6+
describe('reportInstallTelemetry', () => {
7+
it.each([
8+
['fresh', 12, '8-15'],
9+
['update', 5, '4-7'],
10+
['noop', 0, '0'],
11+
] as const)('reports a successful %s install', async (kind, copied, count) => {
12+
const send = vi.fn().mockResolvedValue(new Response(null, { status: 204 }));
13+
14+
await reportInstallTelemetry(
15+
{ kind, copied },
16+
{ env: {}, version: '0.18.0', send },
17+
);
18+
19+
expect(send).toHaveBeenCalledOnce();
20+
const [url, request] = send.mock.calls[0] as [string, RequestInit];
21+
expect(url).toBe('https://pgflow-telemetry.workers.dev/');
22+
expect(request.method).toBe('POST');
23+
expect(JSON.parse(request.body as string)).toEqual({
24+
schema: 1,
25+
contributions: [{
26+
metric: `cli_install_${kind}`,
27+
bucket: '0.18.0',
28+
count,
29+
}],
30+
});
31+
});
32+
33+
it.each([
34+
{ CI: 'true' },
35+
{ NODE_ENV: 'test' },
36+
{ DO_NOT_TRACK: '1' },
37+
{ PGFLOW_TELEMETRY_DISABLED: 'true' },
38+
])('does not report when disabled by $env', async (env) => {
39+
const send = vi.fn();
40+
await reportInstallTelemetry(result, { env, version: '0.18.0', send });
41+
expect(send).not.toHaveBeenCalled();
42+
});
43+
44+
it('does not report an invalid version', async () => {
45+
const send = vi.fn();
46+
await reportInstallTelemetry(result, { env: {}, version: 'unknown', send });
47+
expect(send).not.toHaveBeenCalled();
48+
});
49+
50+
it('never fails installation when the request fails', async () => {
51+
const send = vi.fn().mockRejectedValue(new Error('offline'));
52+
await expect(
53+
reportInstallTelemetry(result, { env: {}, version: '0.18.0', send }),
54+
).resolves.toBeUndefined();
55+
});
56+
});

‎pkgs/cli/scripts/test-install‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#!/usr/bin/env bash
22
set -e
3+
export PGFLOW_TELEMETRY_DISABLED=1
34

45
# Script to test pgflow CLI install functionality
56
# Uses a temp directory to avoid conflicts with other tests

‎pkgs/cli/scripts/test-install-duplicates‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
#!/usr/bin/env bash
22
set -e
3+
export PGFLOW_TELEMETRY_DISABLED=1
34

45
# Script to test pgflow CLI install duplicate prevention
56
# Uses a temp directory to avoid conflicts with other tests

‎pkgs/cli/src/commands/install/copy-migrations.ts‎

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -155,13 +155,18 @@ function generateNewTimestamp(
155155
// Find the migrations directory
156156
const sourcePath = findMigrationsDirectory();
157157

158+
export type MigrationInstallResult = {
159+
kind: 'fresh' | 'update' | 'noop';
160+
copied: number;
161+
};
162+
158163
export async function copyMigrations({
159164
supabasePath,
160165
autoConfirm = false,
161166
}: {
162167
supabasePath: string;
163168
autoConfirm?: boolean;
164-
}): Promise<boolean> {
169+
}): Promise<MigrationInstallResult | null> {
165170
const migrationsPath = path.join(supabasePath, 'migrations');
166171

167172
if (!fs.existsSync(migrationsPath)) {
@@ -180,7 +185,7 @@ export async function copyMigrations({
180185
log.info(
181186
'If running in development mode, try building the core package first with: nx build core'
182187
);
183-
return false;
188+
return null;
184189
}
185190

186191
// Get all existing migrations in user's directory
@@ -231,10 +236,10 @@ export async function copyMigrations({
231236
}
232237
}
233238

234-
// If no files to copy, show message and return false (no changes made)
239+
// If no files need copying, this is a successful no-op install.
235240
if (filesToCopy.length === 0) {
236241
log.success('Migrations already up to date');
237-
return false;
242+
return { kind: 'noop', copied: 0 };
238243
}
239244

240245
// Generate new timestamps for migrations to install
@@ -267,7 +272,7 @@ export async function copyMigrations({
267272

268273
if (confirmResult !== true) {
269274
log.warn('Migration installation skipped');
270-
return false;
275+
return null;
271276
}
272277
}
273278

@@ -281,5 +286,8 @@ export async function copyMigrations({
281286

282287
log.success(`Installed ${filesToCopy.length} migration${filesToCopy.length !== 1 ? 's' : ''}`);
283288

284-
return true; // Return true to indicate migrations were copied
289+
return {
290+
kind: skippedFiles.length === 0 ? 'fresh' : 'update',
291+
copied: filesToCopy.length,
292+
};
285293
}

‎pkgs/cli/src/commands/install/index.ts‎

Lines changed: 13 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@ import { updateConfigToml } from './update-config-toml.js';
66
import { createFlowsDirectory } from './create-flows-directory.js';
77
import { createExampleWorker } from './create-example-worker.js';
88
import { supabasePathPrompt } from './supabase-path-prompt.js';
9+
import { reportInstallTelemetry } from './report-install-telemetry.js';
910

1011
export default (program: Command) => {
1112
program
@@ -38,6 +39,10 @@ export default (program: Command) => {
3839
` • Create ${chalk.cyan('supabase/functions/greet-user-worker/')} ${chalk.dim('(example worker)')}`,
3940
'',
4041
` ${chalk.green('✓ Safe to re-run - completed steps will be skipped')}`,
42+
'',
43+
chalk.dim(
44+
'Anonymous telemetry (no identifiers or project values). Opt out: PGFLOW_TELEMETRY_DISABLED=1 · pgflow.dev/reference/telemetry'
45+
),
4146
].join('\n');
4247

4348
log.info(summaryMsg);
@@ -85,8 +90,10 @@ export default (program: Command) => {
8590

8691
// Step 4: Show completion message
8792
const outroMessages: string[] = [];
93+
const migrationsChanged =
94+
migrations?.kind === 'fresh' || migrations?.kind === 'update';
8895

89-
if (migrations || configUpdate || flowsDirectory || exampleWorker) {
96+
if (migrationsChanged || configUpdate || flowsDirectory || exampleWorker) {
9097
outroMessages.push(chalk.green.bold('✓ Installation complete!'));
9198
} else {
9299
outroMessages.push(
@@ -107,7 +114,7 @@ export default (program: Command) => {
107114
stepNumber++;
108115
}
109116

110-
if (migrations) {
117+
if (migrationsChanged) {
111118
outroMessages.push(
112119
` ${stepNumber}. Apply migrations: ${chalk.cyan('supabase migrations up')}`
113120
);
@@ -119,5 +126,9 @@ export default (program: Command) => {
119126
);
120127

121128
outro(outroMessages.join('\n'));
129+
130+
if (migrations) {
131+
await reportInstallTelemetry(migrations);
132+
}
122133
});
123134
};
Lines changed: 67 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,67 @@
1+
import { getVersion } from '../../utils/get-version.js';
2+
import type { MigrationInstallResult } from './copy-migrations.js';
3+
4+
const ENDPOINT = 'https://pgflow-telemetry.workers.dev/';
5+
const SEMVER_RE = /^\d+\.\d+\.\d+(-[0-9A-Za-z.-]+)?$/;
6+
7+
type Environment = Record<string, string | undefined>;
8+
type Send = typeof globalThis.fetch;
9+
10+
const isSet = (value: string | undefined) => {
11+
const normalized = value?.toLowerCase();
12+
return normalized !== undefined
13+
&& normalized !== ''
14+
&& normalized !== '0'
15+
&& normalized !== 'false';
16+
};
17+
18+
function telemetryDisabled(env: Environment): boolean {
19+
return env.NODE_ENV === 'test'
20+
|| isSet(env.CI)
21+
|| isSet(env.DO_NOT_TRACK)
22+
|| isSet(env.PGFLOW_TELEMETRY_DISABLED);
23+
}
24+
25+
function bucketCount(value: number): string {
26+
if (value === 0) return '0';
27+
if (value === 1) return '1';
28+
if (value <= 3) return '2-3';
29+
if (value <= 7) return '4-7';
30+
if (value <= 15) return '8-15';
31+
if (value <= 31) return '16-31';
32+
if (value <= 63) return '32-63';
33+
if (value <= 127) return '64-127';
34+
if (value <= 255) return '128-255';
35+
return '256+';
36+
}
37+
38+
export async function reportInstallTelemetry(
39+
result: MigrationInstallResult,
40+
{
41+
env = process.env,
42+
version = getVersion(),
43+
send = globalThis.fetch,
44+
}: { env?: Environment; version?: string; send?: Send } = {},
45+
): Promise<void> {
46+
if (telemetryDisabled(env) || version.length > 32 || !SEMVER_RE.test(version)) {
47+
return;
48+
}
49+
50+
try {
51+
await send(ENDPOINT, {
52+
method: 'POST',
53+
headers: { 'content-type': 'application/json' },
54+
body: JSON.stringify({
55+
schema: 1,
56+
contributions: [{
57+
metric: `cli_install_${result.kind}`,
58+
bucket: version,
59+
count: bucketCount(result.copied),
60+
}],
61+
}),
62+
signal: AbortSignal.timeout(500),
63+
});
64+
} catch {
65+
// Telemetry must never delay or fail installation.
66+
}
67+
}

‎pkgs/website/src/content/docs/reference/telemetry.mdx‎

Lines changed: 56 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -3,17 +3,48 @@ title: Telemetry
33
description: What anonymous usage data pgflow collects, how to inspect it, and how to disable it
44
---
55

6-
pgflow includes anonymous, opt-out telemetry. It answers one question for the maintainers: **is pgflow actually used, and which features matter?** Your data stays yours — nothing below can identify you, your project, or your users.
6+
pgflow includes anonymous, opt-out telemetry. It answers one question for the maintainers: **is pgflow actually used, and which features matter?**
7+
8+
**pgflow never sends or stores information that can identify you, your project, or your users.** The only variable exact value from your installation that is stored as telemetry is a pgflow semantic version such as `0.18.0`. Everything else is a fixed label defined in the source code or a coarse bucket. No exact usage count, duration, source timestamp, migration filename, or other exact project value leaves your machine or database.
9+
10+
Two narrowly scoped signals are sent:
11+
12+
- After `pgflow install` completes successfully outside CI and tests, the CLI sends one installation-result event.
13+
- Once per active day, a non-local database sends one aggregate usage report. Nothing is sent per run, step, or task, so telemetry never touches the execution path.
714

815
## What is collected
916

10-
Once per day, your database aggregates yesterday's activity into coarse buckets and sends one small JSON document. Nothing is sent per run, per step, or per task — telemetry never touches the execution path.
17+
### Installation command
1118

12-
The payload contains only:
19+
A successful `pgflow install` sends only:
20+
21+
- The exact pgflow CLI version
22+
- A fixed result label: `fresh`, `update`, or `noop`
23+
- The number of copied pgflow migrations as a coarse bucket (`0`, `1`, `2-3`, `4-7`, and so on)
24+
25+
`fresh` means that no recognized pgflow migration existed, `update` means that older pgflow migrations existed and new ones were copied, and `noop` means that all bundled pgflow migrations were already present. Raw migration timestamps, filenames, project migration counts, paths, and installation errors are never sent. The event looks like this:
26+
27+
```json
28+
{
29+
"schema": 1,
30+
"contributions": [
31+
{
32+
"metric": "cli_install_update",
33+
"bucket": "0.18.0",
34+
"count": "4-7"
35+
}
36+
]
37+
}
38+
```
39+
40+
### Daily database report
41+
42+
Once per day, your database aggregates yesterday's activity into coarse buckets and sends one small JSON document. The payload contains only:
1343

1444
- Whether any runs happened that day
1545
- Which pgflow versions started workers, and whether a version changed
1646
- Coarse counts: runs started, run outcomes, workers started/stopped, registered worker functions
47+
- Registered worker start modes (`http` or `process`)
1748
- Flow shape in buckets: steps per flow, which features appear (map steps, conditions, retries, delays, graceful failure, step queues)
1849
- Coarse durations and task counts in fixed ranges
1950

@@ -39,11 +70,13 @@ A payload never exceeds 64 contributions or 2 KB. On an unusually varied day the
3970

4071
## What is never collected
4172

42-
No slugs, flow or step names, queue names, function names, run or task IDs, inputs, outputs, error messages, URLs, hostnames, environment names, IP addresses, or any free text. No cookies. No account. Nothing that could identify you, your project, or your users.
73+
pgflow collects no slugs, flow or step names, queue names, function names, migration names or timestamps, run or task IDs, inputs, outputs, error messages, URLs, file paths, hostnames, environment names, operating-system details, IP addresses, or free text. It creates no account, persistent installation ID, machine ID, project ID, or cookie.
74+
75+
The telemetry application cannot connect two reports to the same installation. It cannot count unique users or reconstruct one project's history.
4376

4477
## Inspect exactly what is sent
4578

46-
Every payload is stored before it leaves, so you can audit the real bytes:
79+
The installation event shape is shown above and is fixed in the open-source CLI. Every daily database payload is stored before it leaves, so you can audit its exact bytes:
4780

4881
```sql
4982
select jsonb_pretty(payload)
@@ -61,11 +94,21 @@ select pgflow_telemetry.preview('2026-09-10'); -- any past day
6194

6295
## Disable or re-enable
6396

64-
Telemetry is opt-out and runs as a daily `pg_cron` job. The job's presence is the switch:
97+
Telemetry is opt-out. The CLI shows a disclosure before installation and does not ask for permission.
98+
99+
Disable the installation event for one command:
100+
101+
```bash frame="none"
102+
PGFLOW_TELEMETRY_DISABLED=1 npx pgflow@latest install
103+
```
104+
105+
The CLI also honors the conventional `DO_NOT_TRACK=1` environment variable. It never sends installation telemetry in CI or tests. Set either variable in your shell profile to disable CLI telemetry permanently on that machine.
106+
107+
Daily database telemetry uses a `pg_cron` job. The job's presence is the switch:
65108

66109
```sql
67-
select pgflow_telemetry.disable(); -- stop reporting
68-
select pgflow_telemetry.enable(); -- resume reporting
110+
select pgflow_telemetry.disable(); -- stop daily database reports
111+
select pgflow_telemetry.enable(); -- resume daily database reports
69112
```
70113

71114
`disable()` errors instead of reporting success when it cannot prove the job absent — for example when the job was scheduled by a different database role and `cron.job` row security hides it from yours. In that case run `disable()` as the role that installed pgflow (the migration or `enable()` caller). On a vanilla PostgreSQL install where pgflow's migration owner is neither superuser, `BYPASSRLS`, nor a role that inherits the privileges of the `cron.job` owner, the same row security also hides the job from the privileged check itself; the error then names the one-line fix, reassigning `pgflow_telemetry.job_is_scheduled()` to a capable owner (on Supabase the migration owner `postgres` already qualifies, so this never applies there). Calling `disable()` when no job exists is a safe no-op.
@@ -74,9 +117,9 @@ Local development databases (`supabase start`) never send telemetry, and days wi
74117

75118
## Privacy
76119

77-
- Data goes to `https://pgflow-telemetry.workers.dev`, a [Cloudflare Worker](https://workers.cloudflare.com/) backed by [Workers Analytics Engine](https://developers.cloudflare.com/analytics/analytics-engine/).
78-
- On the ingest side, stored data is retained for three months, then expires automatically.
120+
- Both signals go to `https://pgflow-telemetry.workers.dev`, a [Cloudflare Worker](https://workers.cloudflare.com/) backed by [Workers Analytics Engine](https://developers.cloudflare.com/analytics/analytics-engine/).
121+
- Stored data expires automatically after three months.
79122
- The `sent_reports` audit table in your database is separate: its 90-day prune runs only as part of a successful daily report. Disabled databases and databases with inactive days keep their audit rows until you delete them yourself.
80-
- pgflow sends no identifiers and the ingest application stores no transport metadata (no IPs, no user agents). Cloudflare necessarily processes source IPs as network transport.
81-
- Counts are undercounts in practice: opt-outs, blocked egress, and failed sends are invisible. The maintainers treat every number as a lower bound and as directional product evidence only.
82-
- The sender is the open-source SQL in this repository; the receiver is the open-source Worker in `apps/telemetry-worker/`.
123+
- pgflow sends no identifiers and the telemetry application stores no transport metadata, including IP addresses or user agents. Cloudflare necessarily processes source IPs to carry the network request.
124+
- Counts are undercounts in practice: opt-outs, CI exclusion, blocked egress, and failed sends are invisible. The maintainers treat every number as a lower bound and as directional product evidence only.
125+
- Both senders and the receiver are open source: the CLI is in `pkgs/cli/`, the daily database sender is in `pkgs/core/schemas/`, and the receiver is in `apps/telemetry-worker/`.

0 commit comments

Comments
 (0)