This repository was archived by the owner on Jul 27, 2026. It is now read-only.
Repository navigation
feat: add posthog events for install, conversation creation, and automations - #1036
Merged
Merged
Conversation
…n, and prebuilt automation - canvas_install already fires pre-consent via telemetry.ts - Add conversation_created to NPM library telemetry (trackEvent) in use-create-conversation.ts onSuccess - Add prebuilt_automation_enabled event in recommended-automations-launcher.tsx when a catalog automation is successfully launched - Add prebuilt_automation_enabled event in automations-list.tsx and automation-detail.tsx handleToggle when enabling (not disabling) - All consent-gated events route through telemetry.ts to PostHog prod via POSTHOG_PROD_KEY baked in by build:lib Co-authored-by: openhands <openhands@all-hands.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
- Add trackPrebuiltAutomationEnabled to useTracking hook (typed, named, includes commonProperties like all other tracked events) - Remove duplicate conversation_created trackEvent call in use-create-conversation (trackConversationCreated already covers it) - Replace all trackEvent(telemetry.ts) calls in automation routes and recommended-automations-launcher with useTracking hook functions - Add trackPrebuiltAutomationEnabled to useCallback dep array in launcher Co-authored-by: openhands <openhands@all-hands.dev>
System 1 — telemetry.ts (anonymous, library-level):
- canvas_install and canvas_new_session remain in telemetry.ts
- trackEvent() stays as the public library API (lib/index.ts export)
- Rule: never import trackEvent in app routes/components
System 2 — useTracking hook (identified, app-level):
- Wire posthog_client_key from VITE_POSTHOG_CLIENT_KEY env var in
OptionService.getConfig() so PostHogProvider mounts when key is set
- Document VITE_POSTHOG_CLIENT_KEY in .env.sample
- Add 6 typed tracking functions to useTracking:
trackInitialQuerySubmitted, trackUserMessageSent,
trackDownloadVsCodeButtonClicked, trackSettingsSaved,
trackMcpConfigUpdated, trackDownloadTrajectoryButtonClicked
- All include commonProperties (current_url, user_email) automatically
- Migrate all 6 raw usePostHog()/posthog.capture() call sites:
chat-interface.tsx, conversation-card.tsx, settings-form.tsx,
use-save-settings.ts, use-download-conversation.ts
- Rule: never call posthog.capture() directly from components
Consent unification:
- AnalyticsConsentFormModal now calls setTelemetryConsent() after saving
user_consents_to_analytics so both systems honour the same decision
Documentation:
- AGENTS.md: tracking architecture section with boundary rules,
system ownership table, and instructions for adding new events
Co-authored-by: openhands <openhands@all-hands.dev>
PostHog defaults to capturing enabled on init. The previous hook only called opt_out_capturing() when user_consents_to_analytics was explicitly false, leaving a window where events fired while settings were loading (null) or on first visit before the user had made a decision. Changes: - null and false both route to opt_out_capturing() — only an explicit true opts PostHog in - Remove hasSyncedRef: the hook now reacts to every settings change, which correctly handles consent granted in another tab or after the initial load. handleCaptureConsent is idempotent so repeated calls are safe. - Update comment to document the three-state consent model Co-authored-by: openhands <openhands@all-hands.dev>
New test files:
- use-sync-posthog-consent.test.ts (7 tests)
Covers the null/false/true consent states and reactive re-sync
behaviour. Specifically tests the bug fix: null now opts out.
- use-tracking.test.ts (17 tests)
One test per tracking function. Verifies correct event names,
property mapping (snake_case / SCREAMING_SNAKE_CASE), and that
commonProperties (current_url, user_email) are included.
Also covers git_user_email fallback and null user_email.
- analytics-consent-form-modal.test.tsx (5 tests)
Verifies setTelemetryConsent is called with 'granted'/'denied'
matching the form checkbox state on submission.
- use-save-settings.test.ts (4 tests)
Covers trackMcpConfigUpdated: fires on new config, skips on
absent config, skips on same object reference, handles zero counts.
- use-download-conversation.test.ts (3 tests)
Covers trackDownloadTrajectoryButtonClicked: called on download,
called before the API request, still called even when API rejects.
Updated test files:
- option-service.test.ts
Two new cases for posthog_client_key: returns key from env var
when set, returns null when env var is absent.
- chat-interface.test.tsx
Added vi.mock('#/hooks/use-tracking') to prevent test noise.
New 'Tracking' describe block: verifies trackInitialQuerySubmitted
fires on first message and trackUserMessageSent on follow-ups.
- conversation-card.test.tsx
Added vi.mock('#/hooks/use-tracking') guard.
- settings-form.test.tsx
Added vi.mock('#/hooks/use-tracking') guard.
New test: verifies trackSettingsSaved is called with LLM_API_KEY_SET
and SEARCH_API_KEY_SET on form submission.
Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
Author
Tracked events referenceAlways fires — no consent required
Fires only after telemetry consent granted (
|
| Event | When | Key properties |
|---|---|---|
canvas_new_session |
Each new browser session (deduped by sessionStorage) |
is_first_use |
trackEvent(name, props) |
Caller-defined — public npm API | caller-defined |
Fires only after analytics consent granted (user_consents_to_analytics === true)
All events below go through useTracking → posthog.capture(). They are silently dropped if the user has denied or not yet set consent (null now opts PostHog out via useSyncPostHogConsent). Requires VITE_POSTHOG_CLIENT_KEY to be set.
All events include current_url and user_email (falls back to git_user_email, then null).
| Event | Trigger location | Additional properties |
|---|---|---|
conversation_created |
use-create-conversation.ts → mutation onSuccess |
has_repository |
prebuilt_automation_enabled |
automation-detail.tsx, automations-list.tsx, recommended-automations-launcher.tsx |
automation_id, automation_name, automation_category |
initial_query_submitted |
chat-interface.tsx — first message sent (empty conversation) |
entry_point, query_character_length, replay_json_size |
user_message_sent |
chat-interface.tsx — follow-up messages |
session_message_count, current_message_length |
settings_saved |
settings-form.tsx — form submitted |
LLM_MODEL, LLM_API_KEY_SET, SEARCH_API_KEY_SET, REMOTE_RUNTIME_RESOURCE_FACTOR |
mcp_config_updated |
use-save-settings.ts — when MCP config object reference changes |
has_mcp_config: true, sse_servers_count, stdio_servers_count |
download_trajectory_button_clicked |
use-download-conversation.ts |
— |
download_via_vscode_button_clicked |
conversation-card.tsx |
— |
login_button_clicked |
Auth flow | provider |
push_button_clicked |
Git control bar | — |
pull_button_clicked |
Git control bar | — |
create_pr_button_clicked |
Git control bar | — |
user_signup_completed |
Signup flow | signup_timestamp |
Error telemetry — posthog.captureException
trackError() in error-handler.ts passes through to posthog.captureException. No explicit consent check — PostHog's opt-in/out state applies. Call sites: conversation-websocket-context.tsx (agent errors, token limit errors). actions.ts passes posthog: undefined making it a no-op.
use-tracking.test.ts: COMMON.current_url was hardcoded as 'http://localhost/' but jsdom in CI runs at 'http://localhost:3000/'. Change COMMON from a module-level const to a let populated in beforeEach via window.location.href so it always matches the environment. chat-interface.test.tsx: trackUserMessageSent was never called (0 times) because totalEvents is derived from uiEvents via useFilteredEvents, not from the events array. The test was setting events but leaving uiEvents empty, so totalEvents stayed 0 and the component took the trackInitialQuerySubmitted branch instead. Seed uiEvents with a minimal MessageEvent (has llm_message.role + content, which isMessageEvent and shouldRenderAgentServerEvent both accept) so totalEvents = 1. Co-authored-by: openhands <openhands@all-hands.dev>
?? null was split onto its own line but the full expression fits within the 80-char printWidth (79 chars), so prettier requires them joined on one line. Co-authored-by: openhands <openhands@all-hands.dev>
analytics-consent-form-modal.test.tsx: Spread of unknown[] is not valid — TS requires a tuple type or rest param. Replace (...args: unknown[]) => mock(...args) with a typed single-arg forwarder: (consent: string) => mock(consent). use-save-settings.test.ts: MCPConfig requires shttp_servers (added in a later migration). Add shttp_servers: [] to all three fixture objects (full config, sharedConfig reference-equality test, and zero-count test). Co-authored-by: openhands <openhands@all-hands.dev>
vi.fn(() => Promise.resolve()) infers zero parameters, so calling it with a string argument fails TS2554. Add _consent: string to the implementation so the inferred mock signature matches the call. Co-authored-by: openhands <openhands@all-hands.dev>
All tracking functions now require posthog to be initialized AND
user_consents_to_analytics === true before calling posthog.capture().
Previously, events were blocked only via PostHog's opt-out mechanism
(set by useSyncPostHogConsent), leaving three gaps:
1. posthog undefined (no VITE_POSTHOG_CLIENT_KEY) - silent no-op
2. Desync between useSyncPostHogConsent and useTracking can cause
events to fire or be dropped depending on hook ordering
3. No self-documenting enforcement of the documented requirement
that System 2 requires user_consents_to_analytics === true
The new internal track() helper short-circuits when either condition
is unmet, so each tracking function is self-contained and the consent
contract is explicit in the code that enforces it.
Tests: beforeEach now includes user_consents_to_analytics: true; posthog
mock is made variable so the undefined case can be tested; new 'consent
gate' describe block covers all blocked cases (false, null, undefined
settings, no posthog).
Co-authored-by: openhands <openhands@all-hands.dev>
Calls posthog.identify(userId, { email }) for cloud users who have
granted analytics consent. Required because PostHogProvider is
initialized with person_profiles='identified_only', which silently
drops all events until identify() has been called.
Identity lifecycle:
- consent === true + userId present → posthog.identify()
- consent === false (explicit denial) → posthog.reset()
- userId becomes null after identify (logout) → posthog.reset()
- consent === null / settings loading → no-op
userId sourced from useCloudCurrentUserId() (cloud mode only).
Local mode is skipped — no stable server-issued user ID available.
Co-authored-by: openhands <openhands@all-hands.dev>
HeyItsChloe
marked this pull request as ready for review
June 3, 2026 19:55
Contributor
❌ Mock-LLM Docker E2E Test Results16/22 passed · 2 failed · 4 skipped Commit:
🔍 Failure details (2)❌ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance❌ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routesPosted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
✅ Mock-LLM E2E Tests22/22 passed Commit:
Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
✅ Mock-LLM E2E Tests22/22 passed Commit:
Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
❌ Mock-LLM Docker E2E Test Results16/22 passed · 2 failed · 4 skipped Commit:
🔍 Failure details (2)❌ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance❌ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routesPosted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
|
🚀 Released in v1.2.0. |
Contributor
❌ Mock-LLM Docker E2E Test Results16/22 passed · 2 failed · 4 skipped Commit:
🔍 Failure details (2)❌ mock-llm-cross-connect.spec.ts › cross-connect: frontend-only → backend-only › frontend-only connects to a separate backend-only instance❌ mock-llm-partial-stack.spec.ts › partial stack: --frontend-only › serves the frontend but returns 503 for backend routesPosted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
✅ Mock-LLM E2E Tests22/22 passed Commit:
Posted by the Mock-LLM E2E workflow · results are deterministic (scripted LLM responses) |
Contributor
📸 Snapshot Test ReportWarning Snapshot comparison step crashed (timeout, OOM, or runner error) — diff results below may be incomplete or absent. ✅ No baseline found on main — all 73 snapshots are new and will become the baseline once this PR merges.
🆕 New snapshots (73)These snapshots have no baseline on main and will become the new baseline once this PR merges.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The canvas app had no structured app-level PostHog tracking. Raw
usePostHog()calls were scattered across components with string event names, no shared properties, and no unified consent gate. A gap also existed whereuser_consents_to_analytics === null(settings still loading) left PostHog in its default capturing-enabled state, so events could fire before the user had made a consent decision.Summary
telemetry.tshandles anonymous library-level telemetry (npm consumers, pre-consent);useTrackinghook handles identified app-level analytics (requiresuser_consents_to_analytics === true)useTrackingand migrates all 5 rawusePostHog().capture()call sites to use themuseSyncPostHogConsentnow treatsnullas opted-out (safe default while settings load), not as a pass-throughIssue Number
APP-2060
How to Test
cp .env.sample .envand setVITE_POSTHOG_CLIENT_KEYto a PostHog project keynpm run devconversation_createdwhen starting a new conversationinitial_query_submittedfiresuser_message_sentfiressettings_savedfireslocalStorage, reload — confirmcanvas_installfires in the PostHog dashboard before the consent modal is interacted with, and that no app-level events appearIf
VITE_POSTHOG_CLIENT_KEYis not set the app runs normally with no tracking — this is the default.Video/Screenshots
Type
Notes
VITE_POSTHOG_CLIENT_KEYis not required — omitting it silently disables System 2. Document it in.env.sample.telemetry.ts) useslocalStorage; System 2 uses the backenduser_consents_to_analyticssetting. The consent modal now syncssetTelemetryConsentas a convenience when running the full app, but library consumers using only the npm package are unaffected.build:libsetsVITE_APP_ENV=productionat compile time so the npm-published bundle always uses the production PostHog key. App Docker builds use the staging key unless the release workflow setsVITE_APP_ENV=production.AGENTS.md.🐳 Docker images for this PR
• GHCR package: https://github.com/OpenHands/agent-canvas/pkgs/container/agent-canvas
ghcr.io/openhands/agent-canvasghcr.io/openhands/agent-server:1.24.0-pythonopenhands-automation==1.0.0a60c5629d57865ca518e57886b0ab8e258103b7ebcPull (multi-arch manifest)
# Multi-arch manifest — Docker automatically pulls the correct architecture docker pull ghcr.io/openhands/agent-canvas:sha-0c5629dRun
All tags pushed for this build
About Multi-Architecture Support
sha-0c5629d) is a multi-arch manifest supporting both amd64 and arm64sha-0c5629d-amd64) are also available if needed