Repository navigation
chore: reconcile main history before workshop promotion - #47
Merged
Merged
Conversation
- Add performance comparison SQL files for all 5 examples - Example 1: Basic MV vs raw table query comparison - Example 2: SummingMergeTree vs Regular MergeTree vs raw table - Example 3: AggregatingMergeTree State/Merge vs raw COUNT DISTINCT - Example 4: dictGet() O(1) lookup vs JOIN comparison - Example 5: Medallion Architecture Gold/Silver/Bronze layer comparison - Fix query_log filtering to show only most recent query per approach - Use row_number() OVER (PARTITION BY ... ORDER BY event_time DESC) - Eliminates duplicate rows from previous test runs - Improve mermaid chart readability in README - Add black font color to all chart nodes - Remove Progression Summary chart - Fix validation query for optional order_anomalies table - Add requirements.txt for Python dependencies 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Fix Example 2: Add FINAL keyword after OPTIMIZE to show merged rows correctly - Fix Example 5: Increase data volume from 5K/7d to 500K/1d for meaningful aggregation demonstration (31,000x reduction ratio vs previous 75x) - Add EVENT_COUNT and EVENT_DAYS to .env for configurable test data volume - Update run_tests.sh to read data generation settings from environment Performance improvements demonstrated: - Daily query: Gold (Day) 4ms vs Bronze 149ms (37x speedup) - Real-time query: Gold scans 3.3K rows vs Silver 21K rows (6.4x fewer) - Data reduction: Bronze->Gold(Day) now shows 31,250x reduction 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Remove 11_null_engine.sql - optional pattern not used in core demo - Remove 32_anomaly_detection.sql - tangential to main webinar concepts - Update 31_gold_hourly.sql next_file reference to point to validation queries 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add Environment Configuration section explaining data volume settings - Update generate_events.py example to reference .env configuration - Document recommended configurations (500K quick, 1M standard, 5M large) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a self-contained demo showcasing all key ClickHouse patterns: - Incremental Materialized Views (MV as INSERT trigger) - AggregatingMergeTree with State/Merge functions (the AVG problem) - Dictionaries for O(1) lookups (dictGet vs JOIN) - Medallion Architecture (Bronze -> Silver -> Gold) Demo features: - Single SQL file with 5 sections (setup + 4 demo sections + cleanup) - 50,000 sample events generated automatically - Business context and talking points for each section - Simplified AVG example (50% error) for easy mental math - Live insert finale showing real-time data flow Files: - incremental_materialized_views/demo/webinar_demo.sql - Main demo script - incremental_materialized_views/demo/README.md - Quick start guide - Updated project structure in main README Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Clean up webinar_demo.sql by removing decorative output messages
that interrupt the query flow. Keep only actual SQL queries that
demonstrate the concepts.
Removed:
- Section header SELECTs like ">>> SECTION X <<<"
- Status/progress SELECTs during setup
- Narrative messages ("wow_moment", "the punchline", etc.)
- Explanatory tables rendered as SELECT output
Kept all actual demo queries with inline SQL comments for context.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Gitignore the telco workshop training documents directory (local-only, not pushed to repo), generated librechat.yaml, .claude config, and scratchpad. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Full-stack telco analytics demo: LibreChat + ClickHouse MCP + LiteLLM + Langfuse, with two deployment modes (local Docker and hybrid cloud). Consolidate tests into test/telco_marketing/ directory. Key components: - Docker Compose orchestration (base + local overlay) - ClickHouse schema with materialized views - LibreChat templates with MCP, model presets, and prompt engineering - LiteLLM proxy config for Gemini models with Langfuse tracing - Python data generator with t-shirt sizing (small to 2xl) - Mode-aware Makefile (setup, start, generate, check, clean) - 90-test suite covering all project files and logic Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…agrams Remove explicit MongoDB, Meilisearch, and PostgreSQL references from user-facing documentation since they are internal dependencies users never interact with. Add Material Design color styling to all mermaid architecture diagrams for better visual clarity. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The test/ directory was committed before the .gitignore rule was added. Remove it from tracking -- local files are preserved. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ClickHouse AI Stack Workshop Commits
Update Quick Start cd paths to include the cloned repo directory. Add *.zip to .gitignore to exclude archive files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Ch ai telco marketing
port is used by clickhouse-client, so default to 9440
clickhouse-client docker image is outdated, just use clickhouse-server
Update Makefile.hybrid.mk
Update ClickHouse port in environment example
update clickhouse-client
update to use http port
Update Makefile.hybrid.mk
Update generator.py
Update docker-compose.yml
Update .env.hybrid.example
Stub approach was overengineered. The @swc/helpers error only fires for heavy Radix UI components. System prompt now enforces Mermaid for charts — simple React artifacts work with the public codesandbox bundler. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…gotiable React artifacts keep being generated despite the instruction being buried in a bullet point. Rule is now the first thing in the system message with explicit wording: broken, always fails, do not attempt. Dashboard requests mapped to multiple mermaid artifacts explicitly. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers all 5 UI sections: Search, Service Map, Chart Explorer, Dashboards, Client Sessions. Each section has step-by-step exercises, discussion questions, and key insights. Ends with a comparison table of all 4 workshop tools (Clickstack / LibreChat / SQL / Langfuse). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Replaces EC2 LibreChat with hosted AgentHouse — Google sign-in, no setup. 6-part flow: 1. Getting started — Google OAuth, orient UI, discover datasets (3 min) 2. Service health — discovery, error investigation, RCA (7 min) 3. Distributed tracing — dependency map, trace deep dive, time compare (5 min) 4. Beyond observability — explore other AgentHouse datasets (5 min) 5. Clickstack — search, service map, chart explorer, dashboards (5 min) 6. Direct SQL — validate AI answers in the raw console (optional, 5 min) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…data generator
- Remove LiteLLM proxy; LibreChat now traces to Langfuse natively via
LANGFUSE_PUBLIC_KEY/SECRET_KEY/BASE_URL.
- Hybrid mode switches presets to Anthropic (Claude Opus/Sonnet 4.6, Haiku 3.5)
with MCP target pinned to a single org/service to skip discovery turns.
- Local mode swaps Gemini presets for Ollama-hosted Qwen3.5:2b via the
OpenAI-compatible API at host.docker.internal:11434.
- Rewrite data-generator/generator.py: column-oriented numpy generation,
Faker pools, sequential string IDs, no pandas. ~6-10x faster on medium,
peak memory still bounded by the 2GB container limit.
- Upgrade clickhouse-connect 0.7.19 -> 1.0.0; raise customer_chunk_size to 500.
- Bump image pins: meilisearch v1.44.0, clickhouse-server 26.3 LTS,
langfuse 3.174.1, postgres 17-alpine. Mongo pinned to 8.0.17 (8.0.23 fails
on Linux kernel 6.19+ per MongoDB SERVER-121912).
- Mount librechat.\${DEPLOY_MODE}.yaml directly so config edits no longer
need a cp step before docker restart.
- Update data-size preset table (CDR estimates reflect the actual
~26 CDRs/customer/day from SEGMENT_USAGE).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
refactor(telco_marketing): drop LiteLLM, native Langfuse, vectorized data generator
Bumps [python-dotenv](https://github.com/theskumar/python-dotenv) from 1.0.1 to 1.2.2. - [Release notes](https://github.com/theskumar/python-dotenv/releases) - [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md) - [Commits](theskumar/python-dotenv@v1.0.1...v1.2.2) --- updated-dependencies: - dependency-name: python-dotenv dependency-version: 1.2.2 dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
Reorder the setup flow so the database is populated before LibreChat and the rest of the stack come online. Data generator depends only on ClickHouse (local: brings up the clickhouse container via service_healthy dep; hybrid: connects to Cloud), so it can run as a one-shot before the LibreChat side of the stack exists. - README: swap the order in both Quick Start blocks (local + hybrid) and both Installation Steps sections. Renumber Step 3/4 (local) and Step 4/5 (hybrid). - Makefile: swap the "Next steps" echoes in setup-local and setup-hybrid, plus the clean-target hint. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Windows users cloning to a path with spaces (e.g. C:\Users\Some User\Documents\repo) hit YAML parsing edge cases when docker compose resolves relative bind-mount sources. Wrap the bind-mount strings in double quotes so the source path is unambiguous. Named volumes (mongodb_data, clickhouse_data, etc.) don't need quoting -- they don't resolve to host paths. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ws-flow follow-up(telco_marketing): reorder setup flow + Windows-safe bind mounts
…local mode Local mode no longer uses Gemini -- it runs qwen3.5:2b via host-side Ollama. Drop the stale "Edit .env to set your GOOGLE_KEY" hint from setup-local's Next steps and replace it with the Ollama install + pull instructions. Renumber the trailing steps. Mirror the change in README Step 2 of the Local Installation Steps section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The repo defaults are now mode-specific (Ollama qwen3.5:2b for local, Anthropic Claude for hybrid). Update the README to match: - LLM Provider Flexibility section: replace single Gemini narrative with a mode-by-mode breakdown; drop the Vertex AI subsection (still listed as a swap option for those who want it). - Architecture diagrams: relabel "Gemini API" -> "Ollama qwen3.5:2b on host :11434" (local) and "Anthropic API Claude 4.6" (hybrid). Pick matching node colors. - Model Configuration: per-mode preset table instead of three Gemini rows; note that yaml is bind-mounted so docker restart is enough. - How Tracing Works: drop "Gemini presets" wording, switch to provider-agnostic "modelSpecs presets" describing both modes. - Creating an Agent walkthrough: model picker is now mode-aware (Ollama/qwen3.5:2b vs Anthropic/claude-sonnet-4-6). - Prerequisites + Quick Start blocks + hybrid Step 2 bullets: replace "LLM API key" placeholders with the actual requirement per mode (Ollama install for local; ANTHROPIC_API_KEY for hybrid). Add CH_ORG_ID / CH_SERVICE_ID to the hybrid checklist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…local mode Local mode no longer uses Gemini -- it runs qwen3.5:2b via host-side Ollama. Drop the stale "Edit .env to set your GOOGLE_KEY" hint from setup-local's Next steps and replace it with the Ollama install + pull instructions. Renumber the trailing steps. Mirror the change in README Step 2 of the Local Installation Steps section. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The repo defaults are now mode-specific (Ollama qwen3.5:2b for local, Anthropic Claude for hybrid). Update the README to match: - LLM Provider Flexibility section: replace single Gemini narrative with a mode-by-mode breakdown; drop the Vertex AI subsection (still listed as a swap option for those who want it). - Architecture diagrams: relabel "Gemini API" -> "Ollama qwen3.5:2b on host :11434" (local) and "Anthropic API Claude 4.6" (hybrid). Pick matching node colors. - Model Configuration: per-mode preset table instead of three Gemini rows; note that yaml is bind-mounted so docker restart is enough. - How Tracing Works: drop "Gemini presets" wording, switch to provider-agnostic "modelSpecs presets" describing both modes. - Creating an Agent walkthrough: model picker is now mode-aware (Ollama/qwen3.5:2b vs Anthropic/claude-sonnet-4-6). - Prerequisites + Quick Start blocks + hybrid Step 2 bullets: replace "LLM API key" placeholders with the actual requirement per mode (Ollama install for local; ANTHROPIC_API_KEY for hybrid). Add CH_ORG_ID / CH_SERVICE_ID to the hybrid checklist. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…a-readme docs(telco_marketing): Ollama setup hint + scrub stale Gemini refs from README
…ws-flow Followup/telco marketing windows flow
LibreChat only substitutes ${VAR} in a small set of fields (apiKey, url,
headers) -- not inside `serverInstructions`. So the CH_ORG_ID and
CH_SERVICE_ID env vars I added earlier were never actually read; users
had to hardcode the same UUIDs in librechat.hybrid.yaml regardless.
Remove the env vars to avoid the false-affordance, and update the README
to point users straight at the YAML edit (Step 6 of the hybrid flow).
- .env.hybrid.example: drop the CH_ORG_ID / CH_SERVICE_ID block, keep
a comment explaining where the UUIDs actually go and why.
- README.md: rewrite the Step 2 ClickHouse Cloud bullet and the
hybrid Quick Start hint to reflect the YAML-direct path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two artifacts for presenting the hybrid setup as a 25-minute workshop: - WORKSHOP_HYBRID_SETUP.md: step-by-step handout. 12 numbered steps from collecting cloud credentials through running demo prompts, plus a troubleshooting table covering every gotcha hit during development (Mongo kernel pin, ENDPOINTS=agents,anthropic, temperature+thinking, missing org/service UUIDs, OAuth loops, paths with spaces). Includes presenter notes section with timing budget. - WORKSHOP_HYBRID_SETUP.html: single-file reveal.js deck (23 slides). Styled to the ClickHouse design system: Inter + JetBrains Mono fonts, electric-yellow (#faff69) primary on near-black canvas (#0a0a0a), surface-card panels with hairline borders, badge-yellow STEP tags, and CTA-band slides at the opener / demo prompts / close. No drop shadows; depth comes from canvas vs surface-card contrast. Speaker notes attached to every slide (press S in reveal.js). Both reflect the simplified flow: org/service UUIDs are pasted directly into librechat.hybrid.yaml's serverInstructions, not into .env. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trim the workshop handout's Troubleshooting table and the matching
slide 21 ("Common gotchas") from the deck. The README already has a
troubleshooting section for attendees who hit issues afterward; the
workshop docs stay focused on the happy path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs(telco_marketing): workshop handout + slide deck; drop CH_ORG_ID/CH_SERVICE_ID env vars
The service-page URL in the ClickHouse Cloud console only contains the service UUID, not the organization UUID: console.clickhouse.cloud/services/<SERVICE_UUID> The earlier instructions claimed both UUIDs could be read from a single URL like /organizations/<ORG>/services/<SVC> -- that path pattern isn't used by the console. Org UUID has to be grabbed separately from the org switcher or the Organization > Settings page. Update all four affected surfaces: .env.hybrid.example, README, the workshop handout, and slide 6 of the deck. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… Settings Point users at exactly the right place: Organization > Settings in the ClickHouse Cloud console, which lands on console.clickhouse.cloud/organizations/<ORG_UUID> Drop the speculative "org switcher in the top-left" reference from the prior commit -- there is no such switcher on the URL, and the trailing "/..." after the UUID was misleading (the path ends at the UUID for the Settings page). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
docs(telco_marketing): correct UUID capture instructions
…lds/telco_marketing/data-generator/python-dotenv-1.2.2 build(deps): bump python-dotenv from 1.0.1 to 1.2.2 in /agent_stack_builds/telco_marketing/data-generator
…low-main ci: bootstrap workshop promotion workflow
# Conflicts: # .gitignore # agent_stack_builds/telco_marketing/data-generator/requirements.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Merges current main into dev so the protected dev-to-main promotion is conflict-free. Conflict resolution keeps the workshop ignore entries and accepts the python-dotenv 1.2.2 Dependabot update from main. No workshop content changes.