Skip to content

feat(showcase-ecommerce): add context docs for talk-to-data and operational scenarios#195

Merged
shirshanka merged 1 commit into
mainfrom
feat/showcase-ecommerce-context-docs
Apr 22, 2026
Merged

feat(showcase-ecommerce): add context docs for talk-to-data and operational scenarios#195
shirshanka merged 1 commit into
mainfrom
feat/showcase-ecommerce-context-docs

Conversation

@shirshanka

Copy link
Copy Markdown
Contributor

Summary

Adds 03-context.json to the showcase-ecommerce datapack — 18 Document entities organized in a browsable folder hierarchy, each linked to their source datasets via relatedAssets so the content is indexed alongside datasets for semantic search.

Document tree (54 MCPs):

  • E-Commerce Data Catalog (root, linked to all 5 data products)
    • Core Transactional Tables — orders, order_items, customers, products, inventories, promotions, addresses
    • Fulfillment & Reference Data — warehouses, product_categories, regions/countries
    • Analytics Layer — order_details (grain clarification, when to use vs source tables), order_history (snapshot semantics)
    • Operational Runbooks — order count discrepancy, inventory staleness, promotion attribution
    • Key Metrics Reference — GMV, AOV, CLV, return rate, fill rate, promotion attachment rate

Each document covers: table purpose, column semantics with enum value definitions, known gotchas, common query patterns, and join maps.

Designed for three scenarios:

  1. Talk to Data — metric definitions, column disambiguation, join paths so AI can answer business questions
  2. Operational Debugging — runbooks for the most common data issues
  3. Data Development — layer architecture, canonical table guidance, price/status hierarchies

Test plan

  • Generated and validated 54 MCPs locally (0 errors)
  • Loaded all 3 datapack files into a local DataHub instance (3879 total events, 0 failures)
  • Verified documents appear in sidebar (showInGlobalContext: true on all 18 docs)
  • Verify documents surface in semantic search alongside linked datasets

🤖 Generated with Claude Code

@vercel

vercel Bot commented Apr 21, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
datahub-docs-archive Ignored Ignored Apr 21, 2026 3:58pm

Request Review

…tional scenarios

Adds 03-context.json — 18 Document entities organized in a browsable
folder hierarchy, linked to their source datasets via relatedAssets so
content is indexed alongside datasets for semantic search.

Document tree:
- E-Commerce Data Catalog (root)
  - Core Transactional Tables (orders, order_items, customers, products,
    inventories, promotions, addresses)
  - Fulfillment & Reference Data (warehouses, product_categories, regions)
  - Analytics Layer (order_details, order_history guides)
  - Operational Runbooks (order count discrepancy, inventory staleness,
    promotion attribution)
  - Key Metrics Reference (GMV, AOV, CLV, return rate, fill rate)

Each document covers: purpose, column semantics, enum value definitions,
common query patterns, gotchas, and join maps — supporting talk-to-data,
operational debugging, and data development scenarios.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
@shirshanka shirshanka force-pushed the feat/showcase-ecommerce-context-docs branch from e12d169 to b319f79 Compare April 21, 2026 15:58
@shirshanka shirshanka merged commit fa3ae4c into main Apr 22, 2026
3 checks passed
@shirshanka shirshanka deleted the feat/showcase-ecommerce-context-docs branch April 22, 2026 04:01
nwadams pushed a commit that referenced this pull request Jun 8, 2026
…chor demo (#198)

Adds a corpus of urn:li:query MCPs to back the anchor-generation pipeline
discussed in the talk-to-data demo work that #195 set up the context docs
for. Until now the showcase-ecommerce datapack had no query entities, so
the anchor pipeline had no SQL to fingerprint and rank.

What's added:
- 04-queries.json — 1,246 MCPs spanning 312 urn:li:query entities:
  * 160 "cluster" queries across 50 distinct analyst intents
    (3–5 variants per intent: raw vs. analytics.order_details mart, CTE
    vs. subquery, alternate join paths, a few intentional anti-shapes)
  * 152 "noise" queries — one-off analyst drill-downs against specific
    customer/order/product/promotion IDs, date-window investigations,
    DQ checks, schema exploration, abandoned half-finished queries
  * Each query carries queryProperties, querySubjects (auto-detected
    dataset URNs), dataPlatformInstance (snowflake/b2fd91), and subTypes
  * created/lastModified timestamps spread across the data window so the
    corpus looks like a real activity history
  * authors mapped to the 11 corpuser entities already in 02-data.json
    (Sarah Chen, Andrea Garcia, David Kim, Ben Porter, Julia Novak,
    James Wilson, Karen Okonkwo, Ian Chen)

- queries/ — readable source SQL alongside the build artifact:
  * One directory per intent with a 00-intent.yaml (question, canonical
    definition, variant axes, data caveats) plus the .sql files
  * noise/ subdirectory with the 152 long-tail queries
  * intent-matrix.yaml — top-level index across all 50 intents
  * README.md — contributor guide including data-realities gotchas
    (e.g. orders.order_status is NUMBER 0..4 not strings, mailshot is 0/1
    not Y/N, customers.country_id isn't a real FK)

- index.json — adds 04-queries.json to the file list

The corpus was authored against the real ORDER_ENTRY_DB on snowflake
(every query was executed against the warehouse during authoring) and
surfaces a few demo-grade data anomalies on the way:
- 4028/5027 promoted orders fall outside their promo's date window
- 1250 In-Store Pickup orders carry a delivery_address_id
- 3 warehouses show NEGATIVE avg dispatch lead time
- Some promotions have negative discount rates (unit_price > list_price)
- Order 9866 header_total = $210 but its line items sum to $1000

Verified via `datahub datapack load --dry-run` — all MCPs schema-compatible.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants