Skip to content

Feat/add string derivation detection skill#2472

Open
eugeneboms wants to merge 8 commits into
microsoft:mainfrom
eugeneboms:feat/add-string-derivation-detection-skill
Open

Feat/add string derivation detection skill#2472
eugeneboms wants to merge 8 commits into
microsoft:mainfrom
eugeneboms:feat/add-string-derivation-detection-skill

Conversation

@eugeneboms

Copy link
Copy Markdown
Contributor

feat(skills): add string-derivation detection skill for data science

Description

This PR introduces a new data science skill for detecting string column derivations in datasets. The skill identifies redundant string columns where one column is a deterministic transformation of another (e.g., lookup expansion, concatenation, substring extraction), enabling safe column removal for data reduction without information loss.

The skill uses progressive sampling (3→10→30 rows) to achieve 10-100x performance speedup over full-dataset testing while maintaining high confidence in pattern detection.

Key capabilities:

  • Detects 9 derivation patterns: lookup expansion, concatenation, substring extraction, character removal, case transformation, numeric extraction, edit distance, format strings, and boolean checks
  • Provides Python reference implementations with pandas/numpy dependencies
  • Includes both bash and PowerShell wrapper scripts for cross-platform compatibility
  • Returns confidence scores for each detected derivation relationship

Additional changes:

  • Updated VS Code settings to register additional skill discovery locations (.agents/skills, .claude/skills, user-level directories, and Fabric extension skills)
  • Registered skill in data-science collection manifest
  • Automatically included in hve-core-all mega-collection via plugin generator

Related Issue(s)

None

Type of Change

Select all that apply:

Code & Documentation:

  • Bug fix (non-breaking change fixing an issue)
  • New feature (non-breaking change adding functionality)
  • Breaking change (fix or feature causing existing functionality to change)
  • Documentation update

Infrastructure & Configuration:

  • GitHub Actions workflow
  • Linting configuration (markdown, PowerShell, etc.)
  • Security configuration
  • DevContainer configuration
  • Dependency update

AI Artifacts:

  • Reviewed contribution with prompt-builder agent and addressed all feedback
  • Copilot instructions (.github/instructions/*.instructions.md)
  • Copilot prompt (.github/prompts/*.prompt.md)
  • Copilot agent (.github/agents/*.agent.md)
  • Copilot skill (.github/skills/*/SKILL.md)
  • Copilot hook (.github/hooks/*/*.json)
  • Eval spec added/updated for changed AI artifacts (evals/)

Note for AI Artifact Contributors:

  • Agents: Research, indexing/referencing other project (using standard VS Code GitHub Copilot/MCP tools), planning, and general implementation agents likely already exist. Review .github/agents/ before creating new ones.
  • Skills: Must include both bash and PowerShell scripts. See Skills.
  • Model Versions: Only contributions targeting the latest Anthropic and OpenAI models will be accepted. Older model versions (e.g., GPT-3.5, Claude 3) will be rejected.
  • See Agents Not Accepted and Model Version Requirements.

Other:

  • Script/automation (.ps1, .sh, .py)
  • Other (please describe):

Sample Prompts (for AI Artifact Contributions)

User Request:

"Analyze my customer dataset and identify which columns are derived from others so I can reduce dimensionality without losing information."

or

"I have a dataset with 50+ string columns. Help me find redundant columns that can be safely removed."

Execution Flow:

  1. User reads the string-derivation skill documentation at .github/skills/data-science/data-reduction/string-derivation/SKILL.md
  2. User copies reference algorithm functions from references/algorithms.md into their Python project
  3. User loads their dataset with pandas and identifies string columns
  4. User calls filter_derivation_candidates() once to optimize performance by filtering high-cardinality ID columns
  5. User calls detect_all_string_derivations_optimized() for each target column or in batch mode for all columns
  6. Skill performs progressive sampling (3→10→30 rows) testing 9 derivation patterns
  7. Skill returns sorted list of derivation findings with confidence scores (match_ratio)

Output Artifacts:

Derivation findings dictionary:

{
    'type': 'lookup_expansion',
    'operands': ['GENDER'],
    'formula': 'GENDER_DESCRIPTION = lookup_map[GENDER]',
    'match_ratio': 0.982,
    'lookup_map': {'F': 'Female', 'M': 'Male', 'N': 'Non-binary'}
}

Or for concatenation:

{
    'type': 'concatenation',
    'operands': ['FIRST_NAME', 'LAST_NAME'],
    'formula': "FULL_NAME = FIRST_NAME + ' ' + LAST_NAME",
    'match_ratio': 0.995,
    'separator': ' '
}

Success Indicators:

  • Skill correctly identifies derivation patterns with formulas (e.g., "FIRST_NAME + ' ' + LAST_NAME → FULL_NAME")
  • Match ratios ≥0.95 indicate high-confidence derivations safe for removal
  • Progressive sampling completes analysis 10-100x faster than full dataset testing
  • All 9 pattern types are tested: lookup expansion, concatenation, substring, character removal, case transformation, numeric extraction, edit distance, format strings, boolean checks
  • User can verify detected relationships by inspecting a few sample rows manually

For detailed contribution requirements, see:

Testing

Automated validation:

  • Generated PR reference XML with diff analysis via pr-reference skill
  • Verified skill file structure and frontmatter schema
  • Confirmed both bash and PowerShell scripts are present
  • Validated collection manifest syntax (YAML)
  • Verified plugin regeneration produced consistent outputs

Security analysis:

  • ✅ No sensitive data, credentials, or API keys in any files
  • ✅ Dependencies (pandas, numpy) are standard, well-maintained data science libraries
  • ✅ Reference implementation pattern (users copy code into their projects) avoids direct code execution risks
  • ✅ VS Code settings changes add skill discovery paths but do not execute code automatically

Diff-based assessment:

  • ✅ All changed files follow repository naming conventions
  • ✅ No breaking changes to existing APIs or functionality
  • ✅ Purely additive: new skill registration only
  • ✅ Collection manifests properly updated (data-science manual, hve-core-all automatic)
  • ✅ Plugin outputs synchronized with collection changes

Manual testing:

Note

Manual runtime testing of the skill algorithms should be performed with sample datasets to verify pattern detection accuracy.

Checklist

Required Checks

  • Documentation is updated (if applicable)
  • Files follow existing naming conventions
  • Changes are backwards compatible (if applicable)
  • Tests added for new functionality (N/A — skill provides reference implementation for users to copy)

AI Artifact Contributions

  • Used /prompt-analyze to review contribution
  • Addressed all feedback from prompt-builder review
  • Verified contribution follows common standards and type-specific requirements

Required Automated Checks

The following validation commands must pass before merging:

  • Markdown linting: npm run lint:md
  • Spell checking: npm run spell-check
  • Frontmatter validation: npm run lint:frontmatter
  • Skill structure validation: npm run validate:skills
  • Link validation: npm run lint:md-links
  • PowerShell analysis: npm run lint:ps
  • Eval spec schema and coverage (if AI artifacts changed): npm run eval:lint:schema
  • Plugin freshness: npm run plugin:generate
  • Docusaurus tests: npm run docs:test

Security Considerations

  • This PR does not contain any sensitive or NDA information
  • Any new dependencies have been reviewed for security issues
  • Security-related scripts follow the principle of least privilege (N/A — no security-related scripts)

Additional Notes

Plugin regeneration: The npm run plugin:generate command was run to update all plugin manifests and READMEs. This explains why all 13 plugin READMEs show modifications even though only the data-science and hve-core-all collections were directly affected. The generator applies markdown formatting (lint:md:fix and format:tables) to all plugin files as post-processing.

Skill architecture: The skill follows the reference implementation pattern where users copy algorithm functions from references/algorithms.md into their own projects rather than importing the skill as a library. This design choice provides maximum flexibility for customization while avoiding dependency management complexity.

VS Code settings rationale: The skill discovery path additions enable the skill to be loaded from multiple conventional locations including user-level directories (~/.agents/skills, ~/.copilot/skills, ~/.claude/skills), alternate project locations (.agents/skills, .claude/skills), and extension-provided directories (Fabric extension). This improves developer experience by supporting different workflow preferences and tool conventions.

Eugene Bobukh added 4 commits July 15, 2026 14:23
Introduce data-science/data-reduction/string-derivation skill to detect derived string columns in large datasets. The skill provides detection algorithms for identifying string columns that are computed from other columns.

Also update VS Code settings.json to register additional skill locations (.agents/skills, .claude/skills, user-level directories, and Fabric extension skills).
…ve-core-all collections

- Added string-derivation skill entry to data-science.collection.yml
- Auto-added to hve-core-all collection via plugin generator
- Regenerated affected collection .md files and plugin outputs
- Updated plugin.json manifests for data-science and hve-core-all
- Added description frontmatter to string-derivation/references/algorithms.md
- Applied markdown table formatting via plugin generator post-processing
- All collection .md and plugin README.md files reformatted for consistency
…ation.ps1

- Removed unused ScriptDir variable
- Changed packageCheck assignment to suppress output ()
- Renamed args variable to pythonArgs to avoid shadowing built-in
@eugeneboms
eugeneboms requested a review from a team as a code owner July 15, 2026 22:23
@codecov-commenter

codecov-commenter commented Jul 15, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 82.74%. Comparing base (5fcfb8b) to head (fb0e923).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #2472      +/-   ##
==========================================
- Coverage   82.75%   82.74%   -0.01%     
==========================================
  Files         154      154              
  Lines       20843    20840       -3     
  Branches       13       13              
==========================================
- Hits        17249    17245       -4     
- Misses       3592     3593       +1     
  Partials        2        2              
Flag Coverage Δ
docusaurus 94.38% <ø> (ø)
pester 86.26% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 2 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Eugene Bobukh added 2 commits July 15, 2026 15:28
Add missing copyright and SPDX-License-Identifier headers to
detect-string-derivation.ps1 and detect-string-derivation.sh per
repository conventions.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds a new string-derivation data science skill (documentation + reference algorithms + wrappers) and wires it into the data-science and hve-core-all collections/plugins, plus expands VS Code skill discovery locations.

Changes:

  • Added a new string-derivation skill with documentation and reference algorithm implementations.
  • Registered the skill in data-science and hve-core-all collection manifests and regenerated plugin outputs.
  • Updated .vscode/settings.json to add additional skill discovery locations.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
plugins/hve-core-all/README.md Adds string-derivation to the auto-generated skill list.
plugins/hve-core-all/.github/plugin/plugin.json Includes data science data-reduction skills in the mega-collection plugin.
plugins/data-science/README.md Adds string-derivation to the data-science plugin skill list.
plugins/data-science/.github/plugin/plugin.json Adds the data-reduction skill path to the data-science plugin.
collections/hve-core-all.collection.yml Registers the new skill in the hve-core-all collection.
collections/hve-core-all.collection.md Adds string-derivation to the collection’s auto-generated docs table.
collections/data-science.collection.yml Registers the new skill in the data-science collection.
collections/data-science.collection.md Adds string-derivation to the collection’s auto-generated docs table.
.vscode/settings.json Expands chat.agentSkillsLocations to additional local/user directories and an extension path.
.github/skills/data-science/data-reduction/string-derivation/SKILL.md New skill documentation and usage examples.
.github/skills/data-science/data-reduction/string-derivation/references/algorithms.md Reference implementations for derivation detection patterns and progressive sampling.
.github/skills/data-science/data-reduction/string-derivation/detect-string-derivation.sh Bash wrapper intended to run the detection via Python.
.github/skills/data-science/data-reduction/string-derivation/detect-string-derivation.ps1 PowerShell wrapper intended to run the detection via Python.

Comment thread .vscode/settings.json Outdated
Comment on lines +99 to +101
".github/skills/security": true,
".github/skills/shared": true
".github/skills/shared": true,
"~/.vscode/extensions/synapsevscode.synapse-1.25.0/copilot/skills": true
Comment thread .vscode/settings.json Outdated
Comment on lines +169 to +171
derivations = detect_all_string_derivations_optimized(df, 'FULL_NAME', string_cols)
if derivations and derivations[0]['match_ratio'] >= 0.95:
print(f"✅ Can remove {target_col}: {derivations[0]['formula']}")
Comment on lines +34 to +36
## Common String Operations

### 1. Concatenation with Separator
Comment on lines +209 to +212
s = series.copy()
if s.dtype == 'object':
s = s.astype(str).str.lower().strip()
return s.map(bool_map)
Comment on lines +139 to +155
# Load the detection functions from references/algorithms.md
# NOTE: In production, these should be in a proper Python module
# For now, users must copy the functions from references/algorithms.md
try:
# Import functions - adjust path as needed
import sys
from pathlib import Path

# Try to load from a local module if it exists
# Otherwise, provide helpful error message
print("Note: Detection functions must be copied from references/algorithms.md")
print("Creating inline implementation for demonstration...")

# Minimal inline implementation
# (In production, source these from algorithms.md)
exec(open(Path(__file__).parent / 'references' / 'algorithms.md').read())

Comment on lines +145 to +154
# Filter candidates once
# NOTE: This requires filter_derivation_candidates from algorithms.md
# filtered_candidates = filter_derivation_candidates(
# df, string_cols,
# max_cardinality=max_cardinality,
# max_candidates=max_candidates
# )

# Placeholder for demonstration
filtered_candidates = string_cols[:max_candidates]
Comment on lines +1 to +6
#!/usr/bin/env pwsh
# Copyright (c) 2026 Microsoft Corporation. All rights reserved.
# SPDX-License-Identifier: MIT
# Detect string derivations in tabular data
# Brought to you by microsoft/hve-core

eugeneboms and others added 2 commits July 16, 2026 14:46
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Comment thread .vscode/settings.json
".claude/skills": true,
"~/.agents/skills": true,
"~/.copilot/skills": true,
"~/.claude/skills": true,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The five new chat.agentSkillsLocations entries include user-level tilde paths (~/.agents/skills, ~/.copilot/skills, ~/.claude/skills) and a Claude-specific workspace path (.claude/skills). Tilde paths expand relative to each developer's home directory and carry no stable meaning as shared repository settings; on another machine they may silently load unrelated files. The .claude/skills and ~/.claude/skills entries also introduce an Anthropic Claude convention with no existing precedent in this repository's settings.

These five entries belong in personal user settings and should be removed from the committed file.

Comment on lines +44 to +54
- **Substring**: `EMPLOYEE_ID` contains `DEPT_CODE`
- **Character removal**: `PHONE_NUMBER` = `PHONE_DISPLAY` with formatting removed
- **Case transformation**: `email` vs `EMAIL`
- **Numeric extraction**: `ORDER_123` → `123`
- **Edit distance**: Systematic 1-2 character transformations
- **Format strings**: `{STATE}: {CITY}` patterns
- **Boolean checks**: Derived true/false flags

## Prerequisites

**Python Dependencies:**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The "Common patterns detected" bullet list appears verbatim twice in this file. This second occurrence is a duplicate of the block already present in the Overview section and should be removed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach of building Python code inside a bash heredoc and passing it to the Python interpreter at runtime is not the established convention in this repository. Every other skill that requires Python keeps its logic in standalone .py modules under a scripts/ directory and invokes them via a managed virtual environment from the shell or PowerShell wrapper. See .github/skills/experimental/powerpoint/, .github/skills/experimental/tts-voiceover/, and .github/skills/security/vex/ for examples.

The algorithms currently embedded across the heredoc and references/algorithms.md should be extracted into proper Python module(s) under a scripts/ directory. Both wrappers should be updated to set up a venv and invoke those modules directly, following the pattern established by the skills above. A pyproject.toml declaring pandas and numpy as dependencies and a committed uv.lock are also required. Addressing this restructuring would also resolve the two existing review comments about the exec() call on the Markdown file and the non-functional PowerShell placeholder.

Comment on lines +1 to +5
---
name: string-derivation
description: Detect derivable data columns via string operations for data reduction - Brought to you by microsoft/hve-core
user-invocable: true
---

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The frontmatter is missing license: MIT and the metadata block (authors, spec_version, last_updated) that all other skills in this repository include. See .github/skills/shared/pr-reference/SKILL.md, .github/skills/experimental/powerpoint/SKILL.md, or .github/skills/coding-standards/code-review/SKILL.md for examples of the expected structure.

Comment on lines 48 to +49
# Skills
- path: .github/skills/data-science/data-reduction/string-derivation

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New skills are expected to include a maturity field. Given the open issues in both wrapper scripts, maturity: experimental is appropriate here. Both rai-planner in this same collection and python-foundational in hve-core-all provide examples of the convention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants