Skip to content

Commit 6fc1dda

Browse files
ReviveCodingReviveCoding
committed
Initial Fashionpedia reliability pipeline and CI contracts
0 parents  commit 6fc1dda

62 files changed

Lines changed: 16716 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/ci.yml‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
name: Fashionpedia Evidence CI
2+
3+
on:
4+
push:
5+
branches: [main]
6+
pull_request:
7+
workflow_dispatch:
8+
9+
permissions:
10+
contents: read
11+
12+
jobs:
13+
fixture-contracts:
14+
name: Static evidence fixture contracts
15+
runs-on: ubuntu-latest
16+
timeout-minutes: 10
17+
steps:
18+
- name: Check out repository
19+
uses: actions/checkout@v4
20+
21+
- name: Set up Python 3.11
22+
uses: actions/setup-python@v5
23+
with:
24+
python-version: "3.11"
25+
cache: pip
26+
cache-dependency-path: requirements-ci.txt
27+
28+
- name: Install CI test dependency
29+
run: python -m pip install --disable-pip-version-check --requirement requirements-ci.txt
30+
31+
- name: Compile tracked Python sources
32+
run: python -m compileall -q scripts tests
33+
34+
- name: Run static evidence fixture contracts
35+
run: |
36+
mkdir -p artifacts
37+
python -m pytest -q tests/test_ci_release_fixture_contracts.py \
38+
--junitxml=artifacts/fashionpedia_ci_fixture_contracts.junit.xml
39+
40+
- name: Upload CI reports
41+
if: always()
42+
uses: actions/upload-artifact@v4
43+
with:
44+
name: fashionpedia-ci-reports-${{ github.run_id }}
45+
path: artifacts/
46+
if-no-files-found: error
47+
retention-days: 30

‎.gitignore‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
# Python environments and caches
2+
.venv/
3+
__pycache__/
4+
*.py[cod]
5+
.pytest_cache/
6+
.coverage
7+
htmlcov/
8+
9+
# Local backups and temporary files
10+
*.bak
11+
*.tmp
12+
*.temp
13+
14+
# Generated release and analysis artifacts
15+
dist/
16+
logs/
17+
.local-run/
18+
19+
# Model weights and serialized local artifacts
20+
*.pt
21+
*.pth
22+
*.safetensors
23+
*.joblib
24+
25+
# Extracted helper-package directory, not project source
26+
Amazon07_fashionpedia_final_confirmation_v2/
27+
28+
# Local dataset, cache, and scratch locations
29+
data/
30+
datasets/
31+
cache/
32+
models/
33+
checkpoints/

‎README.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# Fashionpedia Test Hardening v1
2+
3+
This package adds a read-only pytest evidence-contract suite for the local evidence freeze.
4+
5+
It verifies:
6+
7+
1. Final official-validation metrics and selection boundary.
8+
2. Source rule reconstruction count and source/validation image disjointness.
9+
3. Immutable category-33 neckline task configuration.
10+
4. Selected LoRA and frozen checkpoint presence and model-parameter contract.
11+
5. Source-artifact SHA-256 values.
12+
6. File-level SHA-256 evidence manifest integrity.
13+
7. Claim boundaries in frozen release notes.
14+
15+
The tests use only the Python standard library and pytest. They do not import PyTorch,
16+
Transformers, or PEFT and do not load a model, run inference, train, access raw Fashionpedia
17+
images/annotations, or re-score validation.

‎README.txt‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
Fashionpedia source-task rule forensic audit v1
2+
3+
Purpose: identify why the simple final-confirmation eligibility predicate reconstructed 20,829 source pairs while the canonical v1b train/dev manifests contain 20,800 pairs.
4+
5+
This package does NOT open official validation annotations or images and does NOT load a model.
6+
7+
Expand the ZIP into the project repo root, then run:
8+
powershell -NoProfile -ExecutionPolicy Bypass -File .\run_amazon07_fashionpedia_task_rule_forensics_v1.ps1
9+
10+
Read the resulting forensic JSON before touching official validation again.

‎README_CI.md‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
# Hosted CI Boundary
2+
3+
This repository intentionally separates two evidence gates.
4+
5+
## Hosted CI gate
6+
7+
`.github/workflows/ci.yml` runs only `tests/test_ci_release_fixture_contracts.py` against a small static fixture tracked in Git. It validates the immutable category-33 task rule, seven target attributes, source-rule reconstruction count, source-validation disjointness, frozen official-validation metrics, claim boundaries, and fixture SHA-256 integrity.
8+
9+
The workflow does **not** download Fashionpedia, load SigLIP2, load LoRA checkpoints, run inference, train, or score validation.
10+
11+
## Local evidence-release gate
12+
13+
`tests/test_release_evidence_contracts.py` is intentionally local-only because it validates the full local evidence release, including checkpoint presence and source/release SHA-256 contracts. The full bundle remains excluded from Git to avoid committing model artifacts and the local release archive into repository history.
14+
15+
A hosted CI pass therefore means the tracked static evidence contract is intact. It is not a substitute for the local full evidence-release contract suite.

‎configs/fashionpedia_neckline_attribute_v1.json‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"schema_version": "fashionpedia_neckline_attribute_v1",
3+
"task_name": "fashionpedia_neckline_attribute_verification_v1",
4+
"selection_scope": "train_only",
5+
"target_category": {
6+
"id": 33,
7+
"expected_name": "neckline"
8+
},
9+
"target_attributes": [
10+
{ "id": 182, "name": "round (neck)" },
11+
{ "id": 183, "name": "v-neck" },
12+
{ "id": 185, "name": "oval (neck)" },
13+
{ "id": 187, "name": "sweetheart (neckline)" },
14+
{ "id": 189, "name": "boat (neck)" },
15+
{ "id": 190, "name": "scoop (neck)" },
16+
{ "id": 200, "name": "straight across (neck)" }
17+
],
18+
"sample_rule": {
19+
"include_only_target_category": true,
20+
"require_exactly_one_target_attribute": true,
21+
"require_positive_bbox_area": true,
22+
"exclude_zero_target_attribute_instances": true,
23+
"exclude_multi_target_attribute_instances": true
24+
},
25+
"development_split_policy": {
26+
"group_key": "image_id",
27+
"train_fraction": 0.8,
28+
"development_fraction": 0.2,
29+
"seed": 20260701
30+
},
31+
"qualification_thresholds": {
32+
"minimum_eligible_instances_per_class": 1200,
33+
"minimum_total_eligible_instances": 10000,
34+
"maximum_multi_target_rate": 0.05
35+
},
36+
"claim_boundary": "Train-only task-definition and data-quality audit. No validation annotations, images, model predictions, training, hyperparameter selection, or final metrics are used."
37+
}

‎configs/hog_neckline_baseline_v1.json‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
{
2+
"schema_version": "hog_neckline_baseline_v1",
3+
"run_role": "classical_shape_baseline",
4+
"task_name": "fashionpedia_neckline_attribute_verification_v1",
5+
"target_attribute_ids": [182, 183, 185, 187, 189, 190, 200],
6+
"target_attribute_names": {
7+
"182": "round (neck)",
8+
"183": "v-neck",
9+
"185": "oval (neck)",
10+
"187": "sweetheart (neckline)",
11+
"189": "boat (neck)",
12+
"190": "scoop (neck)",
13+
"200": "straight across (neck)"
14+
},
15+
"crop": {
16+
"padding_fraction": 0.15,
17+
"resize_width": 64,
18+
"resize_height": 64,
19+
"interpolation": "bilinear",
20+
"color_mode": "grayscale"
21+
},
22+
"hog": {
23+
"orientations": 9,
24+
"pixels_per_cell": [8, 8],
25+
"cells_per_block": [2, 2],
26+
"block_norm": "L2-Hys"
27+
},
28+
"classifier": {
29+
"family": "LogisticRegression",
30+
"solver": "lbfgs",
31+
"C": 1.0,
32+
"class_weight": "balanced",
33+
"max_iter": 1500,
34+
"tol": 0.0001,
35+
"random_state": 20260701
36+
},
37+
"evaluation": {
38+
"ece_bins": 15,
39+
"primary_metric": "macro_f1",
40+
"secondary_metrics": [
41+
"accuracy",
42+
"balanced_accuracy",
43+
"weighted_f1",
44+
"multiclass_brier",
45+
"negative_log_likelihood",
46+
"top_2_accuracy"
47+
]
48+
},
49+
"claim_boundary": "Train-only development baseline. Uses only the fixed v1b group-disjoint train and dev manifests. Does not load official Fashionpedia validation annotations or images, does not perform hyperparameter sweeps, and is not a final external-confirmation claim."
50+
}

‎configs/siglip2_frozen_neckline_baseline_v1.json‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
{
2+
"schema_version": "siglip2_frozen_neckline_baseline_v1",
3+
"run_role": "frozen_pretrained_embedding_baseline",
4+
"task_name": "fashionpedia_neckline_attribute_verification_v1",
5+
"model": {
6+
"model_id": "google/siglip2-base-patch16-224",
7+
"processor_mode": "image_only",
8+
"use_fast": false,
9+
"encoder_trainable": false,
10+
"embedding_normalization": "l2"
11+
},
12+
"target_attribute_ids": [182, 183, 185, 187, 189, 190, 200],
13+
"target_attribute_names": {
14+
"182": "round (neck)",
15+
"183": "v-neck",
16+
"185": "oval (neck)",
17+
"187": "sweetheart (neckline)",
18+
"189": "boat (neck)",
19+
"190": "scoop (neck)",
20+
"200": "straight across (neck)"
21+
},
22+
"crop": {
23+
"padding_fraction": 0.15,
24+
"color_mode": "RGB"
25+
},
26+
"embedding_extraction": {
27+
"initial_batch_size": 32,
28+
"allow_oom_batch_halving": true,
29+
"device_required": "cuda",
30+
"dtype": "float32"
31+
},
32+
"classifier": {
33+
"family": "LogisticRegression",
34+
"solver": "lbfgs",
35+
"C": 1.0,
36+
"class_weight": "balanced",
37+
"max_iter": 1500,
38+
"tol": 0.0001,
39+
"random_state": 20260701
40+
},
41+
"evaluation": {
42+
"ece_bins": 15,
43+
"primary_metric": "macro_f1",
44+
"secondary_metrics": [
45+
"accuracy",
46+
"balanced_accuracy",
47+
"weighted_f1",
48+
"negative_log_likelihood",
49+
"multiclass_brier",
50+
"top_label_ece",
51+
"top_2_accuracy"
52+
]
53+
},
54+
"claim_boundary": "Train-only frozen-encoder development baseline. Uses only the fixed Fashionpedia v1b group-disjoint train and dev manifests. No LoRA weights are trained, no hyperparameter sweep is performed, and official Fashionpedia validation annotations or images are not loaded."
55+
}

‎configs/siglip2_lora_matched_head_development_v1.json‎

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
{
2+
"schema_version": "siglip2_lora_matched_head_development_v1",
3+
"run_role": "controlled_frozen_head_vs_lora_development",
4+
"task_name": "fashionpedia_neckline_attribute_verification_v1",
5+
"model": {
6+
"model_id": "google/siglip2-base-patch16-224",
7+
"processor_mode": "image_only",
8+
"use_fast": false,
9+
"embedding_dimension": 768
10+
},
11+
"target_attribute_ids": [182, 183, 185, 187, 189, 190, 200],
12+
"target_attribute_names": {
13+
"182": "round (neck)",
14+
"183": "v-neck",
15+
"185": "oval (neck)",
16+
"187": "sweetheart (neckline)",
17+
"189": "boat (neck)",
18+
"190": "scoop (neck)",
19+
"200": "straight across (neck)"
20+
},
21+
"crop": {
22+
"padding_fraction": 0.15,
23+
"color_mode": "RGB"
24+
},
25+
"head": {
26+
"family": "Linear",
27+
"input_dimension": 768,
28+
"output_classes": 7,
29+
"initialization_seed": 20260701
30+
},
31+
"lora": {
32+
"scope": "vision_encoder_self_attention_only",
33+
"rank": 8,
34+
"alpha": 16,
35+
"dropout": 0.05,
36+
"bias": "none",
37+
"expected_target_module_count": 48,
38+
"expected_adapter_parameter_count": 589824,
39+
"excluded_module": "vision_model.head.attention.out_proj"
40+
},
41+
"training": {
42+
"seed": 20260701,
43+
"epochs": 6,
44+
"physical_batch_size": 8,
45+
"gradient_accumulation_steps": 4,
46+
"effective_batch_size": 32,
47+
"head_learning_rate": 0.0003,
48+
"lora_learning_rate": 0.0001,
49+
"weight_decay": 0.01,
50+
"max_grad_norm": 1.0,
51+
"optimizer": "AdamW",
52+
"mixed_precision": "fp16",
53+
"class_weighting": "inverse_frequency_balanced",
54+
"shuffle_policy": "deterministic_numpy_permutation_per_epoch"
55+
},
56+
"selection": {
57+
"scope": "development_only",
58+
"primary_metric": "macro_f1",
59+
"tie_break_metric": "negative_log_likelihood",
60+
"direction": {
61+
"macro_f1": "maximize",
62+
"negative_log_likelihood": "minimize"
63+
}
64+
},
65+
"evaluation": {
66+
"ece_bins": 15,
67+
"metrics": [
68+
"accuracy",
69+
"balanced_accuracy",
70+
"macro_f1",
71+
"weighted_f1",
72+
"negative_log_likelihood",
73+
"multiclass_brier",
74+
"top_label_ece",
75+
"top_2_accuracy"
76+
]
77+
},
78+
"claim_boundary": "Train-only controlled development experiment. Both arms use the fixed Fashionpedia v1b group-disjoint train and dev manifests. Official Fashionpedia validation annotations and images are not loaded. Development metrics are not external confirmation results."
79+
}

0 commit comments

Comments
 (0)