Robust multi-skeleton handling across frontend workflows

[talmo]

Tracking work to make the frontend robust against Labels containing multiple skeletons.

Layer context

• Data layer (sleap-io.js) — already plural-first: Labels exposes skeletons: Skeleton[] only, with no singular skeleton accessor. No equivalent of the Python Labels.skeleton ValueError footgun. Good baseline.
• Frontend (this repo) — picks "the" skeleton per workflow. All decisions about which skeleton to use, how to surface multi-skeleton state, and how to remediate it live here.

If a generic dedup or validation utility would help, it could live in sleap-io.js (reusable across consumers) — happy to spin out a companion issue there.

Background

Legacy talmolab/sleap (Python/Qt) had several GUI codepaths that called labels.skeleton (singular) and crashed with ValueError: Labels.skeleton can only be used when there is only a single skeleton... on multi-skeleton projects. Root cause was the old "Load Skeleton" button (talmolab/sleap#713, fixed 2023) silently appending skeletons. Some symptom paths got ad-hoc try/except patches; others didn't.

Legacy .slp files in the wild still carry redundant skeletons. The frontend should handle these gracefully and not let user workflows recreate that state.

Required frontend behaviour

1. Detect multi-skeleton state on project load and offer a one-click dedup of unused skeletons (port of @roomrys's script — could live in sleap-io.js if useful elsewhere).
2. Prevent creation of multi-skeleton state during normal workflows (e.g. loading a different skeleton template should replace not append when no instances depend on the old one; explicit confirmation otherwise).
3. Surface the state — clear UI indicator when multiple skeletons exist, with affordance to pick a primary or dedup.
4. Validate before launching any workflow that effectively picks a single skeleton, with an actionable error/CTA — not a silent fallback.

Workflows that need explicit multi-skeleton handling

Drawn from where legacy sleap assumed single-skeleton (file refs are for historical context — sleap-app will reimplement these UIs):

• Training pipeline launch ("Run Training..." / equivalent) — choose / confirm the target skeleton. Legacy ref: sleap/gui/app.py:1742. Legacy report: Error occurs during creation of subsequent training configurations and merging predictions in SLEAP GUI sleap#1090.
• Export training job package — same flow as training launch.
• Merge into Project / merging predictions — both sides may carry different skeleton sets; need explicit conflict resolution UI. Legacy ref: sleap/gui/dialogs/merge.py:50, 63-64. Legacy report: Error occurs during creation of subsequent training configurations and merging predictions in SLEAP GUI sleap#1090 (second traceback).
• Open / replace skeleton — replacing the project skeleton must not silently append. Legacy ref: sleap/gui/commands.py:3034 (# Assumes single skeleton).
• Save / export skeleton — exporting the project's skeleton must pick or prompt when multiple exist. Legacy ref: sleap/gui/commands.py:3120 (# Assume single skeleton).

Related legacy sleap issues

• Loading new skeleton from file causes labels to contain multiple conflicting skeletons sleap#713 — root-cause "Load Skeleton" multi-skeleton bug (closed)
• Error occurs during creation of subsequent training configurations and merging predictions in SLEAP GUI sleap#1090 — training + merge dialog crashes (closed; tracked here)
• Unable to train a Centered Instance model during training data (m1 mac SLEAP) sleap#913, #1066, #1144 — related multi-skeleton symptoms referenced in #1090

Acceptance criteria

• Project-open path detects multi-skeleton state and offers dedup
• Load/replace-skeleton path cannot silently produce multi-skeleton state
• Merge path handles or refuses multi-skeleton on either side with a clear message
• Training launch / export validates a single chosen skeleton before proceeding
• (Optional) If a generic dedup/validation helper emerges, file it on sleap-io.js

[talmo]

Triaging talmolab/sleap#2025 surfaced cross-stack data-layer notes worth recording here.

The user-visible inference→multi-skeleton bug was already fixed at the sleap layer by talmolab/sleap#2282 (merged 2025-08-29). The inference runner at sleap/gui/learning/runners.py:767-769 now goes through sio.Labels.merge(new_labels, frame="…"), which dedupes structurally via SkeletonMatcher. 1.5+ projects no longer accumulate skeletons across inference runs.

Cross-stack invariant (the important takeaway): in both modern data layers, Instance.points is stored positionally —

• sleap-io: PointsArray ndarray (sleap_io/model/instance.py:413)
• sleap-io.js: PointsArray = Point[] (src/model/instance.ts:29)

— so inst.skeleton = canonical is safe iff the canonical skeleton has identical nodes in the same order. Skeleton.matches() (skeleton.ts:124, skeleton.py:691) enforces strict-order equality, so dedup helpers should keep using matches(), not a node-set comparison. This is what killed talmolab/sleap#2075 — legacy Python Instance.points was keyed by Node identity, so reassignment blanked points (see follow-up). That class of bug is gone in both modern stacks, but the strict-order requirement is the design constraint any dedup helper must respect.

Residual edges (filed as separate trackers):

• sleap-io: Labels.update() at sleap_io/model/labels.py:446 (and :847, :876) still dedupes by identity for direct-construction paths. Doesn't affect the GUI inference flow (merge route is correct), but leaks for scripts. Filed Labels.update() skeleton dedup uses identity, not structure (direct-construction edge) sleap-io#427.
• sleap-io.js: parseSkeletons (src/codecs/slp/parsers.ts:92) preserves disk-level duplicate skeletons on load. Legacy 1.3/1.4 .slp files in the wild already carry these duplicates — relevant for AC Remove sleap-io.js workarounds once upstream issues are resolved #1 (detect on project-open). Proposed Labels.dedupSkeletons() helper at Expose Labels.dedupSkeletons(); dedup content-equal skeletons on SLP load sleap-io.js#103.

This repo's status: src/commands/editCommands.ts:380 (MergePredictions) → src/lib/merge.ts:114 already uses Skeleton.matches() correctly for the predictions-merge flow, so the inference path here is handled. AC #1 (detect on project-open) is still unimplemented and is the natural caller for Labels.dedupSkeletons() once it lands upstream — until then this repo would need its own dedup pass on legacy file load.

[alicup29]

Partial progress from #163 (PR #171, merged — feat(skeleton): Load From File… / Save As…).

That PR implements the two skeleton-specific workflow bullets called out here:

• Open / replace skeleton — importing a skeleton (.json/.yaml/.slp) into a project mutates the existing single Skeleton object in place rather than appending a new one. OpenSkeletonCommand rebuilds nodes/edges/symmetries on the same object and remaps instance points by node name (or explicit link), so labels.skeletons[0] === state.skeleton is preserved and the count never grows. This directly satisfies item 2 ("loading a different skeleton should replace not append") for the load-from-file path — the legacy commands.py:3034 "# Assumes single skeleton" footgun does not recur here. When a skeleton already exists, a ported ReplaceSkeletonTableDialog (link/rename/add/delete) provides the explicit confirmation + node remediation.
• Save / export skeleton — exports the project skeleton to .yaml. With a single project skeleton there is no ambiguity; the multi-skeleton "pick or prompt" case (legacy commands.py:3120 "# Assume single skeleton") is not yet handled (see below).

Still open under this issue (deliberately out of scope of #163)

#163 only guarantees the load/save paths don't create multi-skeleton state. The broader robustness work remains:

1. Detect + dedup existing multi-skeleton state on project load — PyQt's OpenSkeleton.delete_extra_skeletons was intentionally not ported; OpenSkeletonCommand operates only on labels.skeletons[0], so legacy .slp files carrying redundant skeletons are left as-is. The one-click dedup of unused skeletons (item 1) is unbuilt.
2. Surface multi-skeleton state in the UI (item 3) — no indicator/affordance yet.
3. Validate before workflows that pick a single skeleton (item 4) — training launch, export training job package, and merge-into-project still assume a single skeleton; the merge-dialog conflict resolution is unbuilt.

So #163 closes the "replace not append" gap for the skeleton-load entry point but does not close this issue.

Upstream note: byte-for-byte .json/.h5 skeleton export parity is tracked separately at talmolab/sleap-io.js#167.