Skip to content

pstack: Hillclimb opens its log "via show-me-your-work" but defines its own file name and columns #1297

Open

Description

@robbiesierra

Hillclimb says to open its decision log through show-me-your-work, then defines a different file and a different set of columns.
show-me-your-work asks callers not to do that.

Hillclimb step 3 (hillclimb.md:9):

Open the decision log via the show-me-your-work skill. A decision.tsv, one row per attempt: id, hypothesis, change, before, after, delta, tests, verdict (kept or reverted), note.

The Reply line asks for "the decision.tsv path" (hillclimb.md:21).

show-me-your-work:

  • Has one canonical format with six columns: ts, phase, decision, why, evidence, result (SKILL.md:9-22).
  • Its helper scripts/log.sh takes exactly those fields and exits 1 on any other argument count, so it cannot write a Hillclimb row.
  • Keeps the log at decisions.tsv or .audit/<task-slug>.tsv (SKILL.md:46).
  • Says: "Other skills route their audit trail here instead of inventing one. Reference it by name and let it own the format. Don't restate the columns." (SKILL.md:80-82)

In practice an agent running Hillclimb has to pick one of the two formats.
Either log.sh cannot be used, or the per-attempt fields Hillclimb reads before each attempt (before, after, delta, verdict) end up packed into decision, evidence and result.

Possible fixes:

  1. Have Hillclimb use the show-me-your-work format and say how an attempt maps onto it.
    For example: phase is the attempt id, decision is the change, why is the hypothesis, evidence is the commit and the before and after numbers, and result is kept or reverted with the delta.
  2. Keep Hillclimb's nine-column table as its own experiment log, drop "via the show-me-your-work skill" from step 3, and log only decision points to show-me-your-work as usual.

Found by reading the files at fbb86c0.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions