Repository navigation
fix(sleep): skip Claude Code isMeta records when harvesting user prompts - #295
Merged
Yifan Yang (Yif-Yang) merged 1 commit intoSep 30, 2026
Conversation
Claude Code writes text it injects on the user's behalf as role "user"
records marked isMeta: true. The most common is the full SKILL.md body
of every skill the agent loads; others are messages relayed from other
sessions and usage-limit notices. The harvester counted all of them as
user prompts and ran the feedback heuristic over them, so words a skill
document uses ("wrong", "revert", "did not") labelled the session a
failure, and a session opened with a slash command mined the skill body
itself as the task intent.
Skip isMeta user records in digest_transcript, except one that carries
an _AGENT_SESSION_MARKERS marker, so a session driven by the plugin's
own /skillopt-sleep body is still dropped as an agent session. The skill
stays attributed through the assistant's Skill tool call.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Claude Code writes some text it injects on the user's behalf into the transcript as a
type: "user"record withmessage.role: "user", markedisMeta: true. The most common case is the fullSKILL.mdbody of each skill the agent loads ("Base directory for this skill: …"). Others are messages relayed from another session, usage-limit notices, and "Continue from where you left off." The user typed none of it.digest_transcriptcounts every such record as a user prompt and runs_detect_feedbackover it. Skill documents are full of words the feedback heuristic reads as complaints ("wrong", "revert", "did not", "broken"). A session that loaded a skill is then mined asfailwhatever the user actually said. A session opened with a slash command (the<command-name>message is already filtered) takes the skill body itself as the task intent.Measured over the 400 most recent sessions in one real
~/.claude/projects(Claude Code 2.1.x, heavy skill use), withharvest.digest_transcriptandmine.heuristic_mine:main(79124b3)SKILL.mdbodySKILL.mdbodyfailEvery one of the 320 skill-body records carries
isMeta: true. The 23 tasks that disappear are sessions whose only "user" text was injected (a skill body, a relayed message, a/contextreport).End to end,
skillopt-sleep harvest --backend mock --source claudeon one of those sessions writes[val/fail]onmainand[val/unknown]with this change. The difference in the task draft is the skill body leavingcontext_excerpt, and the outcome no longer being decided by it.Fix
In
digest_transcript, skip user records whoseisMetaistrue, before the text is counted as a prompt or scanned for feedback. The one exception is an injected body that contains an_AGENT_SESSION_MARKERSmarker. It is kept, so a session driven by the plugin's own/skillopt-sleepcommand body ("You are driving **SkillOpt-Sleep**") is still dropped by_is_agent_session, as it is onmain. Skill attribution does not change:skills_usedcomes from the assistant'sSkilltool call, so the fan-out's skill hint is kept.This follows #99, which filtered sub-agent transcripts and expanded slash-command bodies for the same reason: prompts the user never wrote were being mined as their tasks.
Not changed
isMetais a Claude Code field; the other harvesters read other formats._is_meta_prompt's text heuristics are left as they are. TheisMetaflag is the structural signal, so no new text marker is added.n_user_turns == 0and is dropped like any other empty session. In the sample above that was 20 sessions: a skill slash command with no typed follow-up, a session driven only by relayed messages, and a/context-only session. I think dropping them is right (there is no user intent to mine), but it is your call.ARGUMENTS:tail Claude Code appends to an injected skill body is dropped with the body. For a user-typed/skill args, the arguments are also in the<command-args>record, which_is_meta_promptalready skipped before this change.### Fixedif you want it in the PR.Test
Three tests in
tests/test_sleep_engine.py::TestHarvest. The first two use a transcript shaped like the real one: a typed prompt, aSkilltool call, the injectedisMetaskill body, and anisMetarelayed message.test_digest_skips_claude_injected_meta_messages: only the typed prompt is harvested, no feedback signals,skills_usedstill names the skill.test_injected_skill_body_does_not_decide_the_mined_outcome: with a closing "perfect, thanks", the mined task issuccesswith the typed intent.test_injected_agent_marker_body_still_drops_the_session: a/skillopt-sleepsession whose command body arrives as anisMetarecord is still excluded byharvest(). This one passes onmainand guards the existing behaviour. It fails ifisMetarecords are skipped without the marker exception.On
mainthe first two fail: the first with the skill body and the relayed message inuser_prompts, the second with'fail' != 'success'.Mutations, each run against the three tests:
"Base directory"in text)Validation
python -m pytest -q: 1500 passed, 11 skipped on Python 3.10 and 3.14 (main: 1497 passed, 11 skipped).ruff check tests/test_sleep_engine.pyclean.ruff check skillopt_sleep/harvest.pyreports oneI001that is already onmain; the repo-wide count is 149 on both.skillopt-sleep harvest --backend mock --source claude --scope all --lookback-hours 0under an isolatedHOME, as above.