fix(agent): tell the model when to act and stop offering ids it must not fill (CRM-238)

[pastoriniMatheus]

Problema

Três defeitos de prompt/schema por trás de "a IA não move o card". São a causa upstream do CRM-237 — o guard de servidor de lá continua como rede de segurança, mas o problema nasce aqui.

1. O schema contradizia o prompt

A instrução dizia que o id vinha do contexto:

"The conversation_id will be automatically extracted from the context."

Enquanto a docstring da tool — que vira o schema que o modelo lê — oferecia o campo:

conversation_id: ID of the conversation (optional, auto-extracted from context)

O modelo vê campo preenchível e preenche. Mandou o id do contato e o CRM respondeu 400 CONVERSATION_NOT_FOUND. Agora os dois ids são DO NOT SET ... ignored.

Detalhe que o teste pegou: a docstring é reescrita em runtime (pipeline_manipulation.__doc__ = f"""...""" mais abaixo na factory). A estática, que o leitor encontra primeiro, é descartada — o modelo recebe a dinâmica. Meu primeiro patch corrigiu só a estática e não teria efeito nenhum; o teste test_tool_schema_agrees_with_the_prompt falhou e expôs isso.

2. O prompt nunca dizia se a conversa já tinha card

Sem essa informação o modelo escolhia no escuro — e escolheu add_to_pipeline para uma conversa já no funil. Agora a instrução diz: move_to_stage é o caso normal de conversa em andamento; add_to_pipeline só para conversa fora de qualquer funil.

3. A instrução era só proibição

"Apply a rule only when its instructions clearly match the current situation.
 Do not move conversations between stages without a matching rule."

Duas negativas seguidas, nenhum critério de quando agir. Numa execução ao vivo o modelo não chamou ferramenta alguma, com as regras corretamente presentes. Agora há um WHEN TO ACT explícito ("acting is expected", "do not wait for the customer to ask"), preservando o guardrail de não inventar estágio e de deixar o card parado quando nada casa.

Estrutura

O bloco saiu para _pipeline_tool_instruction(), para ser testado como texto renderizado em vez de raspado do código-fonte — a primeira versão do teste lia inspect.getsource e quebrava conforme onde a concatenação de strings do Python cortava a linha.

Verificação ao vivo

Container rebuildado, terceiro contato, primeira tentativa:

Joao Pedro | conversa=3 | ANTES: Qualificado
  [1] Qualificado
  [2] >>> MOVEU: Fechado        (~12s)

Antes deste PR, a mesma situação produziu: uma execução sem chamada de tool, e outra com add_to_pipeline + id errado.

Testes

tests/unit/test_pipeline_prompt_guidance.py — 9 examples: critério positivo, escolha da ação, ids fora do alcance do modelo e o schema concordando com o prompt.

Baseline: 290 passed contra 266 no develop limpo — delta +24 (8 do CRM-235 + 7 do CRM-237 + 9 daqui). As 3 falhas e 7 erros de coleta são idênticos ao baseline e pré-existentes.

Sobre a prova negativa, com honestidade: removendo o fix, o arquivo de teste falha na coleta (o import de _pipeline_tool_instruction não resolve), não em asserção. Prova que o teste não passa sem o fix, mas não exercita as asserções contra o texto antigo — diferente das provas negativas do CRM-235 e do CRM-237, que falham por assertion.

Base deste PR

Encadeado em fix/CRM-237-context-ids-win (PR #50), que por sua vez sai do #49. Ordem de merge: #49 → #50 → este.

Trade-off assumido

A instrução ficou mais longa e mais diretiva ("acting is expected"). Isso aumenta a chance de o agente mover o card — que é o objetivo — e, no limite, o risco de mover quando a correspondência é fraca. Mitigado por manter explícito que só valem os estágios listados e que situação sem correspondência deixa o card onde está. Se aparecer movimentação indevida, o ajuste é no texto das instruções por estágio, que é o que o operador controla.

Summary by Sourcery

Improve pipeline agent guidance so it acts on matching stage rules, selects the correct pipeline action, and does not attempt to provide context-derived IDs.

Bug Fixes:

• Align pipeline tool prompts and schemas so conversation and contact IDs are supplied exclusively from conversation context.
• Guide the agent to act when a configured stage rule matches and choose stage moves versus pipeline additions based on whether a card already exists.
• Clarify context-only error messages when pipeline operations are invoked outside a conversation.

Enhancements:

• Extract the rendered pipeline guidance into a reusable, testable instruction block while preserving safeguards against inventing stages or moving unmatched conversations.

Tests:

• Add unit coverage for action guidance, move-versus-add selection, context-owned IDs, and prompt/schema consistency.

[sourcery-ai]

Reviewer's Guide

Refactors the pipeline manipulation agent instruction into a reusable helper, tightens the tool schema and documentation so the model must not provide conversation/contact IDs, and adds tests asserting the rendered prompt text provides clear, positive guidance on when and how to manipulate pipeline cards.

Sequence diagram for pipeline card movement guidance

sequenceDiagram
    participant Context as ConversationContext
    participant Agent as LLM_Agent
    participant Tool as pipeline_manipulation
    participant CRM as CRM
    Context->>Agent: Provide current conversation and pipeline rules
    Agent->>Agent: Match customer message to a configured stage
    alt Existing pipeline card
        Agent->>Tool: pipeline_manipulation(action="move_to_stage", pipeline_id, stage_id)
        Tool->>Context: Read conversation_id and contact_id
        Tool->>CRM: Move current conversation card
    else No pipeline card
        Agent->>Tool: pipeline_manipulation(action="add_to_pipeline", pipeline_id, stage_id)
        Tool->>Context: Read conversation_id and contact_id
        Tool->>CRM: Add current conversation card
    else No matching stage
        Agent-->>Context: Leave card unchanged
    end

Loading

Flow diagram for pipeline tool action selection

flowchart TD
    A[Customer message matches a configured stage] --> B{Conversation already has a card?}
    B -->|Yes| C[Call pipeline_manipulation with action move_to_stage]
    B -->|No| D[Call pipeline_manipulation with action add_to_pipeline]
    C --> E[Provide pipeline_id and stage_id]
    D --> E
    E --> F[conversation_id and contact_id come from context]
    G[No stage matches] --> H[Leave the card where it is]

Loading

File-Level Changes

Change	Details	Files
Extracts and rewrites the pipeline tool prompt block to clearly define when to act, which action to use, and which IDs the model must not provide.	Introduces a new helper _pipeline_tool_instruction(rules_text) that returns the full instruction text for the pipeline manipulation tool, including WHEN TO ACT, WHICH ACTION, and IDS sections plus the formatted rules list. Replaces the previous inline string construction for the pipeline tool instructions in _create_llm_agent with a call to _pipeline_tool_instruction, ensuring the agent prompt uses the new guidance whenever pipeline rules exist.	src/services/adk/agents/llm_agent_builder.py
Aligns the pipeline manipulation tool schema/docstring with the prompt by forbidding the model from setting contact_id and conversation_id and clarifying move/add semantics.	Updates the parameter docs in pipeline_manipulation to mark contact_id and conversation_id as DO NOT SET, noting they are taken from context and any provided value is ignored, and documents the previous bug with conversation_id being filled with the contact ID. Adds high-level guidance in the tool function docstring on when to use move_to_stage vs add_to_pipeline, mirroring the agent prompt expectations.	src/services/adk/tools/evo_crm/pipeline_manipulation.py
Adds tests that lock in the rendered pipeline instruction text and its agreement with the tool schema.	Creates tests/unit/test_pipeline_prompt_guidance.py to construct the rendered prompt via _pipeline_tool_instruction and assert the presence of key guidance phrases (WHEN TO ACT, acting is expected, move vs add, non-invention of stages, and leaving the card in place when no rule matches). Adds a test that inspects the dynamically generated tool docstring from create_pipeline_manipulation_tool() to ensure it contains DO NOT SET for IDs and no longer offers conversation_id as an optional, auto-extracted parameter.	tests/unit/test_pipeline_prompt_guidance.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[sourcery-ai]

Hey - I've found 2 issues

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/services/adk/agents/llm_agent_builder.py" line_range="195-197" />
<code_context>
+        "description below, call the tool with that stage's stage_id. Acting is expected "
+        "in that case — do not wait for the customer to ask to be moved.\n"
+        "WHICH ACTION: use action=\"move_to_stage\" for a conversation that already has a card "
+        "in the pipeline — the normal case for an ongoing conversation. Use "
+        "action=\"add_to_pipeline\" only for a conversation that is not in any pipeline yet; "
+        "if a move fails because the card does not exist, then add it.\n"
+        "IDS: do NOT set conversation_id or contact_id — they come from the conversation "
+        "context and anything you pass is ignored. Provide pipeline_id and stage_id.\n"
</code_context>
<issue_to_address>
**issue (bug_risk):** The instruction explains which action corresponds to an existing or absent card, but it never tells the model whether the current conversation actually has a card. When that status is not present elsewhere in the model context, the model still cannot distinguish `move_to_stage` from `add_to_pipeline` and can choose `add_to_pipeline` for an existing card—the exact failure this change claims to fix.

**Triggers:** When the current conversation's pipeline-card status is not injected into the prompt or tool context visible to the model.

**Suggested fix:** Include the current card/pipeline status in the rendered instruction, or have the tool expose a status before requiring the model to choose between the two actions.
</issue_to_address>

### Comment 2
<location path="src/services/adk/tools/evo_crm/pipeline_manipulation.py" line_range="147-151" />
<code_context>
                    - 'complete_task': Mark a task as completed
-            contact_id: ID of the contact (optional, auto-extracted from context)
-            conversation_id: ID of the conversation (optional, auto-extracted from context)
+            contact_id: DO NOT SET. The contact of the current conversation is
+                   taken from the context; any value passed here is ignored.
+            conversation_id: DO NOT SET. The current conversation is taken from
+                   the context; any value passed here is ignored. (CRM-238: this
+                   field used to read as "auto-extracted from context" while still
+                   being offered as a parameter — the model filled it with the
+                   CONTACT id and the CRM answered 404/400.)
</code_context>
<issue_to_address>
**issue (bug_risk):** The schema still declares `contact_id` and `conversation_id` as callable function parameters, and the implementation still accepts model-supplied values when context metadata is absent. Marking them `DO NOT SET` in the docstring discourages filling them but does not remove them from the schema or prevent a model-generated id from being used outside a populated conversation context.

**Triggers:** When the tool is invoked without `tool_context` ids or with context metadata that does not contain the relevant id.

**Suggested fix:** Remove these ids from the model-facing function schema and obtain them exclusively from context, or reject any non-context value instead of falling back to the model argument.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. 2 findings to address first, and if the guidance is wrong, the agent could add conversations to the wrong pipeline or move cards to an incorrect stage, leaving incorrect CRM state after the prompt is reverted. Those changes are bounded and can be corrected or recomputed, but reverting the code does not undo pipeline mutations already made.

Blocking findings: src/services/adk/agents/llm_agent_builder.py:197, src/services/adk/tools/evo_crm/pipeline_manipulation.py:151

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[sourcery-ai]

issue (bug_risk): The instruction explains which action corresponds to an existing or absent card, but it never tells the model whether the current conversation actually has a card. When that status is not present elsewhere in the model context, the model still cannot distinguish move_to_stage from add_to_pipeline and can choose add_to_pipeline for an existing card—the exact failure this change claims to fix.

Triggers: When the current conversation's pipeline-card status is not injected into the prompt or tool context visible to the model.

Suggested fix: Include the current card/pipeline status in the rendered instruction, or have the tool expose a status before requiring the model to choose between the two actions.

[sourcery-ai]

issue (bug_risk): The schema still declares contact_id and conversation_id as callable function parameters, and the implementation still accepts model-supplied values when context metadata is absent. Marking them DO NOT SET in the docstring discourages filling them but does not remove them from the schema or prevent a model-generated id from being used outside a populated conversation context.

Triggers: When the tool is invoked without tool_context ids or with context metadata that does not contain the relevant id.

Suggested fix: Remove these ids from the model-facing function schema and obtain them exclusively from context, or reject any non-context value instead of falling back to the model argument.