Skip to content

fix(agent): send the pipeline rules' STAGES to the model (CRM-235) - #49

Merged
gomessguii merged 2 commits into
developfrom
fix/CRM-235-pipeline-rules-prompt
Aug 22, 2026
Merged

fix(agent): send the pipeline rules' STAGES to the model (CRM-235)#49
gomessguii merged 2 commits into
developfrom
fix/CRM-235-pipeline-rules-prompt

Conversation

@pastoriniMatheus

@pastoriniMatheus pastoriniMatheus commented Aug 22, 2026

Copy link
Copy Markdown

Problema

O lead fala na conversa, o agente deveria mover o card para o estágio configurado — e nada acontece. Não é o modelo se recusando: as regras de pipeline configuradas no agente nunca chegam ao prompt.

Causa-raiz

O frontend salva (evo-ai-frontend-communityPipelineRules.tsx:17-32):

StageRule    { id; stageId; stageName?; instructions }
PipelineRule { id; pipelineId; pipelineName?; generalInstructions; stages: StageRule[] }

Os estágios ficam dentro de rule.stages.

A tool sempre leu certo (pipeline_manipulation.py:451pipeline_rule.get("stages", [])). O builder do prompt (llm_agent_builder.py:763-773) lia stageName/instructions no nível da regra, onde não existem:

stage_name   = rule.get("stageName") or rule.get("stage_name") or rule.get("stageId") ...
instructions = rule.get("instructions") or rule.get("description") or ""

Um funil com dois estágios renderizava assim (executado, não deduzido):

PROMPT QUE O LLM RECEBE:
- Vendas

stages perdidos: ['Qualificado', 'Fechado']

Sem nomes, sem stageId, e sem as instruções de "quando mover" — que são exatamente o que o operador configura no modal.

Por que isso trava a IA

O mesmo bloco de prompt termina com "do not move conversations between stages without a matching rule". O modelo é instruído a só agir com regra correspondente, e nenhuma chegou.

Comprovado ao vivo (Gemini gemini-2.5-flash), lead dizendo "Fechado, pode gerar o pedido! Quero comprar o plano anual agora.":

// ANTES — prompt continha apenas "- Vendas"
{"stage_name": "Fechamento", "pipeline_id": "Vendas", "action": "move_to_stage"}
//              ^ estágio INVENTADO (os reais são Qualificado/Fechado)
//                                  ^ nome do funil onde vai o id

// DEPOIS
{"pipeline_id": "pipe-1", "stage_id": "stg-fechado", "action": "move_to_stage"}

É pior do que não chamar: com esses argumentos, _move_to_stage procura a regra por pipelineId == "Vendas", não encontra (o id real é pipe-1), fica com stages vazio e responde "stage_id or stage_name is required. Available stages: none". O card não se move e o operador não vê motivo — o erro morre dentro do retorno da tool.

Correção

Extraído _format_pipeline_rules_for_prompt, que percorre rule["stages"] e emite uma linha por estágio, com nome, id e o critério de quando mover, mais as generalInstructions do funil:

- Pipeline: Vendas (pipeline_id: pipe-1) — Funil principal de vendas
    - Stage: Qualificado (stage_id: stg-qualificado) — move here when: quando o lead confirma interesse no produto
    - Stage: Fechado (stage_id: stg-fechado) — move here when: quando o lead confirma a compra

Formatos antigos (estágio no nível da regra) continuam renderizando, para não quebrar config salva por UI anterior.

Testes

tests/unit/test_pipeline_rules_prompt.py8 examples, fixando o formato exato que a UI escreve: cada estágio aparece, os stageId são expostos (sem eles a tool não tem como ser chamada), as instruções por estágio chegam, e o caso da regressão (funil renderizado como uma linha só).

  • Prova negativa: restaurando a leitura antiga no nível da regra, 5 dos 8 falham.
  • Baseline: suíte do processor 274 passed contra 266 no develop limpo — delta de exatamente +8 (os novos). As 3 falhas e 7 erros de coleta são idênticos ao baseline e pré-existentes (test_exception_handlers, mcp_headers_call_path), sem relação com este PR.

Escopo e relação com o CRM-213

evo-ai-processor-community. A tool, o endpoint do CRM (PATCH /pipelines/:id/pipeline_items/:id/move_to_stage, que aceita conversation_id) e a autorização (chamadas de serviço passam por service_authenticated?) já estavam corretos.

É o mesmo modal do CRM-213, mas defeito diferente: aquele fez as regras serem salvas; este faz com que, uma vez salvas, elas cheguem ao modelo. Por isso o CRM-213 sozinho não fez a IA mover o card.

Trade-off assumido

O E2E com o Gemini não entrou na suíte: depende de rede e da resposta do modelo, então falharia no CI por motivo alheio ao código. Rodei uma vez à mão e colei a saída acima e no card. O que protege contra a regressão no CI é o teste determinístico do formato.

Summary by Sourcery

Fix pipeline manipulation prompts so agents receive the configured pipeline stages and can move conversations using valid pipeline and stage identifiers.

Bug Fixes:

  • Ensure configured pipeline stages, IDs, and move criteria are included in the LLM prompt so the agent can select valid stages and move conversations correctly.

Enhancements:

  • Centralize pipeline-rule prompt formatting with support for current nested stage configurations and legacy flat rules.
  • Skip incomplete stages without IDs and retain general pipeline instructions while avoiding misleading empty-rule tool guidance.

Tests:

  • Add unit coverage for nested UI rule formatting, stage IDs and instructions, legacy compatibility, incomplete configurations, and invalid inputs.

Ordem de merge — este é o primeiro de uma cadeia de 3

Os três PRs tocam o mesmo fluxo ("a IA não move o card") em camadas diferentes e foram
abertos encadeados, cada um com base no anterior:

#49 (CRM-235, este) → #50 (CRM-237) → #51 (CRM-238)

Este aqui tem base em develop e deve ser mergeado primeiro. Ao mergear, a base do #50
precisa ser re-apontada para develop (o GitHub costuma fazer isso sozinho quando a branch
de origem é apagada — vale conferir).

Mergear fora de ordem traz os commits dos outros dois junto e o diff deixa de corresponder
ao que cada card descreve.

The lead says something, the agent should move the card, and nothing happens.
Not because the model refuses: the configured rules never reached the prompt.

The frontend saves PipelineRule { pipelineId, pipelineName, generalInstructions,
stages: StageRule[] } with StageRule { stageId, stageName, instructions }
(PipelineRules.tsx:17-32). The prompt builder read stageName/instructions off
the RULE, where they do not exist, so a two-stage funnel rendered as a single
line with the funnel name — no stage names, no stage ids, and none of the
per-stage "when to move" instructions the operator configured. The tool always
read rule["stages"] correctly; only the prompt side was wrong.

Since the same prompt says "do not move conversations between stages without a
matching rule", the model had nothing to match. Live check with Gemini
(gemini-3.5-flash), lead message "Fechado, pode gerar o pedido!":

  before: {"stage_name": "Fechamento", "pipeline_id": "Vendas", ...}
          ^ stage hallucinated (real ones are Qualificado/Fechado), and the
            funnel NAME sent where the id belongs
  after:  {"pipeline_id": "pipe-1", "stage_id": "stg-fechado", ...}

With the hallucinated args _move_to_stage looks the rule up by
pipelineId == "Vendas", finds nothing, and answers "stage_id or stage_name is
required. Available stages: none" — the card stays put and the operator sees
no reason, because the error dies inside the tool's return value.

Extracted _format_pipeline_rules_for_prompt: walks rule["stages"] and emits one
line per stage with name, id and its move-here-when instructions, plus the
funnel's generalInstructions. Legacy flat rules (stage on the rule itself)
still render, so configs saved by an older UI keep working.

Tests: 8 examples in tests/unit/test_pipeline_rules_prompt.py pinning the exact
shape the UI writes. Negative proof: restoring the old rule-level read makes 5
of them fail. Suite: 274 passed vs 266 on the clean develop baseline (+8, the
new ones); the 3 failures and 7 collection errors are identical to the baseline
and pre-existing (test_exception_handlers, mcp_headers_call_path).
@sourcery-ai

sourcery-ai Bot commented Aug 22, 2026

Copy link
Copy Markdown

Reviewer's Guide

Refactors how pipeline rules are formatted into the LLM prompt so that stage-level data (IDs, names, and per-stage move instructions) is correctly included, adds support for legacy flat rule shapes, and introduces targeted unit tests to pin the new prompt contract and guard against regressions.

Sequence diagram for pipeline rules reaching the LLM prompt

sequenceDiagram
    participant Agent as LLM Agent Builder
    participant Config as Agent Config
    participant Formatter as _format_pipeline_rules_for_prompt
    participant Model as LLM
    participant Tool as Pipeline Manipulation Tool

    Agent->>Config: get pipeline_rules
    Config-->>Agent: PipelineRule with stages
    Agent->>Formatter: _format_pipeline_rules_for_prompt(pipeline_rules)
    Formatter-->>Agent: pipeline and stage prompt lines
    Agent->>Model: create prompt with stage_id and move instructions
    Model->>Tool: move_to_stage(pipeline_id, stage_id)
    Tool-->>Model: stage movement result
Loading

File-Level Changes

Change Details Files
Ensure pipeline STAGE rules (IDs, names, and per-stage instructions) are correctly surfaced in the LLM prompt instead of being dropped.
  • Introduce a dedicated _format_pipeline_rules_for_prompt helper that iterates rule['stages'] and emits one prompt line per pipeline and per stage, including pipeline_id, stage_id, and per-stage instructions when present.
  • Preserve support for legacy/flat rule shapes where stage data lives at the rule level, so older saved configurations still render usable prompt lines.
  • Harden formatting against malformed inputs by validating types and skipping non-dict rules or stages while still emitting pipeline headers when possible.
src/services/adk/agents/llm_agent_builder.py
Wire the new formatter into the LLM agent builder and add unit tests to pin the prompt format.
  • Replace the inline pipeline_rules prompt-building logic in _create_llm_agent with the new _format_pipeline_rules_for_prompt helper.
  • Add a focused unit test module to verify that real UI-shaped rules produce prompt text exposing all configured stages, IDs, and instructions, that legacy flat rules still render, and that edge/garbage inputs are handled gracefully.
src/services/adk/agents/llm_agent_builder.py
tests/unit/test_pipeline_rules_prompt.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue

Prompt for AI Agents
Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="src/services/adk/agents/llm_agent_builder.py" line_range="110-112" />
<code_context>
+    ``instructions`` off the rule yielded a prompt with the funnel name and
+    nothing else: no stage names, no stage ids, and none of the per-stage
+    "when to move" instructions the operator configured. Since the prompt also
+    tells the model "do not move conversations between stages without a
+    matching rule", an empty rule set means the model never moves the card —
+    and ``pipeline_manipulation`` answers "stage_id or stage_name is required"
+    when it is called blind. The tool itself always read ``rule["stages"]``
+    correctly; only this prompt side was wrong.
</code_context>
<issue_to_address>
**nitpick:** The new docstring states that an empty rule set means the model never moves the card, but `_create_llm_agent` takes a separate no-rules branch that explicitly tells the model to use the tool when warranted. The comment therefore misdescribes the actual behavior and can lead maintainers to incorrectly change the no-rules path.

**Triggers:** When an agent has no configured pipeline rules but still has pipeline manipulation enabled.

**Suggested fix:** Describe the old failure as the model lacking a matching configured rule, rather than claiming that an empty rule set categorically prevents moves.
</issue_to_address>

Sourcery assessment

Needs a human reviewer. The new prompt can cause the model to move conversations into CRM pipeline stages based on the rendered rules, so an incorrect format or interpretation could leave conversations in wrong stages after the change is reverted. Those assignments are bounded and can be corrected or re-run, but reverting only prevents future moves and does not undo ones already made.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment thread src/services/adk/agents/llm_agent_builder.py Outdated
Code review follow-ups on the same defect, at the edges the fix left open.

A stage with no stageId was still rendered as "- Stage: unnamed stage — move
here when: ...". The UI creates stages with stageId="" (PipelineRules.tsx) and
AgentEditPage saves pipeline_rules verbatim, so an operator who writes the
criteria and forgets to pick the stage in the select produces exactly that.
The model then gets a rule that looks actionable with no id and no name, and
invents a stage_name — the failure this card is about. Such stages are now
skipped with a warning.

The legacy flat branch dropped the stageId when the rule also carried a
stageName, so it rendered the same id-less line the fix removes; and a flat
rule cannot be recovered by name, because the tool resolves a name through
rule["stages"], which a flat rule does not have. It now emits the id, and a
flat rule without one is skipped for the same reason.

When the formatter renders nothing (a pipeline_rules list holding no usable
rule), the prompt used to say "Configured pipeline rules:" followed by an
empty list and then "do not move without a matching rule" — advertising the
tool and disabling it in the same breath. It now falls back to the generic
text the no-rules branch already had.

Comments trimmed to what the code does not say; the bug history lives in the
PR and the card.

Tests: 10 examples (8 kept, 2 added for the skipped shapes, 1 renamed to pin
the id). Suite 294 passed vs 284 on the clean develop baseline (+10); the 2
failures and 1 collection error are identical to the baseline and pre-existing.
@gomessguii
gomessguii merged commit c5b0de4 into develop Aug 22, 2026
5 checks passed
@gomessguii
gomessguii deleted the fix/CRM-235-pipeline-rules-prompt branch August 22, 2026 15:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants