added screenshot for debugging

[sauravpanda]

Summary by cubic

Added per-step debug screenshots and switched extraction steps to a lightweight, LLM-only flow to speed up runs and reduce cost. Also improved semantic targeting and matching for inputs, anchors, and dynamic IDs to make workflows more reliable and reusable. Implements Linear #124 by avoiding agent startup for content extraction.

• New Features

  • Debug mode: saves before/after/error screenshots per step to a configurable folder; optional step wait time.
  • LLM-only extraction for extract and extract_page_content steps (no agent), using clean page markdown.

• Refactors

  • Better target_text derivation: uses agent reasoning tags [ELEMENT: "..."], smarter attribute priority, and normalization of dynamic IDs to generic labels (with position/container hints).
  • Semantic executor: adds pattern matching for IDs/codes, supports label_text for inputs, and treats “Search” as submit; assumes success on same-page updates if no validation errors.
  • Element extraction: pulls label text from table headers/cells and includes it in the mapping.
  • Healing: skips selector population (pure semantic matching); filters JS href “text” and infers common anchor/button labels from id/class.
  • Storage: cleans workflow YAML by dropping verbose/legacy fields and filtering invalid selector strategies.

^{Written for commit ae75a84. Summary will update automatically on new commits.}

[cubic-dev-ai]

3 issues found across 6 files

Prompt for AI agents (all 3 issues)

Understand the root cause of the following 3 issues and fix them.


<file name="workflows/workflow_use/healing/deterministic_converter.py">

<violation number="1" location="workflows/workflow_use/healing/deterministic_converter.py:611">
Guard against agent_context[&#39;reasoning&#39;] being None before calling .lower(); otherwise dynamic-ID clicks crash the converter.</violation>
</file>

<file name="workflows/workflow_use/workflow/semantic_executor.py">

<violation number="1" location="workflows/workflow_use/workflow/semantic_executor.py:381">
Priority 1 of _find_element_by_pattern ignores the requested pattern text and will pick the first ID-looking element even if it is unrelated, so hinted clicks may target the wrong entry. Please filter these matches using the requested pattern (e.g., compare against the actual ID string or required keywords) before appending them.</violation>
</file>

<file name="workflows/workflow_use/storage/service.py">

<violation number="1" location="workflows/workflow_use/storage/service.py:146">
Removing the `output` field here strips valid workflow outputs, so steps can no longer store their results in context; workflows depending on this will fail.</violation>
</file>

_{React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

Guard against agent_context['reasoning'] being None before calling .lower(); otherwise dynamic-ID clicks crash the converter.

Prompt for AI agents

Address the following comment on workflows/workflow_use/healing/deterministic_converter.py at line 611:

<comment>Guard against agent_context[&#39;reasoning&#39;] being None before calling .lower(); otherwise dynamic-ID clicks crash the converter.</comment>

<file context>
@@ -463,11 +587,70 @@ def _convert_action_to_step(
+
+				# Check agent reasoning for context to determine the semantic meaning
+				reasoning = agent_context.get(&#39;reasoning&#39;, &#39;&#39;) if agent_context else &#39;&#39;
+				reasoning_lower = reasoning.lower()
+
+				original_target = target_text
</file context>

[cubic-dev-ai]

Removing the output field here strips valid workflow outputs, so steps can no longer store their results in context; workflows depending on this will fail.

Prompt for AI agents

Address the following comment on workflows/workflow_use/storage/service.py at line 146:

<comment>Removing the `output` field here strips valid workflow outputs, so steps can no longer store their results in context; workflows depending on this will fail.</comment>

<file context>
@@ -123,8 +123,57 @@ def save_workflow(
+					&#39;position_hint&#39;,  # Usually null
+					&#39;interaction_type&#39;,  # Usually null
+					&#39;default_value&#39;,  # Usually null for inputs
+					&#39;output&#39;,  # Usually null
+				]
+				for field in fields_to_remove:
</file context>

[cubic-dev-ai]

No issues found across 7 files