fixed workflow creation

[sauravpanda]

Summary by cubic

Fixes deterministic workflow creation by correctly interpreting browser-use actions and reliably populating selectors, so generated workflows execute without manual fixing.

• Bug Fixes

  • Parse new browser-use action format and box indices to find real DOM elements.
  • Robust target text extraction and element hashing; map index ➝ elementHash.
  • Populate cssSelector/xpath/elementTag from captured DOM data instead of unreliable history.
  • Convert scroll pages to pixel values; support extract_content and click/input mapping.
  • Filter workflow-only fields before controller calls; handle empty actions (go_back/go_forward).
  • Automatic semantic fallback when cssSelector is missing but target_text is present.

• New Features

  • Capture selector map during agent run via a custom controller to enrich steps.
  • Add go_back and go_forward workflow steps with executor support.
  • Direct text-based DOM clicking and smarter selector preference (hierarchical > basic).
  • Workflow version field added to definitions.
  • Docs: switch examples to ChatBrowserUse (bu-latest).

[cubic-dev-ai]

4 issues found across 8 files

Prompt for AI agents (all 4 issues)

Understand the root cause of the following 4 issues and fix them.


<file name="workflows/workflow_use/healing/deterministic_converter.py">

<violation number="1" location="workflows/workflow_use/healing/deterministic_converter.py:84">
Please remove these raw print-based debug statements from the workflow converter or replace them with the project&#39;s logging facility to avoid flooding stdout in production.</violation>
</file>

<file name="workflows/workflow_use/workflow/semantic_executor.py">

<violation number="1" location="workflows/workflow_use/workflow/semantic_executor.py:790">
Triggering the text-based click before the selector logic can hit the wrong element when multiple elements share the same text, because it ignores the resolved selector context.</violation>
</file>

<file name="workflows/workflow_use/workflow/service.py">

<violation number="1" location="workflows/workflow_use/workflow/service.py:120">
Filtering out cssSelector means every deterministic click/input/select/key_press action now instantiates its ActionModel without the required selector, causing immediate validation failures.</violation>
</file>

<file name="workflows/workflow_use/healing/service.py">

<violation number="1" location="workflows/workflow_use/healing/service.py:420">
The captured selector map is stored under the raw selector_map key, which is a string. Later lookups in `_create_workflow_deterministically` use integer indices, so the captured data path never hits and selectors stay empty. Please normalise the key before storing so deterministic workflows can populate selectors reliably.</violation>
</file>

_{React with 👍 or 👎 to teach cubic. Mention @cubic-dev-ai to give feedback, ask questions, or re-run the review.}

[cubic-dev-ai]

Please remove these raw print-based debug statements from the workflow converter or replace them with the project's logging facility to avoid flooding stdout in production.

Prompt for AI agents

Address the following comment on workflows/workflow_use/healing/deterministic_converter.py at line 84:

<comment>Please remove these raw print-based debug statements from the workflow converter or replace them with the project&#39;s logging facility to avoid flooding stdout in production.</comment>

<file context>
@@ -36,46 +38,263 @@ def convert_history_to_steps(self, history_list: AgentHistoryList) -&gt; List[Dict[
+					action_params = action_dict
+
+				# Debug: Log the action type and params
+				print(f&#39;🔍 Processing action type: &quot;{action_type}&quot;&#39;)
+				print(f&#39;   Action params: {action_params}&#39;)
+				reasoning = agent_context.get(&#39;reasoning&#39;)
</file context>

[cubic-dev-ai]

Triggering the text-based click before the selector logic can hit the wrong element when multiple elements share the same text, because it ignores the resolved selector context.

Prompt for AI agents

Address the following comment on workflows/workflow_use/workflow/semantic_executor.py at line 790:

<comment>Triggering the text-based click before the selector logic can hit the wrong element when multiple elements share the same text, because it ignores the resolved selector context.</comment>

<file context>
@@ -664,10 +675,124 @@ async def click_verifier():
 		page = await self.browser.get_current_page()
 
+		# STRATEGY 0: Try direct text-based clicking first (most semantic)
+		if target_text and target_text.strip():
+			element_tag = element_info.get(&#39;tag&#39;, &#39;&#39;).lower() if element_info else None
+			if await self._click_element_by_text_direct(target_text, element_tag):
</file context>

[cubic-dev-ai]

Filtering out cssSelector means every deterministic click/input/select/key_press action now instantiates its ActionModel without the required selector, causing immediate validation failures.

Prompt for AI agents

Address the following comment on workflows/workflow_use/workflow/service.py at line 120:

<comment>Filtering out cssSelector means every deterministic click/input/select/key_press action now instantiates its ActionModel without the required selector, causing immediate validation failures.</comment>

<file context>
@@ -110,7 +110,24 @@ async def _run_deterministic_step(self, step: DeterministicWorkflowStep, step_in
+		workflow_metadata_fields = {
+			&#39;description&#39;, &#39;output&#39;, &#39;agent_reasoning&#39;,
+			&#39;page_context_url&#39;, &#39;page_context_title&#39;,
+			&#39;cssSelector&#39;, &#39;xpath&#39;, &#39;elementTag&#39;, &#39;elementHash&#39;,  # These are workflow-specific selector fields
+			&#39;target_text&#39;, &#39;container_hint&#39;, &#39;position_hint&#39;, &#39;interaction_type&#39;  # Semantic workflow fields
+		}
</file context>

[cubic-dev-ai]

The captured selector map is stored under the raw selector_map key, which is a string. Later lookups in _create_workflow_deterministically use integer indices, so the captured data path never hits and selectors stay empty. Please normalise the key before storing so deterministic workflows can populate selectors reliably.

Prompt for AI agents

Address the following comment on workflows/workflow_use/healing/service.py at line 420:

<comment>The captured selector map is stored under the raw selector_map key, which is a string. Later lookups in `_create_workflow_deterministically` use integer indices, so the captured data path never hits and selectors stay empty. Please normalise the key before storing so deterministic workflows can populate selectors reliably.</comment>

<file context>
@@ -243,27 +306,155 @@ async def generate_workflow_from_prompt(
+								}
+
+							# Store in the shared map
+							element_text_map[index] = element_data
+
+							# Show first few captures for debugging
</file context>