Support structured content results from custom tools

[astrobot-houston]

Originally proposed by @chroxify in #161. Their description is reproduced verbatim below.

All community-submitted pull requests are automatically converted to issues (bugs) & discussions (feature requests, enhancements) where they can be triaged and prioritized. Once prioritized, a PR implementation is created automatically.

Allows custom tools to return either the existing string result or a native AgentToolResult, preserving text/image content blocks instead of forcing all custom tool output into text

---

Original implementation from #161 by @chroxify

[astrobot-houston]

Possibly related contribution from @sepehr500 — opened #255 which appears to be addressing this.

Their summary: Widens the ToolDefinition.execute return type so custom tools defined via defineTool can return structured content blocks (text and image) instead of only plain strings. The session layer now passes these content blocks through to pi-agent-core directly rather than wrapping every result as text. Includes test coverage and API documentation updates.

Implementation: sepehr500/flue@mizan/custom-tool-image-results (diff)

This comment was posted automatically when #255 was redirected. The implementation is preserved on the branch above so it can inform the work here.

[sepehr500]

Hello! I made #255 as a small tweak to allow tools to return images. session.prompt already supported images as well as pi so this was just a small tweak to allow for the capability. In the mean time i'm doing a patch script locally. Not having this prevents the agent from doing robust design work or verifying visual output.

[viktortnk]

Ideally not only text/images, but Pi-like support for details and terminate as well would be useful.

/** Final or partial result produced by a tool. */
export interface AgentToolResult<T> {
	/** Text or image content returned to the model. */
	content: (TextContent | ImageContent)[];
	/** Arbitrary structured details for logs or UI rendering. */
	details: T;
	/**
	 * Hint that the agent should stop after the current tool batch.
	 * Early termination only happens when every finalized tool result in the batch sets this to true.
	 */
	terminate?: boolean;
}