Skip to content
This repository was archived by the owner on Jul 27, 2026. It is now read-only.
6 changes: 4 additions & 2 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -179,14 +179,16 @@ you are running inside of — NOT the automation backend.
- **State isolation**: `OH_CANVAS_SAFE_STATE_DIR=.tmp/mock-llm-state` isolates test state from the user's real `~/.openhands/agent-canvas/` directory. Both `STATE_DIR` (`.tmp/mock-llm-state`) and the automation DB dir (`.tmp/automation/`) are cleaned before each test run — the automation DB now lives outside STATE_DIR at `dirname(STATE_DIR)/automation/automations.db`, mirroring Docker's `~/.openhands/automation/automations.db`.
- **Session API key**: A random key is generated per test run and passed to the stack via `SESSION_API_KEY` / `OH_SESSION_API_KEYS_0` / `VITE_SESSION_API_KEY`. The static server injects it into `index.html` at serve time so the frontend authenticates automatically.
- **Mock LLM server** (`tests/e2e/mock-llm/scripts/mock-llm-server.py`): Python HTTP server using openhands-sdk's `TestLLM` to return scripted tool-call + text trajectories. Supports admin API endpoints for dynamic trajectory management:
- `POST /admin/reset` — reset to the default trajectory (terminal printf + text reply)
- `POST /admin/reset` — reset to the default trajectory (terminal printf + text reply); also clears the stored completion-request history
- `POST /admin/trajectory/register` — register a named trajectory (JSON body: `{name, turns}` where each turn is `{tool_call: {name, arguments}}` or `{text: "..."}`)
- `POST /admin/trajectory/activate` — activate a previously registered trajectory
- `GET /admin/requests` — return the list of all `/v1/chat/completions` request bodies captured since the last reset (used by the image-upload test to verify the image was forwarded to the LLM)
- **Real automation backend**: The automation test uses the production automation backend (started by `bin/agent-canvas.mjs`), NOT a mock server. Terminal `curl` commands from the agent hit the automation API through the ingress proxy at the test's `BACKEND_URL` (default `http://localhost:18300`). Auth uses the `X-Session-API-Key` header matching the stack's session key. The `mock-automation-server.py` file still exists as a reference but is not used by current tests.
- **Test helpers** (`tests/e2e/mock-llm/utils/mock-llm-helpers.ts`): Exports `registerTrajectory()`, `activateTrajectory()`, `resetMockLLM()`, `ensureMockLLMProfile()`, etc.
- **Test helpers** (`tests/e2e/mock-llm/utils/mock-llm-helpers.ts`): Exports `registerTrajectory()`, `activateTrajectory()`, `resetMockLLM()`, `ensureMockLLMProfile()`, `getMockLLMRequests()` (fetches captured completion bodies from `GET /admin/requests`), `IMAGE_REPLY_TOKEN` + `MINIMAL_PNG_BASE64` (constants for the image-upload spec), and more.
- **Padding response for internal LLM call**: The agent-server makes an internal LLM call (condenser/skill-analysis) before the agent's main loop starts when skills are activated. This consumes one trajectory response. Automation tests prepend a throwaway `{ text: "" }` response as padding. The conversation test does NOT need this because its user message doesn't trigger skill activation.
- **Test specs**:
- `mock-llm-conversation.spec.ts` — Creates LLM profile via UI, runs a conversation with a terminal tool call, verifies bash execution and agent reply.
- `mock-llm-image-upload.spec.ts` — Attaches a 1×1 PNG via the hidden file input, sends "What is in this image?", verifies the agent replies, that the user message event stores image_urls, and that the mock LLM received an image_url content block with a base64 data: URL in at least one completion call.
- `mock-llm-automation.spec.ts` — Full automation lifecycle: registers a trajectory (7 responses total — 4 for the main conversation + 3 for the automation run's spawned conversation) where the LLM creates a cron automation and dispatches a run via terminal `curl` commands to the real automation backend. Verifies: automation created with correct schedule, run reaches COMPLETED status with a conversation_id, automation appears on the `/automations` list page, detail page shows COMPLETED badge (`data-testid="run-status-icon-completed"`), and clicking the run's conversation link navigates to the correct `/conversations/{id}` page.
- `mock-llm-partial-stack.spec.ts` — Partial stack mode tests. Unlike other specs, these spawn their own `bin/agent-canvas.mjs` child processes instead of relying on the config's webServer entries. Three describe blocks: (1) `--frontend-only` verifies static frontend is served (200 on `/`), backend routes return 503 (`/server_info`, `/api/settings`, `/api/automation/v1`), and the browser shows the manage-backends modal; (2) `--backend-only` verifies `/server_info` returns 200, `/api/settings` is reachable, automation endpoint works, and root/asset requests return 503; (3) port conflict verifies the process exits non-zero with a clear error message when the ingress port is occupied, then starts successfully on a free port. Each test uses isolated state dirs and high port numbers (18310+ range) to avoid collisions with the main full-stack instance.
- Tests run serially (`workers: 1`, `mode: "serial"` per describe block). Files are discovered alphabetically so automation tests run before conversation tests; each spec is self-contained (automation test configures its own LLM profile via the settings API). The `afterEach` hook resets the mock LLM to its default trajectory so subsequent specs start fresh even when a preceding test fails.
Expand Down
82 changes: 82 additions & 0 deletions __tests__/api/agent-server-conversation-service.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,9 @@ describe("AgentServerConversationService", () => {
);
return response.data;
},
// @spec WUP-001 — createConversation resolves relative working dirs
// via FileClient.getHome before sending the conversation-start payload.
getHome: async () => ({ home: "/Users/agent" }),
});
mockSettingsClient.mockReturnValue({
listSecrets: vi.fn().mockResolvedValue({ secrets: [] }),
Expand Down Expand Up @@ -284,6 +287,85 @@ describe("AgentServerConversationService", () => {
`/state/workspaces/${secondHex}`,
);
});

// @spec WUP-001 — When the default working_dir is relative, the
// conversation-start payload must be anchored against the agent-server
// home dir so the worktree and later file uploads agree on a writable
// absolute path.
it("resolves relative default working dirs against /api/file/home", async () => {
const { buildConversationWorkingDir: mockedBuilder } =
await import("#/api/agent-server-config");
vi.mocked(mockedBuilder).mockImplementationOnce(
(id: string) => `workspace/project/${id.replace(/-/g, "")}`,
);
const { clearAgentServerHomeDirCache } =
await import("#/api/agent-server-home");
clearAgentServerHomeDirCache();

mockGetSettings.mockResolvedValue({
agent_settings: { llm: { model: "gpt-4o" } },
conversation_settings: {},
});
mockGetSettingsForConversation.mockResolvedValue({
agentSettings: { llm: { model: "gpt-4o" } },
conversationSettings: {},
secretsEncrypted: true,
});
mockHttpPost.mockResolvedValue({
data: {
id: "ignored-server-id",
created_at: "2024-01-01",
updated_at: "2024-01-01",
},
});

await AgentServerConversationService.createConversation();

const [payloadCall] = mockHttpPost.mock.calls;
const payload = payloadCall[1] as {
conversation_id: string;
workspace: { working_dir: string };
};
const hex = payload.conversation_id.replace(/-/g, "");
expect(payload.workspace.working_dir).toBe(
`/Users/agent/workspace/project/${hex}`,
);
});

// @spec WUP-001 — User-supplied workspace overrides are already absolute
// (they come from `search_subdirs`), so they must pass through verbatim.
it("leaves an absolute workingDirOverride untouched", async () => {
mockGetSettings.mockResolvedValue({
agent_settings: { llm: { model: "gpt-4o" } },
conversation_settings: {},
});
mockGetSettingsForConversation.mockResolvedValue({
agentSettings: { llm: { model: "gpt-4o" } },
conversationSettings: {},
secretsEncrypted: true,
});
mockHttpPost.mockResolvedValue({
data: {
id: "ignored-server-id",
created_at: "2024-01-01",
updated_at: "2024-01-01",
},
});

await AgentServerConversationService.createConversation(
undefined,
undefined,
undefined,
undefined,
"/Users/jane/projects/foo",
);

const [payloadCall] = mockHttpPost.mock.calls;
const payload = payloadCall[1] as {
workspace: { working_dir: string };
};
expect(payload.workspace.working_dir).toBe("/Users/jane/projects/foo");
});
});

describe("downloadConversation local branch", () => {
Expand Down
49 changes: 47 additions & 2 deletions __tests__/api/conversation-file-upload.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -7,15 +7,23 @@ import {
} from "#/api/backend-registry/active-store";
import type { Backend } from "#/api/backend-registry/types";
import { uploadFilesToConversation } from "#/api/conversation-file-upload.api";
import { clearAgentServerHomeDirCache } from "#/api/agent-server-home";

const fileUploadMock = vi.fn();
const getHomeMock = vi.fn();

vi.mock("@openhands/typescript-client/workspace/remote-workspace", () => ({
RemoteWorkspace: vi.fn(function RemoteWorkspaceMock() {
return { fileUpload: fileUploadMock };
}),
}));

vi.mock("@openhands/typescript-client/clients", () => ({
FileClient: vi.fn(function FileClientMock() {
return { getHome: getHomeMock };
}),
}));

const batchGetCloudConversations = vi.fn();

vi.mock("#/api/cloud/conversation-service.api", () => ({
Expand All @@ -40,11 +48,16 @@ describe("uploadFilesToConversation", () => {
vi.clearAllMocks();
window.localStorage.clear();
__resetActiveStoreForTests();
clearAgentServerHomeDirCache();
fileUploadMock.mockResolvedValue(undefined);
batchGetCloudConversations.mockReset();
getHomeMock.mockReset();
getHomeMock.mockResolvedValue({ home: "/Users/test" });
});

it("uploads local conversations through the bundled agent-server host", async () => {
// @spec WUP-001 — Default-fallback relative working dirs are resolved
// against /api/file/home, not the filesystem root.
it("uploads local conversations under the agent-server home dir when working dir is relative", async () => {
setRegisteredBackends([
{
id: "local-1",
Expand All @@ -62,10 +75,42 @@ describe("uploadFilesToConversation", () => {

expect(fileUploadMock).toHaveBeenCalledWith(
expect.objectContaining({ name: "a.txt" }),
"/workspace/project/a.txt",
"/Users/test/workspace/project/a.txt",
);
expect(result.uploaded_files).toEqual(["a.txt"]);
expect(batchGetCloudConversations).not.toHaveBeenCalled();
expect(getHomeMock).toHaveBeenCalled();
});

// @spec WUP-001 — Absolute working dirs (e.g. the conversation's own
// `workspace.working_dir`) pass through without a /api/file/home round-trip.
it("respects an absolute conversation working_dir verbatim", async () => {
setRegisteredBackends([
{
id: "local-1",
name: "Local",
host: "http://127.0.0.1:18000",
apiKey: "local-key",
kind: "local",
},
]);
setActiveSelection({ backendId: "local-1" });

const result = await uploadFilesToConversation(
"conv-1",
[makeFile("notes.md")],
{
id: "conv-1",
workspace: { working_dir: "/Users/test/projects/foo" },
} as never,
);

expect(fileUploadMock).toHaveBeenCalledWith(
expect.objectContaining({ name: "notes.md" }),
"/Users/test/projects/foo/notes.md",
);
expect(result.uploaded_files).toEqual(["notes.md"]);
expect(getHomeMock).not.toHaveBeenCalled();
});

it("uploads cloud conversations against the provisioned runtime URL", async () => {
Expand Down
19 changes: 16 additions & 3 deletions __tests__/api/conversation-service.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,23 @@ import { describe, expect, it, vi, beforeEach } from "vitest";
import { RemoteWorkspace } from "@openhands/typescript-client/workspace/remote-workspace";

import ConversationService from "#/api/conversation-service/conversation-service.api";
import { clearAgentServerHomeDirCache } from "#/api/agent-server-home";

const fileUploadMock = vi.fn();
const getHomeMock = vi.fn();

vi.mock("@openhands/typescript-client/workspace/remote-workspace", () => ({
RemoteWorkspace: vi.fn(function RemoteWorkspaceMock() {
return { fileUpload: fileUploadMock };
}),
}));

vi.mock("@openhands/typescript-client/clients", () => ({
FileClient: vi.fn(function FileClientMock() {
return { getHome: getHomeMock };
}),
}));

function makeFile(name: string) {
return new File(["content"], name, { type: "text/plain" });
}
Expand All @@ -19,9 +27,14 @@ describe("ConversationService", () => {
beforeEach(() => {
vi.clearAllMocks();
ConversationService.setCurrentConversation(null);
clearAgentServerHomeDirCache();
getHomeMock.mockResolvedValue({ home: "/Users/agent" });
});

describe("uploadFiles", () => {
// @spec WUP-001 — The default fallback working dir is relative
// (`workspace/project`); the upload path is resolved against the
// agent-server's home directory via /api/file/home.
it("uploads files through RemoteWorkspace and reports successes", async () => {
fileUploadMock.mockResolvedValue(undefined);

Expand All @@ -38,11 +51,11 @@ describe("ConversationService", () => {
);
expect(fileUploadMock).toHaveBeenCalledWith(
expect.objectContaining({ name: "a.txt" }),
"/workspace/project/a.txt",
"/Users/agent/workspace/project/a.txt",
);
expect(fileUploadMock).toHaveBeenCalledWith(
expect.objectContaining({ name: "b.txt" }),
"/workspace/project/b.txt",
"/Users/agent/workspace/project/b.txt",
);
expect(result).toEqual({
uploaded_files: ["a.txt", "b.txt"],
Expand All @@ -59,7 +72,7 @@ describe("ConversationService", () => {

expect(fileUploadMock).toHaveBeenCalledWith(
expect.objectContaining({ name: "../../evil.txt" }),
"/workspace/project/evil.txt",
"/Users/agent/workspace/project/evil.txt",
);
expect(result).toEqual({
uploaded_files: ["evil.txt"],
Expand Down
Loading
Loading