🔎 Share one read profile of File, Glob and Syntax (#762) (#789)

`xmd run` and every `host="run"` child stated one evaluation profile whose
`read` table was the self-closing `<File />` alone. Plan information requests
need a fragment to find files and read about the vocabulary it has, so the
table is now File, Glob and canonical Syntax; the write table is unchanged and
`read` still never reaches a write form.

Glob is admitted as a capability, and admitting it did not create a second
Glob. `src/glob-source.ts` now owns the props, the return contract, the source
rules and the sanitized failure sentence, and both the ordinary component and
the captured capability read them — so an author and an admitted fragment are
held to one dialect and told the same thing about the same pattern. The
capability declares what it binds, which is what makes `as` mandatory before
the host's provider is asked to traverse anything, and it performs its search
through the captured `globFiles` operation and captured working directory
rather than `API.Files` and `Env.cwd`.

Syntax is a component answer at canonical core's own origin, key and revision,
not a capability: a protected name is one core already owns, and a second body
for it would be a second answer to what the name means. Canonical resolution
states core's identity for the exact answer it produced, from the innermost
position, so middleware that returns that answer unchanged leaves the claim on
it and middleware that replaces it returns something carrying no identity.
Core answers for that one protected name — an entry hand-written at
`<Evaluate>`'s canonical identity is refused, because being shown a vocabulary
is a different grant from evaluation inside evaluation.

`FragmentFileAccess` gains `globFiles`, captured, bound and revoked with the
other operations. The workflow profile states it and admits no Glob entry, the
way the ordinary run states `ensureDirectory` and admits no `<Dir>`.

The architecture's complete #762 design is reconciled with the accepted issue
contract ahead of the remaining two PRs. Information requests carry no local
deadline and no rendered-output size limit, so the ten-second deadline, the
65,536-byte findings budget, their refusal messages, `EvaluationLimitError`,
the bounds fingerprint and the bounded streaming meter are removed rather than
left as inactive machinery. What Plan composes around the public throwing
`<Evaluate>` is narrow typed recovery of one namespaced candidate class after
teardown, where a cleanup failure always wins over a retry. PI6, PI10 and PI11
now describe cleanup before the next turn, the recoverable-versus-terminal
split, and every built-in construct with untaken-branch preflight. Delivery is
stated as the agreed three PRs — shared read profile, generated structural
constructs, then the Plan loop — with request-aware Syntax rewriting left to
#787 and a general `<Timeout>` to #788.

Author: taras

architecture.md

@@ -8,11 +8,18 @@
  *
  * ## The two tables
  *
- * `read` is core's self-closing `<File />` and nothing else. `write` is core's
- * paired `<File>…</File>` and self-closing `<File.Delete />`. A fragment run by
- * `xmd run` therefore reaches the caller's own filesystem through the Files
- * provider this command installed, and reaches nothing else at all: no network
- * read, no process, no repository, no Git, no credential and no agent.
+ * `read` is core's self-closing `<File />`, its self-closing `<Glob />` and
+ * canonical `<Syntax />`. `write` is core's paired `<File>…</File>` and
+ * self-closing `<File.Delete />`. A fragment run by `xmd run` therefore reaches
+ * the caller's own filesystem through the Files provider this command
+ * installed, and reaches nothing else at all: no network read, no process, no
+ * repository, no Git, no credential and no agent.
+ *
+ * Reading and writing stay separate spellings of one name. A selection of
+ * `read` admits `<File />` and not `<File>…</File>`, so a fragment that asks to
+ * search and read cannot also write, and `<Syntax />` describing a component
+ * says only what the vocabulary holds — a name it renders is not a name the
+ * selection admits.
  *
  * `<Fetch>` is absent rather than present-and-bounded. An unbounded network
  * read is a decision this command does not make on a document's behalf, and
@@ -32,7 +39,13 @@
  * spelling, so there is no document written against it to keep working.
  */
 
-import { fileDeleteEntry, fileReadEntry, fileWriteEntry } from "@executablemd/core/host";
+import {
+  fileDeleteEntry,
+  fileReadEntry,
+  fileWriteEntry,
+  globReadEntry,
+  syntaxReadEntry,
+} from "@executablemd/core/host";
 import type {
   ExecutionInstallation,
   FragmentEvaluationInput,
@@ -49,15 +62,21 @@ import { cwd, hostFilesHandler } from "@executablemd/runtime";
  * nothing a document, a repository component or middleware installs is between
  * a fragment and this instance.
  *
- * Only the five operations a fragment can name are copied across. The handler
- * also offers globbing and temporary directories; an admitted fragment has
- * neither, and not copying them is what makes that true.
+ * Only the six operations a fragment can name are copied across. The handler
+ * also offers temporary directories; an admitted fragment has none, and not
+ * copying it is what makes that true.
  */
 function ordinaryFiles(): FragmentFileAccess {
   const handler = hostFilesHandler();
   return {
     checkFilePath: (input) => handler.checkFilePath(input),
     readTextFile: (input) => handler.readTextFile(input),
+    globFiles: (input) =>
+      handler.globFiles({
+        cwd: input.cwd,
+        include: [...input.include],
+        exclude: [...input.exclude],
+      }),
     writeTextFile: (input) => handler.writeTextFile(input),
     deleteFile: (input) => handler.deleteFile(input),
     ensureDirectory: (input) => handler.ensureDirectory(input),
@@ -72,7 +91,7 @@ function ordinaryFiles(): FragmentFileAccess {
 /** The ceiling `xmd run` and its run children state. */
 export function ordinaryEvaluationProfile(): FragmentEvaluationInput {
   return {
-    read: [fileReadEntry()],
+    read: [fileReadEntry(), globReadEntry(), syntaxReadEntry()],
     write: [fileWriteEntry(), fileDeleteEntry()],
     files: ordinaryFiles(),
   };

packages/cli/src/evaluation-profile.ts

@@ -74,16 +74,35 @@ describe("Tier FE — the ordinary run profile", () => {
     );
   });
 
+  it("FE21: a fragment searches and reads through the profile's own operations", function* () {
+    yield* useWorkspace(
+      {
+        "notes.md": NOTE,
+        "doc.md":
+          `<Evaluate text={'<Glob include={["*.md"]} as="found" />\\n<Json value={found} />\\n'} ` +
+          `as="answer" />\n\n<Json value={answer} />\n`,
+      },
+      function* (dir) {
+        const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join();
+        expect(result.code).toBe(0);
+        expect(result.stdout).toContain("notes.md");
+      },
+    );
+  });
+
   it("FE21: a fragment cannot reach an operation the ordinary profile withheld", function* () {
     yield* useWorkspace(
       {
         "notes.md": NOTE,
-        "doc.md": `<Evaluate text={'<Glob pattern="*.md" />\\n'} />\n`,
+        "doc.md": `<Evaluate text={'<Fetch url="https://example.test/notes" />\\n'} />\n`,
       },
       function* (dir) {
         const result = yield* runCli(["run", join(dir, "doc.md")], { cwd: dir }).join();
-        // `<Glob>` is an ordinary component of this run and is not a name the
-        // fragment has: the ceiling is the profile's, not the document's.
+        // `<Fetch>` is an ordinary component of this run and is not a name the
+        // fragment has: the ceiling is the profile's, not the document's. An
+        // unbounded network read is a decision `xmd run` does not make on a
+        // document's behalf, so the identity is absent rather than admitted
+        // with an empty ceiling.
         expect(result.code).not.toBe(0);
         expect(`${result.stdout}${result.stderr}`).toContain("did not admit");
       },

packages/cli/tests/evaluate-component.test.ts

@@ -408,6 +408,53 @@ describe("nested execution under the production run host", () => {
     expect(ordinary.stdout).not.toContain("No long-form documentation");
   });
 
+  /**
+   * The run profile's read table reaches a child, whichever command hosts it.
+   *
+   * `host="run"` means the run profile, and the evaluation ceiling is part of
+   * it: a child of `xmd test` evaluates a generated fragment under exactly what
+   * `xmd run` states. So the same three read entries are here — the file, the
+   * search and the vocabulary — and the write table is still not what `read`
+   * selects.
+   */
+  it("gives a run child the ordinary read profile: File, Glob and Syntax", function* () {
+    const project = yield* useProject({
+      "notes.md": doc("the retained note"),
+      "reader.md": doc(
+        `<Evaluate text={'<Glob include={["notes.md"]} as="paths" />\\n` +
+          `<File path="notes.md" as="note" />\\n` +
+          `<Json value={{ paths, note }} />\\n'} as="answer" />`,
+        "",
+        "<Json value={answer} />",
+      ),
+      "writer.md": doc(
+        `<Evaluate text={'<File path="made.md">written by the fragment</File>\\n'} ` +
+          `allow={["read"]} />`,
+      ),
+      "README.md": doc(
+        '<Test name="run child reads">',
+        '<Execution host="run" target="./reader.md" as="child">',
+        '<CollectOutput as="output" />',
+        "",
+        '<AssertStringIncludes actual={output} expected="notes.md" />',
+        '<AssertStringIncludes actual={output} expected="the retained note" />',
+        "</Execution>",
+        "</Test>",
+        "",
+        '<Test name="run child cannot write under read">',
+        '<Execution host="run" target="./writer.md" as="child">',
+        "<AssertEquals actual={child.result.ok} expected={false} />",
+        "</Execution>",
+        "</Test>",
+      ),
+    });
+    const result = yield* runCli(["test", "README.md"], { cwd: project }).join();
+    expect(result.code).toBe(0);
+    // The refused write left nothing behind, which is what "before any
+    // operation" means for a selection that never held the write table.
+    expect(yield* readdir(project)).not.toContain("made.md");
+  });
+
   it("refuses <Execution> outside a canonical <Test>", function* () {
     const project = yield* useProject({
       "child.md": doc("child"),

packages/cli/tests/testing-execution-host.test.ts

@@ -119,6 +119,7 @@ export type {
   FragmentFetchAccess,
   FragmentFileAccess,
   FragmentPath,
+  FragmentSearch,
   FragmentWrite,
 } from "./src/fragment-capabilities.ts";
 /**
@@ -136,8 +137,10 @@ export { detachHeaders, detachStatus } from "./src/fetch-response.ts";
  *
  * Constructors rather than a table a host assembles from `CORE_REGISTRY`,
  * because each of them states a constraint the registry does not hold: which
- * spelling of `<File>` is being admitted, and that admitting `<Fetch>` requires
- * the exact requests it may perform.
+ * spelling of `<File>` is being admitted, that admitting `<Fetch>` requires the
+ * exact requests it may perform, and that `<Syntax />` is admitted as the
+ * answer canonical resolution gives for it rather than as a body core supplies
+ * a second time.
  */
 export {
   directoryEntry,
@@ -146,6 +149,8 @@ export {
   fileDeleteEntry,
   fileReadEntry,
   fileWriteEntry,
+  globReadEntry,
+  syntaxReadEntry,
 } from "./src/evaluation-profile.ts";
 /**
  * The symbols a host's profile describes, when they are not the ones the

packages/core/host.ts

@@ -14,7 +14,10 @@
  * segment is `..` — is refused rather than quietly contributing nothing: an
  * empty result is the answer to "there are no such files", and it must not also
  * be the answer to "that pattern was a mistake". The same stage refuses an
- * empty pattern, which matches nothing by construction.
+ * empty pattern, which matches nothing by construction. Those rules and the
+ * sanitized sentence a failed search produces are `glob-source.ts`, shared with
+ * the Glob capability an evaluation profile admits, so one document and one
+ * generated fragment are held to one dialect.
  *
  * Everything else about matching belongs to the `API.Files` provider, which is
  * the dialect and owns the whole search. This component adds no syntax of its
@@ -43,29 +46,17 @@
 import type { Operation } from "effection";
 import { printErrors } from "../component-failures.ts";
 import { cwd, parseFilesFailure } from "@executablemd/runtime";
-import type { FilesFailureData } from "@executablemd/runtime";
 import { globFiles } from "../files.ts";
+import { GLOB_PROPS, GLOB_RETURNS, globFailure, globPatterns } from "../glob-source.ts";
 import type { Json } from "../types.ts";
-import { reason } from "./fs-error-phrases.ts";
 
-export const props = {
-  type: "object",
-  properties: {
-    include: { type: "array", items: { type: "string" }, minItems: 1 },
-    exclude: { type: "array", items: { type: "string" }, default: [] },
-  },
-  required: ["include"],
-  additionalProperties: false,
-};
+export const props = GLOB_PROPS;
 
 /**
  * Declaring `returns` is what makes this a value component: the engine requires
  * `as`, renders nothing, and validates what comes back (§6.10).
  */
-export const returns = {
-  type: "array",
-  items: { type: "string" },
-};
+export const returns = GLOB_RETURNS;
 
 /** A pattern that cannot be used, or a directory that cannot be searched. */
 export class GlobError extends Error {
@@ -76,98 +67,24 @@ export class GlobError extends Error {
 }
 
 export default printErrors(function* (props: Record<string, Json>): Operation<string[]> {
-  const include = patterns("include", props.include);
-  const exclude = patterns("exclude", props.exclude);
-
-  const found = yield* globFiles({ cwd: yield* cwd(), include, exclude });
-  if (!found.ok) {
-    throw failed(parseFilesFailure(found.error), [...include, ...exclude]);
-  }
-  return found.value;
-});
-
-/**
- * The patterns a prop holds, checked for the two things that make one unusable.
- *
- * Prop validation has already established an array of strings, so the shape is
- * re-read rather than asserted (`as` would claim it instead) and a value that
- * somehow is not one contributes nothing. What validation cannot express is
- * *meaning*: patterns match paths relative to `Env.cwd`, so an absolute pattern
- * and one that starts by leaving cannot match anything a search can produce.
- *
- * Only a whole `..` first segment leaves. `..notes.md` is an ordinary name, and
- * a `..` further along — `docs/../*.md` — is a path the search never generates,
- * so it matches nothing for the ordinary reason and needs no special refusal.
- */
-function patterns(prop: string, value: Json | undefined): string[] {
-  if (!Array.isArray(value)) {
-    return [];
+  const include = globPatterns("include", props.include);
+  if (!include.ok) {
+    throw new GlobError(include.error.message);
   }
-
-  const found: string[] = [];
-  for (const pattern of value) {
-    if (typeof pattern !== "string") {
-      continue;
-    }
-    if (pattern.length === 0) {
-      throw new GlobError(
-        `${prop} holds an empty pattern, which matches nothing; ` +
-          "give a pattern relative to the working directory.",
-      );
-    }
-    if (absolute(pattern)) {
-      throw new GlobError(
-        `${prop} pattern "${pattern}" is absolute; ` +
-          "give a pattern relative to the working directory.",
-      );
-    }
-    if (pattern === ".." || pattern.startsWith("../")) {
-      throw new GlobError(`${prop} pattern "${pattern}" reaches outside the working directory.`);
-    }
-    found.push(pattern);
+  const exclude = globPatterns("exclude", props.exclude);
+  if (!exclude.ok) {
+    throw new GlobError(exclude.error.message);
   }
-  return found;
-}
 
-/**
- * Whether a pattern names an absolute location.
- *
- * Decided from the pattern's own grammar rather than the running platform's.
- * Patterns match POSIX-relative paths on every host — that is what makes one
- * document mean one thing everywhere — so a leading `/` is absolute wherever
- * this runs, and so is a drive-letter prefix, which is absolute on the host that
- * has drives and matches nothing on the hosts that do not. A leading backslash
- * is left alone: in this dialect it escapes the character after it.
- */
-function absolute(pattern: string): boolean {
-  return pattern.startsWith("/") || /^[A-Za-z]:[\\/]/.test(pattern);
-}
-
-/**
- * One sanitized sentence for a failed search.
- *
- * The two questions an author can act on are separated from the rest. A working
- * directory that is missing or is a file is something about the document's own
- * environment; a pattern the dialect cannot compile — an unterminated character
- * class — is something about the document's own text. Which pattern it was does
- * not survive the provider boundary, so the sentence lists the candidates
- * instead of naming one. They are the document's own text.
- *
- * Everything else names no path. What failed is the working directory or
- * something under it, and both are absolute paths the document did not write
- * (§1.2).
- */
-function failed(data: FilesFailureData | undefined, candidates: string[]): GlobError {
-  if (data?.phase === "target" && data.reason === "missing") {
-    return new GlobError("the working directory does not exist.");
-  }
-  if (data?.phase === "target" && data.reason === "not-directory") {
-    return new GlobError("the working directory is not a directory.");
-  }
-  if (data?.phase === "pattern") {
-    return new GlobError(
-      `one of these patterns cannot be used: ${candidates.map((p) => `"${p}"`).join(", ")}.`,
+  const found = yield* globFiles({
+    cwd: yield* cwd(),
+    include: include.value,
+    exclude: exclude.value,
+  });
+  if (!found.ok) {
+    throw new GlobError(
+      globFailure(parseFilesFailure(found.error), [...include.value, ...exclude.value]),
     );
   }
-  return new GlobError(`cannot search the working directory: ${reason(data?.reason)}.`);
-}
+  return found.value;
+});