Add --allowed-extensions and --append-only flags; make note writes atomic and serialized - #162
Add --allowed-extensions and --append-only flags; make note writes atomic and serialized#162ErycM wants to merge 3 commits into
Conversation
Expose PathFilters existing allowedExtensions config through the CLI so users can permit non-default file types (e.g. .html, .csv, .json) on read/write without recompiling. Additive to the built-in defaults; createServer is unchanged - server.ts injects a preconfigured PathFilter. Arg parsing is extracted into src/cli.ts (parseAllowedExtensions, stripKnownFlags) so it is unit-testable without importing server.ts, which starts the server on import. Adds src/cli.test.ts and documents the flag in the README.
write_note and patch_note read-modify-write a note and write it back with a plain writeFile. Two gaps remain after bitbonsai#98 (which fixed the r+ length bug): - Not crash-safe: writeFile truncates then writes, so a crash or a stalled network-backed mount (rclone/cloud) mid-write leaves a truncated file. - Not concurrency-safe: two overlapping write_note/patch_note calls to the same path interleave and lose each others changes. Add atomicWrite (write a temp sibling, then rename over the target - atomic on the same filesystem) and withPathLock (serialize mutations per absolute path). writeNote and patchNote run their read-modify-write under the lock; all in-place note writes (writeNote, patchNote, updateFrontmatter, manageTags) go through atomicWrite. Adds src/atomic-write.test.ts (concurrent-append integrity, temp-file cleanup).
Some vault files are appended by multiple clients (e.g. an operations log). A client that reads such a file and then calls write_note with mode:overwrite silently clobbers entries another client appended after that read - and write_note defaults to overwrite, so an LLM that forgets the mode can wipe the file wholesale. Add an opt-in --append-only=<name,...> flag, threaded CLI -> createServer -> FileSystemService. For the listed basenames, mode:overwrite is refused with a message pointing to append/prepend (which re-read the file server-side under the per-path lock). Off by default, so existing behavior is unchanged. Parsing lives in src/cli.ts; adds src/append-only.test.ts, cli.test.ts cases, and a README entry.
|
Hey @ErycM, thanks for putting these together. Atomic writes are especially aligned with the project's data-safety goals, and the two opt-in controls are worth reviewing too. Could you split this into three PRs, one for allowed extensions, one for atomic serialized writes, and one for append-only protection? They have different safety and product trade-offs, and keeping them independent means each can be reviewed and merged without coupling the others. Please rebase each on current main, include its focused tests, and rebuild |
|
Done — split into three independent PRs, each rebased on current
Three notes, repeated in the individual PR bodies:
Closing this in favor of the three. |
Three independent changes, one per commit, so each can be taken or dropped on its own.
1.
feat: add --allowed-extensions CLI flagExposes
PathFilters existingallowedExtensionsconfig through the CLI, so users can permit non-default file types (.html,.csv,.json, ...) on read/write without recompiling. Additive to the built-in defaults;createServeris unchanged (server.ts injects a preconfiguredPathFilter). Arg parsing is extracted into a unit-testedsrc/cli.ts.2.
fix: make note writes atomic and serializedComplements #98 (which fixed the
r+length bug).writeFilestill truncates-then-writes, so two gaps remain:write_note/patch_notecalls to the same path interleave and lose each others changes.Adds
atomicWrite(write a temp sibling, thenrenameover the target) andwithPathLock(per-path serialization). All in-place note writes go throughatomicWrite;write_note/patch_noteadditionally run their read-modify-write under the lock.3.
feat: add --append-only flag to protect files from overwriteOpt-in
--append-only=<name,...>: for the listed basenames,write_notemode:overwriteis refused (clients mustappend/prepend).write_notedefaults to overwrite, so an AI client that forgets the mode can wipe an append-heavy file (e.g. an operations log) with a stale-read overwrite. Off by default, so existing behavior is unchanged.Tests
Adds
src/cli.test.ts,src/atomic-write.test.ts,src/append-only.test.ts(13 tests). Full suite passes locally on Node 20 exceptsrc/shutdown.test.ts > "exits on SIGTERM", which is timing-sensitive and unrelated to these changes: its 500ms boot wait races handler registration on a slow host (verified a 2s boot exits code 0).dist/is intentionally not rebuilt in these commits — happy to rebuild it into the PR if you prefer.