Skip to content

feat: add modusgraph-gen code generator with private field support and accessor name overrides#13

Open
mlwelles wants to merge 165 commits intomatthewmcneely:mainfrom
mlwelles:feature/add-modusgraphgen
Open

feat: add modusgraph-gen code generator with private field support and accessor name overrides#13
mlwelles wants to merge 165 commits intomatthewmcneely:mainfrom
mlwelles:feature/add-modusgraphgen

Conversation

@mlwelles
Copy link
Copy Markdown

@mlwelles mlwelles commented Mar 25, 2026

Summary

Adds modusgraph-gen, a code generation tool that reads Go structs with json/dgraph tags and produces typed CRUD clients, query builders, iterators, and a CLI — with full private field support and customizable accessor names.

Core Generator

Output Purpose
client_gen.go Typed Client with per-entity sub-clients
page_options_gen.go First(n) / Offset(n) pagination helpers
iter_gen.go Auto-paging SearchIter / ListIter (Go 1.23+ iter.Seq2)
<entity>_gen.go Per-entity CRUD: Get, Add, Update, Delete, Search, List
<entity>_options_gen.go Functional options per scalar field
<entity>_query_gen.go Fluent query builder with filters, ordering, pagination
<entity>_accessors_gen.go Getters, setters, slice/edge helpers (private fields only)
<entity>_marshal_gen.go DgraphMap(), UnmarshalJSON(), ValidateWith() (private fields only)
cmd/<pkg>/main.go Kong CLI with per-entity subcommands and raw DQL query

Private Field Support

Entities with unexported fields get:

  • Getters/setters with Go-idiomatic names (nameName() / SetName())
  • Singular edge accessors for *Entity, bare Entity, or validate:"max=1" fields → *Type signatures
  • Multi-edge helpers: Append (variadic), Remove (by UID), RemoveFunc (predicate)
  • Primitive slice helpers: Append, Remove (by value), RemoveFunc
  • DgraphMap() for write-path serialization (bypasses dgman's reflect limitation on unexported fields)
  • UnmarshalJSON() for read-path deserialization
  • ValidateWith(ctx, validator) for struct validation via a mirror struct with exported fields
  • Field opt-out: no json tag → skip; dgraph:"-" → explicit skip

Accessor Name Overrides (New)

Initialism-aware naming: toCamelCase now recognizes the canonical Go lint initialisms (38 entries from golang.org/x/lint). Fields like id produce ID() / SetID() automatically.

Per-field overrides via accessor struct tag:

type Widget struct {
    id   string `json:"id,omitempty" accessor:"ID"`
    url  string `json:"url,omitempty"`           // auto: URL() / SetURL()
    name string `json:"name,omitempty"`           // auto: Name() / SetName()
}

The override applies everywhere: getters, setters, Append*, Remove*, With*Option, UnmarshalJSON alias, ValidateWith mirror, and CLI struct fields.

Engine Changes

  • DgraphMapper interface: Insert/Update detect entities with private fields and route through map-based mutation
  • InsertRaw for raw struct insertion
  • Validator support via WithValidator(...) and SelfValidator interface

Edge Type Coverage

All edge variants are supported and tested:

  • []Entity, []*Entity — multi-edge (getter, setter, append, remove, removeFunc)
  • *Entity — singular edge (getter, setter)
  • Entity (bare value) — singular edge (getter returns *Entity, setter accepts *Entity)
  • []Entity / []*Entity + validate:"max=1" or validate:"len=1" — singular edge

Zero New Dependencies

The generator uses only the Go standard library (go/ast, go/parser, text/template, embed).

Usage

//go:generate go run github.com/matthewmcneely/modusgraph/cmd/modusgraph-gen

Testing

  • Unit tests: toCamelCase with 33 initialism/compound/edge cases, accessorName with override and fallback, accessor tag parsing
  • Golden file tests: all entity types including Studio (private field test fixture with all edge variants)
  • Parser tests: parseValidateTag, dgraph:"-", private field detection, singular edge detection, accessor tag
  • Generator tests: accessor output, marshal output, CLI output, validator output, edge variants, external imports
  • All existing modusGraph tests pass unchanged
  • go vet / go build clean

ryanfoxtyler and others added 30 commits October 2, 2024 15:12
This PR removes the WithDataDir() function, which implies that the
datadirectory is optional and enforces it on NewDefaultConfig, since on
its own, the NewDefaultConfig will not spin up a new modusDB instance.
Addressed in

https://linear.app/hypermode/issue/DGR-822/modusdb-newdefaultconfig-fails
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from
0.29.0 to 0.31.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/golang/crypto/commit/b4f1988a35dee11ec3e05d6bf3e90b695fbd8909"><code>b4f1988</code></a>
ssh: make the public key cache a 1-entry FIFO cache</li>
<li><a
href="https://github.com/golang/crypto/commit/7042ebcbe097f305ba3a93f9a22b4befa4b83d29"><code>7042ebc</code></a>
openpgp/clearsign: just use rand.Reader in tests</li>
<li><a
href="https://github.com/golang/crypto/commit/3e90321ac7bcee3d924ed63ed3ad97be2079cb56"><code>3e90321</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="https://github.com/golang/crypto/commit/8c4e668694ccbaa1be4785da7e7a40f2ef93152b"><code>8c4e668</code></a>
x509roots/fallback: update bundle</li>
<li>See full diff in <a
href="https://github.com/golang/crypto/compare/v0.29.0...v0.31.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/crypto&package-manager=go_modules&previous-version=0.29.0&new-version=0.31.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/hypermodeinc/modusDB/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This PR adds the initial issue and PR templates, a changelog, code of
conduct, contributing guide, and security reporting instructions. We can
enhance these over time.
**Description**

Initialize Trunk for broader linting and security monitoring. Fix
addressable issues.

**Checklist**

- [x] Code compiles correctly and linting passes locally
**Description**

add readme to repo
Add generator helper functions for filtering private field categories
(scalar, singular edge, multi-edge, primitive slice). Create two new
per-entity templates:

- accessors.go.tmpl: getters, setters, Append/Remove/RemoveFunc helpers
- marshal.go.tmpl: DgraphMap() for write path, UnmarshalJSON() for reads

Only emitted for entities that have at least one private field. Singular
edge methods use field name (Founder/SetFounder) not EdgeEntity to avoid
name collisions when multiple fields reference the same entity type.
Define DgraphMapper interface with DgraphMap() method. Add toDgraphMap
helper that detects DgraphMapper on single structs and slices, and
mutateWithMap/writeUIDBack helpers for the map-based mutation path.

Insert, InsertRaw, and Update now check for DgraphMapper before calling
dgman's MutateBasic, routing private-field entities through the map
path while preserving the existing code path for exported-field structs.
Define DgraphMapper interface with DgraphMap() method. Add toDgraphMap
helper that detects DgraphMapper on single structs and slices, and
mutateWithMap/writeUIDBack helpers for the map-based mutation path.

Insert, InsertRaw, and Update now check for DgraphMapper before calling
dgman's MutateBasic, routing private-field entities through the map
path while preserving the existing code path for exported-field structs.
…elds

Options template now generates setter-based functional options for
private fields (e.SetName(v)) while keeping direct assignment for
exported fields. CLI AddCmd uses struct literal for exported fields
and setters for private fields.
…elds

Options template now generates setter-based functional options for
private fields (e.SetName(v)) while keeping direct assignment for
exported fields. CLI AddCmd uses struct literal for exported fields
and setters for private fields.
Regenerate golden files to include Studio entity with private field
accessors, marshal/unmarshal, and updated options. Mark all plan
checkboxes as complete.
Regenerate golden files to include Studio entity with private field
accessors, marshal/unmarshal, and updated options. Mark all plan
checkboxes as complete.
feat: Private field accessors with generated serialization
- Fix time import detection to use contains instead of hasPrefix,
  catching *time.Time and []time.Time (options.go.tmpl, accessors.go.tmpl)
- Sort file iteration in parser for deterministic entity ordering
- Guard iter.go.tmpl imports behind entity count check
- Regenerate golden files
@mlwelles mlwelles force-pushed the feature/add-modusgraphgen branch from 369401d to e077d84 Compare March 25, 2026 20:37
Merges the review feedback fixes (time import detection, deterministic
parser iteration, iter.go.tmpl import guard) into main.
…graph-gen

- Update Code Generation section with modusgraph-gen name, private field
  accessors, serialization (DgraphMap/UnmarshalJSON), field opt-out,
  singular edge detection, and updated generated file table
- Add Private Field Support subsection with accessor table, serialization
  explanation, field opt-out examples, and functional option/CLI usage
- Update CLI Commands to note cmd/query deprecation and add cmd/modusgraph-gen
- Add Code Generation Interaction section to VALIDATOR.md explaining how
  validate:"max=1"/"len=1" affects generated accessor signatures
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

11 issues found across 74 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="cmd/modusgraph-gen/internal/generator/templates/cli.go.tmpl">

<violation number="1" location="cmd/modusgraph-gen/internal/generator/templates/cli.go.tmpl:131">
P2: Add command only applies string-typed scalar fields; non-string scalar flags are accepted but never copied into the entity, so user input is ignored and zero values are inserted.</violation>
</file>

<file name="cmd/modusgraph-gen/internal/parser/parser.go">

<violation number="1" location="cmd/modusgraph-gen/internal/parser/parser.go:130">
P2: parseStruct only processes the first name in a multi-name field declaration (f.Names[0]). In Go, `A, B string` is valid; this implementation silently drops `B`, so metadata for additional names is never generated.</violation>
</file>

<file name="cmd/modusgraph-gen/internal/parser/parser_test.go">

<violation number="1" location="cmd/modusgraph-gen/internal/parser/parser_test.go:525">
P2: Test uses a hardcoded absolute `/tmp` path, making the expectation host-dependent and potentially flaky.</violation>
</file>

<file name="README.md">

<violation number="1" location="README.md:628">
P2: README documents the generator at `cmd/modusgraphgen`, but the actual package is `cmd/modusgraph-gen`, causing install/generate commands to fail.</violation>

<violation number="2" location="README.md:676">
P2: Entity detection documentation is inaccurate: parser includes private/unexported fields with json tags, but README claims only exported fields are parsed.</violation>
</file>

<file name="docs/plans/2026-02-27-merge-query-into-generated-cli-design.md">

<violation number="1" location="docs/plans/2026-02-27-merge-query-into-generated-cli-design.md:94">
P2: Mutual-exclusion logic is tied to a duplicated default address literal, creating a brittle regression point if defaults drift.</violation>
</file>

<file name="cmd/modusgraph-gen/internal/generator/templates/options.go.tmpl">

<violation number="1" location="cmd/modusgraph-gen/internal/generator/templates/options.go.tmpl:6">
P2: Time import detection only matches GoType prefix "time.", missing pointer/slice/map time types (e.g., *time.Time, []time.Time), which can lead to generated code referencing time without importing it.</violation>

<violation number="2" location="cmd/modusgraph-gen/internal/generator/templates/options.go.tmpl:24">
P2: Option function names can collide between private and exported fields that differ only by case, leading to duplicate generated functions and a compile-time error.</violation>
</file>

<file name="mutate.go">

<violation number="1" location="mutate.go:265">
P1: `toDgraphMap` can panic by calling `Interface()` on invalid reflection values after `Elem()` on nil pointers (single object and slice element paths).</violation>
</file>

<file name="docs/plans/2026-02-27-merge-modusgraphgen-plan.md">

<violation number="1" location="docs/plans/2026-02-27-merge-modusgraphgen-plan.md:293">
P2: The "full test suite" command only runs tests in the current package. Use `./...` to include subpackages so the plan actually checks all regressions.</violation>
</file>

<file name="cmd/modusgraph-gen/internal/generator/templates/accessors.go.tmpl">

<violation number="1" location="cmd/modusgraph-gen/internal/generator/templates/accessors.go.tmpl:11">
P2: Accessors for private slice fields can reference external types, but the import list is built only from private scalar fields. This can omit required imports (e.g., time or uuid) when an entity has only private slice fields, causing generated code to fail to compile.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

…graph-gen

- Update Code Generation section with modusgraph-gen name, private field
  accessors, serialization (DgraphMap/UnmarshalJSON), field opt-out,
  singular edge detection, and updated generated file table
- Add Private Field Support subsection with accessor table, serialization
  explanation, field opt-out examples, and functional option/CLI usage
- Update CLI Commands to note cmd/query deprecation and add cmd/modusgraph-gen
- Add Code Generation Interaction section to VALIDATOR.md explaining how
  validate:"max=1"/"len=1" affects generated accessor signatures
Copy link
Copy Markdown

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="VALIDATOR.md">

<violation number="1" location="VALIDATOR.md:176">
P2: Documentation claims runtime cardinality enforcement unconditionally, but validation is optional, so enforcement is not guaranteed in all setups.</violation>
</file>

<file name="README.md">

<violation number="1" location="README.md:682">
P2: README examples still show `[]*Entity` relationships, but the generator only detects edges for `[]Entity` and `*Entity`. This documentation inconsistency can mislead users into modeling pointer-slice edges that won’t be recognized by the parser.</violation>
</file>

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

mlwelles added 13 commits March 27, 2026 11:46
- Fix toDgraphMap nil pointer panic: add nil checks before Elem() on
  reflect values for nil pointers, nil slice elements, and nil
  interfaces. Add TestToDgraphMapNilSafety with 4 subtests.
- Fix parser multi-name field declarations: iterate all names in
  "A, B Type" declarations instead of only the first. Add
  TestParseMultiNameDeclaration.
- Fix hardcoded /tmp path in parser test: use t.TempDir() subdir.
- Fix accessor import list: use all private fields (not just scalars)
  for external import detection, also scan slice fields for time types.
- Fix VALIDATOR.md: clarify runtime enforcement requires WithValidator().
- Regenerate golden files.
Remove brainstorm and plan docs from version control — these are
internal working documents. Add docs/plans/ and docs/brainstorms/
to .gitignore. Files preserved on disk.
CLI Add commands now declare flags with correct Go types (int, float64,
bool) instead of always using string. Kong handles type parsing natively.
time.Time fields are accepted as RFC3339 strings and parsed with error
handling. Slice fields are excluded from CLI flags (not representable as
single flags). Add yearFounded int field to Studio test entity.

Also fix typo: outputremoDir → outputDir in generator.go (introduced by
external edit).
Parser now detects []*Entity as a multi-edge (strips pointer from element
type). Accessor template handles pointer-element slices correctly in
signatures, Append (variadic *Entity), Remove (nil-safe UID check), and
RemoveFunc. Marshal template handles nil pointer elements in DgraphMap.

Add comprehensive EdgeFieldVariants test covering all 6 combinations:
- []Entity (public) — Film.Genres
- []*Entity (public) — Film.Directors
- []Entity (private) — Studio.films
- []*Entity (private) — Studio.advisors
- *Entity (private singular) — Studio.founder
- []Entity+validate:"max=1" (private singular) — Studio.currentHead
Add test fixtures and tests for all singular edge type combinations:
- *Entity (type-based singular)
- []Entity + validate:"max=1" (validate-based singular)
- []*Entity + validate:"max=1" (pointer-slice validate-based singular)
- []Entity + validate:"len=1" (len-based singular)
- []*Entity + validate:"len=1" (pointer-slice len-based singular)

Fix accessor template to handle []*Entity singular edges correctly:
getter returns element directly (already a pointer), setter wraps in
[]*Entity{v} (not []Entity{*v}). Fix marshal template for same case.

Generator tests verify correct accessor signatures, nil checks,
absence of Append/Remove on singulars, and marshal alias types.
Parser now detects bare entity types (e.g., `headquarters Country`)
as singular edges, in addition to *Entity, []Entity, and []*Entity.

Accessor template returns &e.field for bare entity getter (address of
value), and dereferences pointer in setter (*v). Marshal template
serializes bare entity by calling DgraphMap() directly.

Add Studio.headquarters Country test fixture and parser + generator
tests verifying correct signatures and behavior.

Complete edge type matrix now covers all 7 variants:
- Entity (bare value, singular)
- *Entity (pointer, singular)
- []Entity (slice, multi)
- []*Entity (pointer slice, multi)
- []Entity + max=1 (slice, singular via validate)
- []*Entity + max=1 (pointer slice, singular via validate)
- []Entity/[]*Entity + len=1 (singular via validate)
Add private bool, time.Time, and *dg.VectorFloat32 fields to Studio
test entity. Add per-type accessor generation tests verifying correct
signatures for all supported scalar types: string, int, float64, bool,
time.Time, and *dg.VectorFloat32 (vector).
Add []int, []float64, []bool, []time.Time private slice fields to
Studio test entity. Verify generated Get/Set/Append/Remove/RemoveFunc
signatures for each primitive slice type.
…truct

Private fields can't be validated by go-playground/validator directly
(it panics on reflect.Value.Interface() for unexported fields). Solve
this by generating a ValidateWith(ctx, validator) method that builds a
temporary exported mirror struct with the same values and validate tags,
then delegates to validator.StructCtx().

Changes:
- model: add ValidateTag string field to capture raw validate tags
- parser: store full validate tag in ValidateTag
- generator: add hasValidateTags/fieldsWithValidation helpers
- marshal.go.tmpl: generate ValidateWith method when validate tags exist
- client.go: add SelfValidator interface, validateOne helper that checks
  for SelfValidator before falling back to StructCtx

Custom validators registered on the *validator.Validate instance work
automatically since the mirror struct preserves the raw validate tags.

Add Studio test fields with validate tags (name: required/min/max,
yearFounded: gte/lte, revenue: gte) and generator tests verifying
ValidateWith method generation, mirror struct tags, and field assignments.
…agged

ValidateWith mirror struct now includes ALL fields (not just those with
validate tags) so custom validators registered by field name or type
work correctly. The mirror is always generated for entities with private
fields, even if no validate tags are present.

Add comprehensive validation integration tests:
- TestSelfValidatorDispatch: 8 subtests covering valid/invalid values
  for string (required, min), int (gte/lte boundaries), float64 (range),
  and boundary value testing
- TestSelfValidatorWithCustomValidator: 4 subtests verifying custom
  validator registration (studio_code format) works through ValidateWith
- TestValidateOneDispatchesSelfValidator: 3 subtests verifying engine
  dispatches to SelfValidator for private-field entities and falls back
  to StructCtx for regular entities
Add accessor:"." struct tag for per-field name overrides and
initialism-aware toCamelCase using the canonical golang.org/x/lint
list (38 entries). Fields like 'id' now generate ID()/SetID()
automatically; explicit overrides take precedence.
@mlwelles mlwelles changed the title feat: add modusgraph-gen code generator with private field support feat: add modusgraph-gen code generator with private field support and accessor name overrides Apr 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants