This repository was archived by the owner on Jan 23, 2026. It is now read-only.
Repository navigation
Prompt eval sdk changes #184
Merged
Merged
Changes from all commits
Commits
Show all changes
9 commits
Select commit
Hold shift + click to select a range
75192fe
added some base
potofpie 31e6d57
checkpoint
potofpie dfee537
checkpoint
potofpie 3dbe64f
Update code generation rules for optional field handling
potofpie 059d884
clean up
potofpie 7ef7d85
fixed a bunch of stuff
potofpie adf770b
fix tests
potofpie 2e9efc1
Merge branch 'main' of https://github.com/agentuity/sdk-js into promp…
potofpie 7708d3a
version bump
potofpie File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,169 @@ | ||
| --- | ||
| description: Code Generation Development Rules for SDK | ||
| globs: src/apis/**/*.ts, src/server/*.ts, src/types.ts | ||
| alwaysApply: true | ||
| --- | ||
|
|
||
| # Code Generation Development Rules | ||
|
|
||
| > **⚠️ IMPORTANT**: These rules work in conjunction with the CLI rules. When updating these SDK rules, also update `cli/.cursor/rules/code-generation.mdc` to keep them in sync. | ||
|
|
||
| ## Core Principles | ||
|
|
||
| ### Optional Field Handling | ||
| - ✅ Generated code should NEVER require optional chaining (`?.`) | ||
| - ✅ Always generate both `system` and `prompt` fields, even if empty | ||
| - ✅ Empty fields should return empty strings, not undefined | ||
| - ❌ Never generate partial objects that require optional chaining | ||
|
|
||
| ### Never Modify Generated Content in Source Files | ||
| - ❌ NEVER edit files in `src/` that contain generated content | ||
| - ❌ NEVER hardcode generated content like `copyWriter` prompts in source files | ||
| - ✅ ALWAYS load generated content dynamically at runtime | ||
| - ✅ Provide fallbacks for missing generated content | ||
|
|
||
| ### Dynamic Loading Pattern | ||
| Generated content doesn't exist at SDK build time, so use dynamic loading patterns: | ||
|
|
||
| ```typescript | ||
| // ✅ Good: Dynamic loading with fallbacks | ||
| public async loadGeneratedContent(): Promise<void> { | ||
| try { | ||
| const path = require('path'); | ||
| const fs = require('fs'); | ||
|
|
||
| const possiblePaths = [ | ||
| path.join(process.cwd(), 'node_modules', '@agentuity', 'sdk', 'dist', 'generated', 'content.js'), | ||
| path.join(process.cwd(), 'node_modules', '@agentuity', 'sdk', 'src', 'generated', 'content.js') | ||
| ]; | ||
|
|
||
| for (const possiblePath of possiblePaths) { | ||
| if (fs.existsSync(possiblePath)) { | ||
| const generatedModule = require(possiblePath); | ||
| this.content = generatedModule.content || defaultContent; | ||
| break; | ||
| } | ||
| } | ||
| } catch (error) { | ||
| this.content = defaultContent; | ||
| console.warn('No generated content found'); | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ## SDK-Specific Rules | ||
|
|
||
| ### Path Resolution | ||
| - Use absolute paths only (relative paths don't work in bundled environments) | ||
| - Check `dist/` directory first, then `src/` directory | ||
| - Always provide fallbacks for missing generated content | ||
|
|
||
| ### Context Integration | ||
| ```typescript | ||
| // ✅ Good: Load generated content in context creation | ||
| export async function createServerContext(req: ServerContextRequest): Promise<AgentContext> { | ||
| // ... other initialization | ||
|
|
||
| // Load generated content dynamically | ||
| await promptAPI.loadPrompts(); | ||
|
|
||
| return { | ||
| // ... other context properties | ||
| prompts: () => promptAPI.prompts, | ||
| }; | ||
| } | ||
| ``` | ||
|
|
||
| ### Type Safety | ||
| - Generate TypeScript definitions for generated content | ||
| - Use proper type annotations for dynamic imports | ||
| - Maintain type safety throughout the loading process | ||
|
|
||
| ## Common Patterns | ||
|
|
||
| ### Generated Content API Class | ||
| ```typescript | ||
| export default class GeneratedContentAPI { | ||
| public content: typeof defaultContent; | ||
|
|
||
| constructor() { | ||
| this.content = defaultContent; | ||
| } | ||
|
|
||
| public async loadContent(): Promise<void> { | ||
| // Dynamic loading logic here | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| ### Error Handling | ||
| ```typescript | ||
| try { | ||
| // Try to load generated content | ||
| const generatedModule = require(possiblePath); | ||
| this.content = generatedModule.content || defaultContent; | ||
| } catch (error) { | ||
| // Fallback to default content | ||
| this.content = defaultContent; | ||
| console.warn('No generated content found'); | ||
| } | ||
| ``` | ||
|
|
||
| ## Common Pitfalls to Avoid | ||
|
|
||
| ### ❌ Don't Do This | ||
| ```typescript | ||
| // Hardcoding generated content in source files | ||
| export const prompts = { | ||
| copyWriter: { /* hardcoded content */ } | ||
| }; | ||
|
|
||
| // Using relative imports that don't work in bundles | ||
| const generatedModule = require('./generated/_index.js'); | ||
|
|
||
| // Not providing fallbacks | ||
| const generatedModule = require(possiblePath); // Will crash if file doesn't exist | ||
| ``` | ||
|
|
||
| ### ✅ Do This Instead | ||
| ```typescript | ||
| // Dynamic loading with absolute paths | ||
| const possiblePaths = [ | ||
| path.join(process.cwd(), 'node_modules', '@agentuity', 'sdk', 'dist', 'generated', '_index.js') | ||
| ]; | ||
|
|
||
| // With proper error handling | ||
| try { | ||
| const generatedModule = require(possiblePath); | ||
| this.content = generatedModule.content || defaultContent; | ||
| } catch (error) { | ||
| this.content = defaultContent; | ||
| } | ||
| ``` | ||
|
|
||
| ## Build Considerations | ||
|
|
||
| - Generated content is loaded at runtime, not build time | ||
| - Use `require()` for CommonJS compatibility in bundled environments | ||
| - Avoid `import()` statements for generated content | ||
| - Ensure fallbacks work when generated content is missing | ||
|
|
||
| ## Development Workflow | ||
|
|
||
| ### When Making Changes to Generated Content Structure: | ||
| 1. **Update CLI code generation** in `cli/internal/bundler/prompts/code_generator.go` | ||
| 2. **Update SDK to handle new structure** in `src/apis/prompt/index.ts` | ||
| 3. **Bump SDK version** in `package.json` | ||
| 4. **Build SDK**: `npm run build` | ||
| 5. **Install in test project**: `npm install /path/to/sdk-js` | ||
| 6. **Run bundle command**: `agentuity-cli bundle` to regenerate content | ||
| 7. **Copy dist to node_modules**: `cp -r dist/* node_modules/@agentuity/sdk/dist/` (if needed) | ||
|
|
||
| **Note**: After the first setup, you only need to run step 7 (`cp -r dist/* node_modules/@agentuity/sdk/dist/`) for subsequent changes to avoid reinstalling the entire package. | ||
|
|
||
| ### Required Exports | ||
| - ✅ Always export `interpolateTemplate` from main SDK index | ||
| - ✅ Generated content must import from `@agentuity/sdk` (not relative paths) | ||
| - ✅ Ensure all dependencies are properly exported | ||
|
|
||
| Remember: The SDK's job is to load generated content dynamically, not contain hardcoded generated content. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,36 @@ | ||
| /** | ||
| * Singleton class for PatchPortal | ||
| */ | ||
| export default class PatchPortal { | ||
| private static instance: PatchPortal | null = null; | ||
| private state: Record<string, unknown> = {}; | ||
|
|
||
| private constructor() { | ||
| // Private constructor to prevent direct instantiation | ||
| } | ||
|
|
||
| /** | ||
| * Get the singleton instance of PatchPortal | ||
| */ | ||
| public static async getInstance(): Promise<PatchPortal> { | ||
| if (!PatchPortal.instance) { | ||
| PatchPortal.instance = new PatchPortal(); | ||
| } | ||
| return PatchPortal.instance; | ||
| } | ||
|
|
||
| /** | ||
| * Example method - you can add your specific functionality here | ||
| */ | ||
| public async process(key: string, data: unknown): Promise<unknown> { | ||
| this.state[key] = data; | ||
| return data; | ||
| } | ||
|
|
||
| /** | ||
| * Example method for demonstrating the singleton | ||
| */ | ||
| public getInstanceId(): string { | ||
| return `PatchPortal-${Date.now()}`; | ||
| } | ||
|
Comment on lines
+30
to
+35
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Misleading method name and behavior.
Consider either:
private readonly instanceId: string;
private constructor() {
this.instanceId = `PatchPortal-${Date.now()}`;
}
public getInstanceId(): string {
return this.instanceId;
}
public getCurrentTimestamp(): string {
return `PatchPortal-${Date.now()}`;
}
🤖 Prompt for AI Agents |
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| export const prompts = { | ||
| }; | ||
|
|
||
| // Export types for compatibility | ||
| export const PromptConfig = undefined; // Type-only export, value is not used | ||
| export const PromptName = undefined; // Type-only export, value is not used | ||
| export const PromptsCollection = undefined; // Type-only export, value is not used |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| // biome-ignore lint/suspicious/noEmptyInterface: <explanation> | ||
| export interface PromptsCollection {} | ||
| export declare const prompts: PromptsCollection; | ||
| export type PromptConfig = any; | ||
| export type PromptName = any; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| export type PromptConfig = any; | ||
| export type PromptName = any; | ||
|
|
||
| // This will be replaced by the CLI with the actual generated types | ||
| export interface PromptsCollection { | ||
| [promptSlug: string]: { | ||
| slug: string; | ||
| system: { compile: (variables?: Record<string, any>) => string }; | ||
| prompt: { compile: (variables?: Record<string, any>) => string }; | ||
| }; | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.