-
Notifications
You must be signed in to change notification settings - Fork 1
Add optimized CI workflow with generated-artifact freshness checks #393
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
78bf1b1
Add optimized CI pipeline and generated-artifact validation
KillariDev a34229c
Merge remote-tracking branch 'origin/main' into t3code/a5b527db
KillariDev 943d4f0
Migrate CI gate to GitHub Actions workflow
KillariDev fb0434d
Document validation gate flow and add inactive CI template
KillariDev b713827
use ci
KillariDev File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| import { spawnSync } from 'node:child_process' | ||
| import { promises as fs } from 'node:fs' | ||
| import * as path from 'node:path' | ||
| import * as url from 'node:url' | ||
|
|
||
| const scriptDirectory = path.dirname(url.fileURLToPath(import.meta.url)) | ||
| const repositoryRoot = path.join(scriptDirectory, '..') | ||
|
|
||
| const explicitlyRequiredGeneratedOutputs = ['shared/js/.freshness-hash', 'solidity/artifacts/Contracts.json', 'solidity/artifacts/.freshness-hash', 'solidity/.contract-hash.json', 'solidity/ts/types/contractArtifact.ts', 'solidity/types/contractArtifact.ts', 'ui/ts/abis.ts', 'ui/ts/contractArtifact.ts'] | ||
|
|
||
| const generatedReviewPaths = ['shared/js', 'solidity/artifacts', 'solidity/.contract-hash.json', 'solidity/ts/types/contractArtifact.ts', 'solidity/types', 'ui/js', 'ui/ts/abis.ts', 'ui/ts/contractArtifact.ts', 'ui/ts/deploymentArtifacts.ts', 'ui/ts/deploymentsArtifacts.ts', 'ui/vendor'] | ||
|
|
||
| function isRecord(value: unknown): value is Record<string, unknown> { | ||
| return typeof value === 'object' && value !== null && !Array.isArray(value) | ||
| } | ||
|
|
||
| async function assertExists(relativePath: string) { | ||
| try { | ||
| await fs.stat(path.join(repositoryRoot, relativePath)) | ||
| } catch (error) { | ||
| if (error instanceof Error && 'code' in error && error.code === 'ENOENT') { | ||
| throw new Error(`Generated artifact is missing after generation: ${relativePath}`) | ||
| } | ||
| throw error | ||
| } | ||
| } | ||
|
|
||
| async function readJsonObject(relativePath: string) { | ||
| const parsed = JSON.parse(await fs.readFile(path.join(repositoryRoot, relativePath), 'utf8')) | ||
| if (!isRecord(parsed)) throw new Error(`${relativePath} must contain a JSON object`) | ||
| return parsed | ||
| } | ||
|
|
||
| async function assertContractsJsonReadable() { | ||
| const contractsJsonPath = path.join(repositoryRoot, 'solidity/artifacts/Contracts.json') | ||
| try { | ||
| JSON.parse(await fs.readFile(contractsJsonPath, 'utf8')) | ||
| } catch (error) { | ||
| if (error instanceof SyntaxError) throw new Error('Generated solidity/artifacts/Contracts.json is not valid JSON') | ||
| throw error | ||
| } | ||
| } | ||
|
|
||
| function normalizeRepositoryRelativePath(baseDirectory: string, relativePath: string) { | ||
| if (!relativePath.startsWith('./') && !relativePath.startsWith('../')) { | ||
| throw new Error(`Expected a local generated path, received: ${relativePath}`) | ||
| } | ||
| return path.posix.normalize(path.posix.join(baseDirectory, relativePath)) | ||
| } | ||
|
|
||
| async function getSharedPackageGeneratedOutputs() { | ||
| const packageJson = await readJsonObject('shared/package.json') | ||
| const exportsValue = packageJson['exports'] | ||
| if (!isRecord(exportsValue)) throw new Error('shared/package.json exports must be an object') | ||
|
|
||
| const outputs: string[] = [] | ||
| for (const [exportName, exportValue] of Object.entries(exportsValue)) { | ||
| if (!isRecord(exportValue)) throw new Error(`shared/package.json export ${exportName} must be an object`) | ||
| const defaultPath = exportValue['default'] | ||
| if (typeof defaultPath !== 'string') throw new Error(`shared/package.json export ${exportName} must define a default path`) | ||
| const generatedJavaScriptPath = normalizeRepositoryRelativePath('shared', defaultPath) | ||
| outputs.push(generatedJavaScriptPath) | ||
| if (generatedJavaScriptPath.endsWith('.js')) { | ||
| outputs.push(generatedJavaScriptPath.replace(/\.js$/, '.d.ts')) | ||
| } | ||
| } | ||
| return outputs | ||
| } | ||
|
|
||
| async function getUiImportMapGeneratedOutputs() { | ||
| const indexHtml = await fs.readFile(path.join(repositoryRoot, 'ui/index.html'), 'utf8') | ||
| const importMapMatch = indexHtml.match(/<script\b[^>]*\btype\s*=\s*['"]importmap['"][^>]*>([\s\S]*?)<\/script>/i) | ||
| if (importMapMatch === null) throw new Error('ui/index.html is missing an import map') | ||
|
|
||
| const importMapText = importMapMatch[1] | ||
| if (importMapText === undefined) throw new Error('ui/index.html import map is empty') | ||
| const importMap = JSON.parse(importMapText) | ||
| if (!isRecord(importMap)) throw new Error('ui/index.html import map must be a JSON object') | ||
| const imports = importMap['imports'] | ||
| if (!isRecord(imports)) throw new Error('ui/index.html import map imports must be an object') | ||
|
|
||
| const outputs: string[] = [] | ||
| for (const [specifier, targetPath] of Object.entries(imports)) { | ||
| if (typeof targetPath !== 'string') throw new Error(`ui/index.html import map target for ${specifier} must be a string`) | ||
| if (!targetPath.startsWith('./') && !targetPath.startsWith('../')) continue | ||
| outputs.push(normalizeRepositoryRelativePath('ui', targetPath)) | ||
| } | ||
| return outputs | ||
| } | ||
|
|
||
| function runGit(args: readonly string[]) { | ||
| const result = spawnSync('git', args, { | ||
| cwd: repositoryRoot, | ||
| encoding: 'utf8', | ||
| }) | ||
| if (result.error !== undefined) throw result.error | ||
| return result | ||
| } | ||
|
|
||
| function getTrackedGeneratedPaths() { | ||
| const result = runGit(['ls-files', '--', ...generatedReviewPaths]) | ||
| if (result.status !== 0) { | ||
| throw new Error(`Unable to list tracked generated paths.\n${result.stdout}${result.stderr}`) | ||
| } | ||
| return result.stdout | ||
| .split('\n') | ||
| .map(line => line.trim()) | ||
| .filter(line => line !== '') | ||
| } | ||
|
|
||
| function assertNoTrackedGeneratedPaths(trackedGeneratedPaths: readonly string[]) { | ||
| if (trackedGeneratedPaths.length === 0) return | ||
|
|
||
| throw new Error(`Generated artifacts must remain untracked. Remove these paths from Git and keep them covered by .gitignore and the generated artifact policy:\n${trackedGeneratedPaths.join('\n')}`) | ||
| } | ||
|
|
||
| const requiredGeneratedOutputs = new Set([...explicitlyRequiredGeneratedOutputs, ...(await getSharedPackageGeneratedOutputs()), ...(await getUiImportMapGeneratedOutputs())]) | ||
|
|
||
| for (const relativePath of requiredGeneratedOutputs) { | ||
| await assertExists(relativePath) | ||
| } | ||
| await assertContractsJsonReadable() | ||
|
|
||
| const trackedGeneratedPaths = getTrackedGeneratedPaths() | ||
| assertNoTrackedGeneratedPaths(trackedGeneratedPaths) | ||
|
|
||
| console.log('Generated artifacts verified. Generated outputs are intentionally untracked, so freshness is validated by successful generation and required-output checks.') |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Architectural concern: This single step embeds approximately 80 lines of bash process management code (start_task, remove_pid, terminate_remaining, wait_for_tasks) inline in the YAML workflow. The codebase has an established pattern of placing CI-related logic in scripts/ as TypeScript files (e.g., check-generated-artifacts.mts, check-mainnet-deployment.mts) that are linted by Biome, type-checked by tsc:scripts, and testable. The inline bash orchestrator cannot be linted, type-checked, or tested by any existing project tooling.
Additionally, this loses GitHub Actions step-level visibility: the old workflow had individually named steps (Tests, Biome and Solidity checks, Dead code analysis, Dependency audit) whose pass/fail status was visible directly in the UI. With the combined step, all sub-task output is interleaved and developers must parse logs to identify which sub-task failed.
Recommendation: extract the orchestration logic into a script under scripts/ to restore testability, lintability, and reusability while preserving the parallelism benefit.