Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Build

on:
pull_request:
branches: [main, test]

jobs:
build:
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/das

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: "npm"
cache-dependency-path: packages/das/package-lock.json

- run: npm ci

- name: Build
run: npm run build
29 changes: 29 additions & 0 deletions .github/workflows/lint.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: Lint

on:
pull_request:
branches: [main, test]

jobs:
lint:
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/das

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: "npm"
cache-dependency-path: packages/das/package-lock.json

- run: npm ci

- name: Prettier check
run: npm run format:check

- name: ESLint
run: npm run lint
26 changes: 26 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: Test

on:
pull_request:
branches: [main, test]

jobs:
test:
runs-on: ubuntu-latest
defaults:
run:
working-directory: packages/das

steps:
- uses: actions/checkout@v4

- uses: actions/setup-node@v4
with:
node-version: 20
cache: "npm"
cache-dependency-path: packages/das/package-lock.json

- run: npm ci

- name: Run tests with coverage
run: npm run test:cov
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ npm-debug.log*

# Build outputs
dist/
coverage/

# IDE
.vscode/
Expand Down
21 changes: 19 additions & 2 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,16 @@ GitHub event occurs

## Raw Tables (already defined)

The existing schema in `packages/db/` covers 8 tables. These are the webhook write layer — upserted on every event, append-only where appropriate.
The existing schema in `packages/db/` covers 10 tables. These are the webhook write layer — upserted on every event, append-only where appropriate.

| Table | Purpose |
|---|---|
| `repos` | Tracked repos + App installation metadata |
| `pull_requests` | One row per PR, upserted on state changes |
| `issues` | One row per issue, upserted on state changes |
| `reviews` | One row per review submission, append-only |
| `comments` | Issue + PR thread comments, append-only (upsert on edit) |
| `review_comments` | Inline code review comments on PR diffs, append-only (upsert on edit) |
| `label_events` | Append-only log of every label add/remove |
| `pr_files` | File-level change metadata (filename, status, additions, deletions) |
| `pr_file_contents` | Actual file content (base + head versions) for AST/token scoring |
Expand All @@ -50,7 +52,22 @@ The `pull_requests.scoring_data_stored` flag indicates whether `pr_files` and `p

### Note on data retention

Raw tables keep data indefinitely. Storage is ~45 MB/month at current scale. The 35-day lookback is a **scoring concern, not a storage concern** — views filter by time window, but historical data is preserved for trend analysis, audits, and dashboard use.
Raw tables keep data indefinitely. Storage is ~45 MB/month at current scale (256 repos), with conversation threads (issue_comments + review_comments) adding <1 MB/month. The 35-day lookback is a **scoring concern, not a storage concern** — views filter by time window, but historical data is preserved for trend analysis, audits, and dashboard use.

### Storage breakdown estimate (256 repos)

| Table | Monthly growth | Notes |
|---|---|---|
| `pull_requests` | ~2 MB | Metadata only, one row per PR |
| `issues` | ~1 MB | Metadata only, one row per issue |
| `reviews` | ~500 KB | One row per review submission |
| `comments` | ~500 KB | ~500 bytes/comment avg, covers issues + PR threads |
| `review_comments` | ~300 KB | Inline code comments, includes file path + line |
| `label_events` | ~200 KB | Append-only label log |
| `pr_files` | ~3 MB | File-level change metadata per PR |
| `pr_file_contents` | ~38 MB | Actual source code (pruned to 35-day window) |
| `webhook_deliveries` | ~100 KB | Pruned periodically |
| **Total** | **~45.5 MB** | Comments are negligible (<1% of total) |

---

Expand Down
42 changes: 42 additions & 0 deletions GITHUB_APP_SETUP.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,19 @@ Check these boxes:
|---|---|
| Pull request | opened, closed, merged, edited, synchronize (new push), reopened |
| Pull request review | submitted (APPROVED, CHANGES_REQUESTED, COMMENTED) |
| Pull request review comment | created, edited, deleted (inline code review comments on diffs) |
| Issues | opened, closed, reopened, transferred, deleted, labeled, unlabeled |
| Issue comment | created, edited, deleted (comments on issues AND PR threads) |
| Label | created, edited, deleted (repo-level label definitions) |

**Note on issue label events:** The `labeled` and `unlabeled` actions come through the **Issues** event subscription, not the Label subscription. The Label subscription covers repo-level label CRUD (creating/renaming/deleting label definitions). You need both.

**Note on comments:** GitHub has two distinct comment types:
- **Issue comments** (`issue_comment` event) — covers both issue comments AND PR thread/conversation comments. These are general discussion comments.
- **Pull request review comments** (`pull_request_review_comment` event) — inline comments on specific lines in a PR diff, attached to a review.

Both are needed for full conversation thread coverage. Stored in `comments` and `review_comments` tables respectively.

### Other settings

| Field | Value |
Expand Down Expand Up @@ -234,6 +242,40 @@ sender.site_admin → (not directly association — see note)

This is a design decision worth noting — the label event handler may need an extra API call or a lookup to populate `actor_association`.

### `issue_comment` event

```
action: created | edited | deleted
comment.id → comment_id (BIGINT, globally unique)
comment.user.id → author_github_id
comment.user.login → author_login
comment.author_association → author_association
comment.body → body (full text)
comment.created_at
comment.updated_at
issue.number → issue_number (works for both issues AND PRs)
issue.pull_request → if present, this comment is on a PR thread (not a plain issue)
```

**Note:** GitHub treats PR thread comments as issue comments. The `issue.pull_request` field is present when the comment is on a PR. The `issue.number` is the PR number in that case.

### `pull_request_review_comment` event

```
action: created | edited | deleted
comment.id → comment_id (BIGINT, globally unique)
comment.user.id → reviewer_github_id
comment.user.login → reviewer_login
comment.pull_request_review_id → review_id (links to reviews table)
comment.path → file path in the diff
comment.line → line number (nullable for outdated comments)
comment.side → LEFT or RIGHT (base vs head)
comment.body → body (full text)
comment.created_at
comment.updated_at
pull_request.number → pr_number
```

### `label` event (repo-level)

```
Expand Down
29 changes: 29 additions & 0 deletions packages/das/.eslintrc.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{
"parser": "@typescript-eslint/parser",
"parserOptions": {
"project": "tsconfig.json",
"tsconfigRootDir": ".",
"sourceType": "module"
},
"plugins": ["@typescript-eslint/eslint-plugin"],
"extends": [
"plugin:@typescript-eslint/recommended",
"plugin:@typescript-eslint/recommended-requiring-type-checking",
"plugin:prettier/recommended"
],
"root": true,
"env": {
"node": true,
"jest": true
},
"rules": {
"@typescript-eslint/no-unused-vars": "error",
"@typescript-eslint/no-explicit-any": "error",
"@typescript-eslint/explicit-function-return-type": "warn",
"@typescript-eslint/no-floating-promises": "error",
"@typescript-eslint/no-misused-promises": "error",
"@typescript-eslint/await-thenable": "error",
"@typescript-eslint/no-unnecessary-type-assertion": "warn"
},
"ignorePatterns": ["dist/", "node_modules/", "coverage/"]
}
Loading
Loading