-
Notifications
You must be signed in to change notification settings - Fork 732
feat: add query layer (CM-1059) #3942
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+527
−16
Merged
Changes from all commits
Commits
Show all changes
59 commits
Select commit
Hold shift + click to select a range
ddb6611
feat: add static api key middleware for dev stats
ulemons 0761019
fix: lint
ulemons 9d265bf
fix: remove local secret
ulemons 9bf2e94
fix: use db oriented api keys
ulemons ef9c57b
fix: remove useless env var
ulemons a32bb5f
fix: lint
ulemons 33e5d2f
fix: review
ulemons fbc2a4b
feat: add query layer
ulemons e602856
feat: add filtering on query layer
ulemons 8db2e26
feat: refactor in dal
ulemons 3eca919
feat: add affiliations
ulemons 00da7e9
feat: adding logs
ulemons 2cb2483
fix: lint
ulemons cd368bc
fix: lint
ulemons ba92c27
fix: created at error
ulemons e16d5bf
fix: createdAt as date
ulemons 75972d4
feat: adding logs
ulemons c753cca
refactor: simplify buildTimeline
ulemons 0d36408
fix: lint
ulemons e88c8f1
fix: remove logs
ulemons e680cbe
fix: remove comments
ulemons 788bfe6
fix: change logging
ulemons 6f26621
refactor: create dal for affiliations
ulemons 234672d
fix: lint
ulemons 5845c67
refactor: export affiliation on dal
ulemons 4c90f1b
refactor: simplify longestDateRange
ulemons 74e3f2d
fix: filter first relevant orgs to avoid timeouts
ulemons ffe12ed
fix: align url
ulemons 7145bea
fix: test not joining for member count
ulemons c25a4dd
fix: add safe wrap
ulemons 6c1fd91
fix: refactor v1Router
ulemons 1b5a0f2
fix: refactor naming
ulemons 84fb096
fix: add 404 error
ulemons 351a3c9
fix: add 404 error in public router
ulemons 3428fb1
fix: page are query aprams
ulemons 68b65fd
fix: revert not found inside the routher
ulemons ce35979
fix: adjust page fields
ulemons 4afe7f0
fix: add contributors in page
ulemons 9065657
fix: cursor review
ulemons cf15c87
fix: reduce log, simplify serializing error
ulemons bfacaad
fix: lint
ulemons b35061c
fix: body parser error interceptor
ulemons 1f115c7
feat: adjust limits
ulemons b0d2507
feat: simplify query
ulemons d4cc31c
feat: use join instead of in
ulemons a47c1b3
feat: revert to memberOrganizations
ulemons 7f70a2d
feat: cursor review
ulemons eaa776a
feat: remove useless logs
ulemons 86da5ec
refactor: simplify affiliations
ulemons 8886263
fix: early return
ulemons 55cb440
fix: use common blacklist variable
ulemons f9ef818
feat: merge
ulemons 8030024
fix: comment after review
ulemons 850cfe2
fix: lint
ulemons 06c0569
refactor: remove dev stats
ulemons 226a9f0
refactor: rename interface
ulemons b20d003
refactor: update endpoint
ulemons ecbd125
refactor: lint
ulemons 7377aae
fix: cursor comment
ulemons File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| import type { Request, Response } from 'express' | ||
| import { z } from 'zod' | ||
|
|
||
| import { | ||
| findMembersByGithubHandles, | ||
| findVerifiedEmailsByMemberIds, | ||
| optionsQx, | ||
| resolveAffiliationsByMemberIds, | ||
| } from '@crowd/data-access-layer' | ||
|
|
||
| import { ok } from '@/utils/api' | ||
| import { validateOrThrow } from '@/utils/validation' | ||
|
|
||
| const MAX_HANDLES = 100 | ||
| const DEFAULT_PAGE_SIZE = 20 | ||
|
|
||
| const bodySchema = z.object({ | ||
| githubHandles: z | ||
| .array(z.string().trim().min(1).toLowerCase()) | ||
| .min(1) | ||
| .max(MAX_HANDLES, `Maximum ${MAX_HANDLES} handles per request`), | ||
| }) | ||
|
|
||
| const querySchema = z.object({ | ||
| page: z.coerce.number().int().min(1).default(1), | ||
| pageSize: z.coerce.number().int().min(1).max(MAX_HANDLES).default(DEFAULT_PAGE_SIZE), | ||
| }) | ||
|
|
||
| export async function getAffiliations(req: Request, res: Response): Promise<void> { | ||
| const { githubHandles } = validateOrThrow(bodySchema, req.body) | ||
| const { page, pageSize } = validateOrThrow(querySchema, req.query) | ||
| const qx = optionsQx(req) | ||
|
|
||
| const offset = (page - 1) * pageSize | ||
|
|
||
| // Step 1: find all verified members across all handles | ||
| const allMemberRows = await findMembersByGithubHandles(qx, githubHandles) | ||
|
|
||
| const foundHandles = new Set(allMemberRows.map((r) => r.githubHandle.toLowerCase())) | ||
| const notFound = githubHandles.filter((h) => !foundHandles.has(h)) | ||
|
|
||
| const pageMemberRows = allMemberRows.slice(offset, offset + pageSize) | ||
|
|
||
| if (pageMemberRows.length === 0) { | ||
| ok(res, { | ||
| total: githubHandles.length, | ||
| totalFound: allMemberRows.length, | ||
| page, | ||
| pageSize, | ||
| contributorsInPage: 0, | ||
| contributors: [], | ||
| notFound, | ||
| }) | ||
| return | ||
| } | ||
|
|
||
| const memberIds = pageMemberRows.map((r) => r.memberId) | ||
|
|
||
| // Step 2: fetch verified emails for current page | ||
| const emailRows = await findVerifiedEmailsByMemberIds(qx, memberIds) | ||
|
|
||
| const emailsByMember = new Map<string, string[]>() | ||
| for (const row of emailRows) { | ||
| const list = emailsByMember.get(row.memberId) ?? [] | ||
| list.push(row.email) | ||
| emailsByMember.set(row.memberId, list) | ||
| } | ||
|
|
||
| // Step 3: resolve affiliations for current page only | ||
| const affiliationsByMember = await resolveAffiliationsByMemberIds(qx, memberIds) | ||
|
|
||
| // Step 4: build response | ||
| const contributors = pageMemberRows.map((member) => ({ | ||
| githubHandle: member.githubHandle, | ||
| name: member.displayName, | ||
| emails: emailsByMember.get(member.memberId) ?? [], | ||
| affiliations: affiliationsByMember.get(member.memberId) ?? [], | ||
| })) | ||
|
|
||
| ok(res, { | ||
| total: githubHandles.length, | ||
| totalFound: allMemberRows.length, | ||
| page, | ||
| pageSize, | ||
| contributorsInPage: contributors.length, | ||
| contributors, | ||
| notFound, | ||
| }) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,13 +1,25 @@ | ||
| import { Router } from 'express' | ||
|
|
||
| import { NotFoundError } from '@crowd/common' | ||
|
|
||
| import { AUTH0_CONFIG } from '../../../conf' | ||
| import { oauth2Middleware } from '../middlewares/oauth2Middleware' | ||
| import { staticApiKeyMiddleware } from '../middlewares/staticApiKeyMiddleware' | ||
|
|
||
| import { memberOrganizationAffiliationsRouter } from './affiliations' | ||
| import { membersRouter } from './members' | ||
| import { organizationsRouter } from './organizations' | ||
|
|
||
| export function v1Router(): Router { | ||
| const router = Router() | ||
|
|
||
| router.use('/members', membersRouter()) | ||
| router.use('/organizations', organizationsRouter()) | ||
| router.use('/members', oauth2Middleware(AUTH0_CONFIG), membersRouter()) | ||
| router.use('/organizations', oauth2Middleware(AUTH0_CONFIG), organizationsRouter()) | ||
| router.use('/affiliations', staticApiKeyMiddleware(), memberOrganizationAffiliationsRouter()) | ||
|
|
||
| router.use(() => { | ||
| throw new NotFoundError() | ||
skwowet marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| }) | ||
|
|
||
| return router | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.