[zahorniak]

Summary

The "unfinished details" step of collectJobs and collectWorkflows selected every workflow in a non-terminal status, without an age limit. A workflow with an approval job that nobody approves stays on_hold for ever, so DevLake collected it again on every run. Because the step appends to the raw tables, _raw_circleci_api_jobs grew on every run.

This PR selects only workflows created in the last 30 days:

• UnfinishedJobsInputClauses joins _tool_circleci_workflows and filters on the workflow created_date. Jobs have no created date of their own (created_date is empty for all on_hold jobs in our instance).
• New UnfinishedWorkflowsInputClauses applies the same filter to the workflow collector.
• The window is the constant UnfinishedDetailsLookback (30 days). The date is computed in Go, so the query works on MySQL and PostgreSQL.

A workflow that gets its approval within 30 days is still collected again, so its deploy jobs after the approval are not lost. In our instance, all approvals except 1 of about 1,860 came within 30 days.

Tests

• TestCircleciUnfinishedJobsInputIterator (updated) and TestCircleciUnfinishedWorkflowsInputIterator (new) seed a workflow older than 30 days and check that it is not selected. Both tests pass on MySQL 8 and PostgreSQL 14, and fail without the change.

Result on real data (one instance, 2 CircleCI projects, same query as the collectors):

	Before	After
Workflows selected by collectJobs unfinished details	14,864	1,851
Job rows added to _raw_circleci_api_jobs per run	166,485	24,119
Workflows selected by collectWorkflows unfinished details	15,168	1,858

Does this close any open issues?

Closes #9304

Other Information

• [Refactor][CircleCI] Make extractJobs and extractWorkflows incremental with NewStatefulApiExtractor #9305 (stateful CircleCI extractors) is the second part. After both changes, an incremental run reads only the new raw rows.
• Old duplicate raw rows stay until the next full sync.