casa21 self-hosted runner for Integration Tests

[plebhash]

integration_github_runner job

legacy runner, capped to github free runners

integration_casa21_runner job

new runner on self-hosted infra @CasaVinteUm

VM parameters:

• 24 CPU Cores
• 32 GB RAM

only PRs coming from members of stratum-mining github org trigger integration_casa21_runner

integration_required job

gates whether the PR is blocked for merging by leveraging github-script to enforce this truth table, where integration_casa21_runner takes precedence:

integration_github_runner	integration_casa21_runner	integration_required (PR blocked?)
passed ✅ (but doesn't matter)	passed ✅	unblocked ✅
failed ❌ (but doesn't matter)	passed ✅	unblocked ✅
running 💫 (but doesn't matter)	passed ✅	unblocked ✅
passed ✅ (but doesn't matter)	failed ❌	blocked ❌
failed ❌ (but doesn't matter)	failed ❌	blocked ❌
running 💫 (but doesn't matter)	failed ❌	blocked ❌
passed ✅	absent (PR is not from org member)	unblocked ✅
failed ❌	absent (PR is not from org member)	blocked ❌
running 💫	absent (PR is not from org member)	waiting 💫

or more concisely, as a mermaid flowchart:

flowchart TD
  R[integration_required]

  R --> C{integration_casa21_runner?}

  C -->|passed ✅| U[PR unblocked ✅]
  C -->|failed ❌| B[PR blocked ❌]
  C -->|absent| G{integration_github_runner?}

  G -->|passed ✅| GU[PR unblocked ✅]
  G -->|failed ❌| GB[PR blocked ❌]
  G -->|running 💫| W[waiting 💫]

Loading

parallelism parameters

cargo nextest parameter	integration_github_runner	integration_casa21_runner
--nocapture (forces sequential execution)	YES	NO
--test-threads (number of threads for parallel execution)	1	24

performance comparison

	integration_casa21_runner	integration_github_runner
execution time:	~5 minutes ⏳	~30 minutes ⏳⏳⏳⏳⏳⏳

6x improvement 🚀

[plebhash]

note that ci job is being replaced with integration_github_runner

currently ci is set as "Required" for branch protection against main:

[plebhash]

we'll need to remove ci from this list (branch protections for main), while replacing for integration-required:

[plebhash]

@Shourya742 suggested researching on how to leverate nextest to spawn each test as a separate OS process

todo: research which config parameter would allow for that, and whether it improves determinism in a meaningful way

[plebhash]

todo:

write a script that will execute the entire test stack (as if it were from CI) multiple repeated times, so we have statistical sampling of how deterministic integration_casa21_runner will be

that's important because the current approach makes integration_casa21_runner take precedence also for the failure cases

overall, if there's something wrong with the PR, we want CI to fail fast (that's kinda the point of accelerating CI execution)

but we DON'T want CI to constantly fail due to flaky tests and force us to constantly re-trigger integration_casa21_runner

TLDR: I'm only going to make this PR ready for review when I get >90% determinism from a sufficiently large statistical sample... otherwise integration_casa21_runner would be just PITA instead of an actual improvement