feat: new RPC, Bank and Cloning Pipeline

[thlorenz]

PR

Summary by CodeRabbit

• New Features

  • Magicblock Aperture: production-ready JSON‑RPC HTTP + WebSocket server with Solana‑compatible endpoints (account/program/token queries, blocks/slots, signatures, send/simulate transactions, airdrop) and live subscriptions.

• Documentation

  • RPC docs wording clarified for WebSocket event subscriptions.

• Chores

  • Updated local Solana test validator version and improved test runner for faster, parallel feedback.

: Major Refactoring & Chainlink Integration

• Major Refactoring: magicblock-bank, magicblock-rpc, magicblock-pubsub, magicblock-geyser-plugin, and several other utility/testing crates have been deleted.
• Consolidation Crate: Functionality (RPC, PubSub, Geyser, Bank logic) is primarily consolidated into the new magicblock-aperture crate. Cloning pipeline (3 crates) has been rewritten as the new magicblock-chainlink crate.
• Testing Overhaul: A new test-kit crate is added (derived from previous test-core crate). Integration tests (test-integration/) require careful review to ensure coverage of refactored and new functionality.
• Cloning Pipeline: The cloning pipeline has been reimplemented in magicblock-chainlink, lots of crates supporting the old cloning pipeline and dependency on conjunto crates were removed.
• Test Adaptations: Since only ephemeral mode is supporte at this point all accounts used in the ephemeral validator during integration tests are now delegated or escrowed

End of Summary by CodeRabbit

---

PR Summary: Major Refactoring & Chainlink Integration

• Major Refactoring: magicblock-bank, magicblock-rpc, magicblock-pubsub, magicblock-geyser-plugin, and several other utility/testing crates have been deleted.
• Consolidation Crate: Functionality (RPC, PubSub, Geyser, Bank logic) is primarily consolidated into the new magicblock-aperture crate. Cloning pipeline (3 crates) has been rewritten as the new magicblock-chainlink crate.
• Testing Overhaul: A new test-kit crate is added (derived from previous test-core crate). Integration tests (test-integration/) require careful review to ensure coverage of refactored and new functionality.
• Cloning Pipeline: The cloning pipeline has been reimplemented in magicblock-chainlink, lots of crates supporting the old cloning pipeline and dependency on conjunto crates were removed.
• Test Adaptations: Since only ephemeral mode is supporte at this point all accounts used in the ephemeral validator during integration tests are now delegated or escrowed

---

Review Guide

Phase 1: Review New Core Crates

1. magicblock-aperture:
   • Entry Point: lib.rs::JsonRpcServer::run -> server/ -> requests/ -> state/.
   • Review: How RPC/WebSocket requests are handled and dispatched. Verify logic migrated from magicblock-rpc, magicblock-pubsub, and parts of magicblock-bank is correct. Understand its interaction with magicblock-processor (sending transactions/requests) and accountsdb/ledger (fetching state). This crate replaces many deleted ones.
2. magicblock-chainlink:
   • Entry Point: lib.rs -> chainlink/ -> remote_account_provider/.
   • Related: magicblock-account-cloner/src/lib.rs
   • Review: The reimplemented cloning pipeline logic, and how it provides account data via RemoteAccountProvider. Check error handling.

---

Phase 2: Review Core Validator Adaptations

3. magicblock-core:
   • Entry Point: link.rs::link.
   • Review: The channels and types defined for communication between ingress components (like aperture) and magicblock-processor. Ensure data structures are appropriate for the refactored architecture. The most important type is TransactionSchedulerHandle which is the only way to interact with execution engine (replaces execute_* methods in the bank)
4. magicblock-processor:
   • Entry Point: lib.rs -> scheduler.rs -> executor/processing.rs.
   • Review: The transaction processing flow now that magicblock-bank is gone. Examine how transactions are received from sources (like aperture via magicblock-core::link) and executed. Verify state updates, fee logic, built-in handling, interacts with the rest of the validator via magicblock-core::link and shared state (accountsdb/ledger)

---

Phase 3: Validate Refactoring & Interactions

5. Confirm Functionality Migration: For each significant deleted crate (bank, rpc, pubsub, geyser), explicitly confirm its core responsibilities are correctly handled within magicblock-aperture or other remaining crates.
6. Trace Key Flows:
   • Trace an RPC request (getAccountInfo) -> aperture -> accountsdb.
   • Trace transaction submission -> aperture -> core::link -> processor.
   • Trace WebSocket subscription -> aperture -> relevant state updates.
   • Trace Chainlink data fetch -> chainlink -> accountsdb/basechain

Tips

1. Removed files can be totally ingored.
2. It's better NOT to use top down approach, and review the crates in isolation starting from recommended entrypoints.
3. Divide and conquer review strategy is encouraged, by distributing the code on the basis of individual crates.
4. Tests are better reviewed last.
5. In most cases reviewing a diff is not very helpful

Code owner relevant code review

1. All the changes in the committor-service crate should be carefully reviewed by @taco-paco.
2. All the changes in the task-scheduler` crate should be carefully reviewed by @Dodecahedr0x.
3. Cloning pipeline changes could be reviewed by either @bmuddha or @GabrielePicco

Mainly the changes in these crates are related to the removal of the bank and different approach to the transaction execution.

Deprecated Tests

I decided to deprecate chainlink unit tests that call into the program directly using the solana test context with banks client underneath. I wrote those and they are also covered with integration tests now.
Without hacking the SVM crate with special exceptions while testing they won't run since we cannot ever add a delegated account to the banks client nor can we run a modification since we cannot make the validator auth privileged either (tried a bunch of stuff).

---

Follow Up Work

• bring back support for non-ephemeral modes (making tests easier + needed to use our validator as part of dev tooling)
• add support for GRPC based pubsub client (work on this has started)
• review configuration options for possible removal of obsolete options, i.e. frequent commits and changing naming to reflect its use better
• more issues are tracking other items/edge cases we need to address
• auto airdrop will be supported temporarily on a separate branch since only few clients still depend on it

Migration to new AccountsDB

Since the account format has been modified in accountsdb, the data migration is necessary, which can be achieved with a migration tool