Cbf add broadcast#30
Conversation
Add stub methods/functions, add basic build and start of the CBF chain source as well as basic struct containing the fields which undoubtedtly are needed.
Previously tests assumed that the chain source of the lightning node and is node which mines. This is not the case with CBF chain source which needs to wait until after mining a new block a new tips propagates to it. `wait_for_block` is made to return a new height and a new function `wait_for_node_tip` is added which waits until the given height is processed (returned via `status.best_block` ) on a given node.
Ask wallet for revealed spks, register them. Implement `Listen` trait ans add register_script method as well as implementation of registered scripts/outputs.
When `process_kyoto_events` processes events it decides whether we need to take any action (e.g. apply block). These actions are sent to a new abstraction — `BlockApplicator` which holds `ChainListener` (which in turn has wallets and can apply blocks / filtered blocks). This `BlockApplicator` has to have a receiver of a channel (and `process_kyoto_events` has to have a sender to this channel. Thus we cannot create them in `new`, because we would own them at `CbfChainSource`, so they are created in `start`. Also this commit ran `cargo fmt --all` which was missed previously.
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Added 4 variants of `ChainOp`: - ConnectFull, - ConnectFiltered, - Disconnect, - Synced Now `process_kyoto_events` reacts to an event from kyoto and sends a `ChainOp` to listener (`BlockApplicator`). Note that on a filter with no match we still need to apply header and we need double check that header in the canonical chain is the relevant one and has not been reorged. Right now it is left as a todo, because of an upstream PR. Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
We implement three fee sources for the CBF chain source. 1) esplora 2) electrum. For electrum we altered the existing code from the electrum chain source to make it reusable for the CBF node We change runtime argument to `start()` to be a field in `CbfChainSource` struct, because electrum fee source explictely needs it. 3) CBF native fee source. It downloads blocks and calculates fees based on previous blocks. We maintain a cache up to `BLOCK_FEE_CACHE_CAPACITY=14` blocks. For confirmation target we use estimation vie percentil of previous blocks' fees. Also if we download a block on a matched block filter, we insert it in the cache to avoid re-downloading. As the block download might be slow, we don't fail the fee estimation, but rather use fallback fees. This is especially relevant to the very first call of `update_fee_rate_estimates` during the start. We add helper pure functions in a new file `src/util.rs`. Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
| }, | ||
| }; | ||
|
|
||
| match Package::from_vec(package.clone()) { |
There was a problem hiding this comment.
I spotted this from codex PR review , and it may be worth checking:
This part assumes that package is already in order, but LDK may give us child-first, parent-later package. LDK's BroadcasterInterface::broadcast_transactions explicitly says implementations must not assume any topological order for multi-transaction packages.
There was a problem hiding this comment.
Yes, that could happen. In that case from_vec fails and we broadcast txs one by one, and in case they are dependent, the receiving node may receive child first and reject it.
But it is what all other chain sources do:
pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
let electrum_client: Arc<ElectrumRuntimeClient> = if let Some(client) =
self.electrum_runtime_status.read().expect("lock").client().as_ref()
{
Arc::clone(client)
} else {
debug_assert!(false, "We should have started the chain source before broadcasting");
return;
};
for tx in package {
electrum_client.broadcast(tx).await;
}
}
}
bitcoind:
pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
// While it's a bit unclear when we'd be able to lean on Bitcoin Core >v28
// features, we should eventually switch to use `submitpackage` via the
// `rust-bitcoind-json-rpc` crate rather than just broadcasting individual
// transactions.
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(DEFAULT_TX_BROADCAST_TIMEOUT_SECS),
self.api_client.broadcast_transaction(tx),
);
match timeout_fut.await {
Ok(res) => match res {
Ok(id) => {
debug_assert_eq!(id, txid);
log_trace!(self.logger, "Successfully broadcast transaction {}", txid);
},
Err(e) => {
log_error!(self.logger, "Failed to broadcast transaction {}: {}", txid, e);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
},
Err(e) => {
log_error!(
self.logger,
"Failed to broadcast transaction due to timeout {}: {}",
txid,
e
);
log_trace!(
self.logger,
"Failed broadcast transaction bytes: {}",
log_bytes!(tx.encode())
);
},
}
}
}
and esplora:
pub(crate) async fn process_broadcast_package(&self, package: Vec<Transaction>) {
for tx in &package {
let txid = tx.compute_txid();
let timeout_fut = tokio::time::timeout(
Duration::from_secs(self.sync_config.timeouts_config.tx_broadcast_timeout_secs),
self.esplora_client.broadcast(tx),
);
...
So I guess we're at least not worse than other chain sources and perhaps currently package comes ordered.
There was a problem hiding this comment.
Maybe we can open a PR to ldk-node to add helper which is called inside continuously_process_broadcast_queue which topologically orders the package before submitting it to the broadcaster?
There was a problem hiding this comment.
I agree that we can add a PR to ldk-node about this. Let's keep this way for now!
Onchain wallet's scripts are pulled each time we receive `IndexedFilter` event. That way we rely on a single source of truth (wallet) instead of having two overlapping script sets. Now we pass a reference to the `onchain_wallet` to make possible get all revealed scripts from it. Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
Co-authored-by: febyeji <yeji.han@sf.snu.ac.kr>
44b6691 to
7146a40
Compare
This PR lives ontop of #29