Repository navigation
Commit 164ca2c
perf(dash-spv): size storage segments per item type
Every segment cache held 50 000 items, whatever an item costs. Near the
tip a block segment holds tens of MB of decoded blocks, while a header
segment spanning the same heights is 5.6 MB and a filter header one
1.6 MB. `Persistable::ITEMS_PER_SEGMENT` lets each type choose: headers
10 000 (~1.1 MB per segment), filter headers 50 000 (~1.6 MB), filters
2 000 (~2 MB near the tip) and blocks 1 000.
This changes the on-disk layout of the header, filter and block
segments. Storage written with 50 000-item segments is misread by this
layout and has to be deleted until a migration or a versioned folder
lands.
Mainnet restore, mainnet.100mbi.100ms, #1015/#1016/#1014 applied, two
resident segments, jemalloc heap profiling, wallet identical in every
run (14114383 sat, 7112 records, 13389 addresses):
segment items time peak RSS segment caches
50 000 for every type 7.0 min 818 MiB 332 MiB
5 000 for every type 7.3-8.3 min 554-687 MiB 23-106 MiB
1 000 for every type 9.2-12.2 min 453-577 MiB 2-66 MiB
per type (this commit) 7.5 min 538 MiB 38 MiB
With 1 000 items everywhere, the header and filter header phases paid an
fsync per evicted segment (105-141 s and 254-332 s instead of ~60 s and
~150 s). Blocks at 500 items saved ~18 MiB of cache but reloaded 50 %
more block segments.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_017DruChNTWXwJoWPartZwCf1 parent 5374756 commit 164ca2c
1 file changed
Lines changed: 10 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
27 | 27 | | |
28 | 28 | | |
29 | 29 | | |
| 30 | + | |
30 | 31 | | |
31 | 32 | | |
32 | 33 | | |
| |||
36 | 37 | | |
37 | 38 | | |
38 | 39 | | |
| 40 | + | |
| 41 | + | |
39 | 42 | | |
40 | 43 | | |
41 | 44 | | |
42 | 45 | | |
43 | 46 | | |
44 | 47 | | |
| 48 | + | |
| 49 | + | |
45 | 50 | | |
46 | 51 | | |
47 | 52 | | |
| |||
57 | 62 | | |
58 | 63 | | |
59 | 64 | | |
| 65 | + | |
| 66 | + | |
60 | 67 | | |
61 | 68 | | |
62 | 69 | | |
63 | 70 | | |
64 | 71 | | |
65 | 72 | | |
| 73 | + | |
| 74 | + | |
66 | 75 | | |
67 | 76 | | |
68 | 77 | | |
| |||
562 | 571 | | |
563 | 572 | | |
564 | 573 | | |
565 | | - | |
| 574 | + | |
566 | 575 | | |
567 | 576 | | |
568 | 577 | | |
| |||
0 commit comments