Repository navigation
Commit b7e694f
authored
test+perf: add bandwidth / cold-start / latency-curve baseline coverage (#7)
This PR establishes a measurement baseline that the open optimization
PRs (decode_bytearray #5, Nagle #6) can be validated against, plus
adds regression-guard coverage for several blind spots.
Designed to be merged FIRST so subsequent perf PRs see the new
scenarios as already-existing baselines in CodSpeed.
## CodSpeed scenarios (perf framework)
* **X7 — bytes round-trip RECV** (1k / 16k / 256k). Each call
round-trips `ByteBuffer.array()` over the wire and routes through
OUTPUT_CONVERTER[BYTES_TYPE] -> decode_bytearray. Without X7 the
decode_bytearray path is a measurement blind spot (every prior
macro returns int / list / void / callback). X7-16k joins the
CodSpeed CI scenario list.
* **X8 — bytes SEND** (1k / 16k / 256k). Complements X7 on the
encode_bytearray + Python -> Java direction via
ByteArrayOutputStream.write(byte[], int, int). X8-16k joins
the CodSpeed list.
Together X7-16k and X8-16k cover both halves of the byte-codec
bandwidth surface that's Nagle-sensitive on Linux.
## perf_coverage_test.py (run with `pytest -v -s`)
* **ColdStartLatencyTest (issue py4j#557)** — three measurements with
progressively more of the cold path excluded so a regression can
be localized:
- `test_full_cold_start` — fresh `java` subprocess + listening-
socket wait + py4j handshake + first call. Includes JVM class
loading; issue py4j#557's reported window. Caps <30 s.
- `test_py4j_handshake_under_500ms` — JVM is already listening;
only `JavaGateway()` + first call is timed. Isolates py4j's
own contribution (<500 ms).
- `test_first_vs_warm_call_ratio` — first call vs warm median,
catches regressions in protocol-cache fill.
* **StringArgLatencyCurveTest** — per-call latency at 16 / 256 /
4K / 16K / 64K byte string args. Fills the gap between M2b
(~5 B arg) and X7-class large payloads. Surfaces the Nagle
knee around the BufferedWriter buffer boundary (~8 K) on Linux.
* **BandwidthSummaryTest** — MB/s at 1K / 16K / 256K for both
directions (recv via ByteBuffer.array, send via BAOS.write).
Reports the numbers; asserts >0.1 MB/s floor to catch
protocol-layer regressions without flaking on hardware variance.
## Validation handle for the open PRs
* PR #5 (decode_bytearray) — `BandwidthSummaryTest` recv MB/s
should jump substantially after the list-comprehension fix.
X7-16k drops on the CodSpeed dashboard.
* PR #6 (Nagle) — `StringArgLatencyCurveTest` flattens across
the 8 K boundary; X7-16k and X8-16k drop dramatically on
Linux CodSpeed runners (issue py4j#516's 4.4s / 100 calls
reproducer is exactly X7-16k's pattern).
## Local sample output (macOS loopback)
```
[full-cold-start] subprocess.Popen -> first call: 0.31 s
[py4j-handshake-only] gateway+first-call: 1.4 ms
[first-vs-warm] first: 196us warm-median: 108.4us ratio: 1.8x
[string-arg latency curve]
16 B arg: 361.2 us/call
256 B arg: 270.6 us/call
4096 B arg: 339.6 us/call
16384 B arg: 387.6 us/call
65536 B arg: 462.9 us/call
[bandwidth summary]
size recv MB/s send MB/s
1024B 7.1 6.8
16384B 60.3 79.3
262144B 94.1 244.7
```
macOS loopback shows no Nagle penalty (Linux CodSpeed runners will);
send is ~2.5x faster than recv at 256K because recv pays the
base64-decode + list-comprehension cost in decode_bytearray that
PR #5 fixes.
Co-authored-by: Isaac1 parent c4dc138 commit b7e694f
3 files changed
Lines changed: 449 additions & 1 deletion
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
42 | 44 | | |
43 | 45 | | |
44 | 46 | | |
45 | | - | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
46 | 58 | | |
47 | 59 | | |
48 | 60 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
276 | 276 | | |
277 | 277 | | |
278 | 278 | | |
| 279 | + | |
| 280 | + | |
| 281 | + | |
| 282 | + | |
| 283 | + | |
| 284 | + | |
| 285 | + | |
| 286 | + | |
| 287 | + | |
| 288 | + | |
| 289 | + | |
| 290 | + | |
| 291 | + | |
| 292 | + | |
| 293 | + | |
| 294 | + | |
| 295 | + | |
| 296 | + | |
| 297 | + | |
| 298 | + | |
| 299 | + | |
| 300 | + | |
| 301 | + | |
| 302 | + | |
| 303 | + | |
| 304 | + | |
| 305 | + | |
| 306 | + | |
| 307 | + | |
| 308 | + | |
| 309 | + | |
| 310 | + | |
| 311 | + | |
| 312 | + | |
| 313 | + | |
| 314 | + | |
| 315 | + | |
| 316 | + | |
| 317 | + | |
| 318 | + | |
| 319 | + | |
| 320 | + | |
| 321 | + | |
| 322 | + | |
| 323 | + | |
| 324 | + | |
| 325 | + | |
| 326 | + | |
| 327 | + | |
| 328 | + | |
| 329 | + | |
| 330 | + | |
| 331 | + | |
| 332 | + | |
| 333 | + | |
| 334 | + | |
| 335 | + | |
| 336 | + | |
| 337 | + | |
| 338 | + | |
| 339 | + | |
| 340 | + | |
| 341 | + | |
| 342 | + | |
| 343 | + | |
| 344 | + | |
| 345 | + | |
| 346 | + | |
| 347 | + | |
| 348 | + | |
| 349 | + | |
| 350 | + | |
| 351 | + | |
| 352 | + | |
| 353 | + | |
| 354 | + | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
| 359 | + | |
| 360 | + | |
| 361 | + | |
| 362 | + | |
| 363 | + | |
| 364 | + | |
| 365 | + | |
| 366 | + | |
| 367 | + | |
| 368 | + | |
| 369 | + | |
| 370 | + | |
| 371 | + | |
| 372 | + | |
| 373 | + | |
| 374 | + | |
| 375 | + | |
| 376 | + | |
| 377 | + | |
| 378 | + | |
| 379 | + | |
| 380 | + | |
| 381 | + | |
| 382 | + | |
279 | 383 | | |
280 | 384 | | |
281 | 385 | | |
282 | 386 | | |
283 | 387 | | |
284 | 388 | | |
285 | 389 | | |
| 390 | + | |
| 391 | + | |
286 | 392 | | |
0 commit comments