Repository navigation
Java SDK: Change coordinator comm to use multiplexing - #69254
Merged
jason810496 merged 1 commit intoJul 30, 2026
Merged
Conversation
uranusjr
reviewed
Jul 2, 2026
uranusjr
reviewed
Jul 2, 2026
uranusjr
reviewed
Jul 2, 2026
uranusjr
reviewed
Jul 2, 2026
uranusjr
reviewed
Jul 2, 2026
FrankYang0529
force-pushed
the
airflow-java-sdk-multiplexing
branch
from
July 3, 2026 07:46
e159284 to
d847241
Compare
uranusjr
approved these changes
Jul 6, 2026
jason810496
reviewed
Jul 18, 2026
jason810496
left a comment
jason810496
left a comment
Member
There was a problem hiding this comment.
Sorry for the late review as it took me a longer time than I expected to understand all the kotlin specific concurrency concept and syntax in this PR. Here're some comments from CC that still make sense to me.
FrankYang0529
force-pushed
the
airflow-java-sdk-multiplexing
branch
2 times, most recently
from
July 26, 2026 04:48
a267532 to
0c6bf38
Compare
Serializing each client's whole send → receive → deserialize round trip behind one lock (apache#69080) allows only a single request in flight at a time, so a task issuing concurrent calls over the shared comm socket pays a full, serialized round trip per call. Every frame already carries a request id, so a single background dispatcher can read the socket and route each response by id, with the lock guarding only the write. Signed-off-by: PoAn Yang <payang@apache.org>
FrankYang0529
force-pushed
the
airflow-java-sdk-multiplexing
branch
from
July 26, 2026 06:07
0c6bf38 to
89fe9f5
Compare
dabla
pushed a commit
to dabla/airflow
that referenced
this pull request
Aug 14, 2026
Serializing each client's whole send → receive → deserialize round trip behind one lock (apache#69080) allows only a single request in flight at a time, so a task issuing concurrent calls over the shared comm socket pays a full, serialized round trip per call. Every frame already carries a request id, so a single background dispatcher can read the socket and route each response by id, with the lock guarding only the write. Signed-off-by: PoAn Yang <payang@apache.org>
imrichardwu
pushed a commit
to imrichardwu/airflow
that referenced
this pull request
Sep 11, 2026
Serializing each client's whole send → receive → deserialize round trip behind one lock (apache#69080) allows only a single request in flight at a time, so a task issuing concurrent calls over the shared comm socket pays a full, serialized round trip per call. Every frame already carries a request id, so a single background dispatcher can read the socket and route each response by id, with the lock guarding only the write. Signed-off-by: PoAn Yang <payang@apache.org>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
#69080 made the coordinator comm thread-safe by holding a single lock across each client's entire send → receive → deserialize round trip. It's correct, but only one request can be in flight at a time, so a task that issues concurrent calls over the shared comm socket pays a full, serialized round trip per call.
How
Every frame already carries a request id, so a single background dispatcher (
readLoop) becomes the sole socket reader and routes each response to the waiter registered under that id — the lock now guards only the write, so concurrent calls can overlap.Was generative AI tooling used to co-author this PR?
{pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.