Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Better handle request with same group id #498

Merged
merged 20 commits into from
Oct 11, 2023

Conversation

akrambek
Copy link
Contributor

@akrambek akrambek commented Oct 6, 2023

Description

The following bugs have been identified while trying to make mqtt pub request with the same client id

Due to the wrong decoder assignment we may get into a bad state
Not being able to handle unknown member id in heartbeat response

Fixes #500

jfallows
jfallows previously approved these changes Oct 6, 2023
@jfallows jfallows marked this pull request as draft October 6, 2023 17:11
@akrambek akrambek changed the title Handle unknown member id for heartbeat Better handle request with same group id Oct 7, 2023
@akrambek akrambek marked this pull request as ready for review October 7, 2023 01:21
jfallows
jfallows previously approved these changes Oct 7, 2023
jfallows
jfallows previously approved these changes Oct 7, 2023
if (!encoders.isEmpty())
{
signaler.signalNow(originId, routedId, initialId, traceId, SIGNAL_NEXT_REQUEST, 0);
}
}

private void onSynGroupRebalance(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

onSyncGroupRebalance

@@ -909,6 +951,9 @@ private void onSignal(
case SIGNAL_STREAM_INITIAL_RESET:
doStreamReset(traceId);
break;
case SIGNAL_STREAM_REPLY_WINDOW:
doStreamWindow(traceId);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is not a doXXX method as it doesn't write a frame for stream.

Please rename doStreamWindow to onStreamSignalReplyWindow.

Also, given that the signals are used to defer something that happened, it might be better to name them for where they originated, rather than how you want them to react.

For example, SIGNAL_STREAM_BEGIN (triggered from onStreamBegin) and then the reaction can either be doStreamBegin directly or doStreamBegin called from onStreamSignalBegin.

{
KafkaClientStream stream = streamsByInitialIds.get(s);
stream.cleanup(traceId);
streamsByInitialIds.remove(s);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to remove this one by one, or perhaps this instead?

...
streams.forEach(s -> streamsByInitialIds.get(s).cleanup(traceId));
streamsByInitialIds.clear();
...

Please rename streamsByInitialIds to streamsByInitialId.

streamsByInitialIds.remove(streamId);

KafkaClientStream stream = streamsByInitialIds.get(streamId);
if (stream.replyAck == stream.replySeq)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we not always set stream.replyAck = stream.replySeq so that a partial stream cannot block progress of connection window?

}
}

private void doStreamWindow(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets rename this to flushStreamWindow to avoid confusion with doStreamWindow(traceId, authorization) which sends an initial window frame on the stream.

Comment on lines 613 to 628

final long traceId = begin.traceId();
final OctetsFW extension = begin.extension();
final ProxyBeginExFW proxyBeginEx = extension.get(proxyBeginExRO::tryWrap);

String host = null;
int port = 0;

if (proxyBeginEx != null)
{
final ProxyAddressInetFW inet = proxyBeginEx.address().inet();
host = inet.destination().asString();
port = inet.destinationPort();
}

connection.doConnectionBegin(traceId, host, port);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementing the behavior here seems inconsistent with the other frame types.
Perhaps we can move it to onStreamBegin instead?

@jfallows jfallows merged commit a789253 into aklivity:develop Oct 11, 2023
5 checks passed
ankitk-me pushed a commit to ankitk-me/zilla that referenced this pull request Oct 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Group stream with same group id may get hang
2 participants