Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

server (ticdc): add timeout for statusServer. #5332

Merged
merged 9 commits into from
May 6, 2022
Merged

server (ticdc): add timeout for statusServer. #5332

merged 9 commits into from
May 6, 2022

Conversation

asddongmen
Copy link
Contributor

@asddongmen asddongmen commented May 5, 2022 •
edited
Loading

What problem does this PR solve?

Issue Number: close #5303

What is changed and how it works?

Invesgation Result:

  1. The requests sent by the client are not closed properly, resulting in http.server connection that cannot be closed and keeps goroutine growing.
  2. adminClient is not closed properly when NewKafkaSaramaSink is failed, which can lead to goruotine leaks.

Waht is changed:

  1. Add ReadTimeout and WriteTimeout for statusServer to limit a connection maxmum alive time.
  2. Add maxConnection for listener of statusServer to limit the maxmum concurrent connections.
  3. Close sarama adminClient when NewKafkaSaramaSink return with error to avoid goroutine leaks.

Check List

Tests

  • Manual test (add detailed scripts or steps below)

Code changes

Side effects

Related changes

Release note 

Fix a bug that may causes goroutine leak in http server.

@ti-chi-bot
Copy link
Member

ti-chi-bot commented May 5, 2022 •
edited
Loading

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • CharlesCheung96
  • liuzix

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 5, 2022
@asddongmen asddongmen added status/ptal Could you please take a look? component/sink Sink component. component/status-server Status server component. labels May 5, 2022
liuzix
liuzix reviewed May 5, 2022
View reviewed changes
cdc/api/open.go Show resolved Hide resolved
cdc/server.go Outdated Show resolved Hide resolved
cdc/sink/mq/mq.go Outdated Show resolved Hide resolved
Rustin170506
Rustin170506 reviewed May 6, 2022
View reviewed changes
Copy link
Member

@Rustin170506 Rustin170506 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rest LGTM

cdc/server.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels May 6, 2022
@asddongmen
Copy link
Contributor Author

/integration-tests

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 6, 2022
@asddongmen
Copy link
Contributor Author

/run-integration-tests

@ti-chi-bot ti-chi-bot removed the status/LGT1 Indicates that a PR has LGTM 1. label May 6, 2022
@ti-chi-bot ti-chi-bot added the status/LGT2 Indicates that a PR has LGTM 2. label May 6, 2022
@asddongmen
Copy link
Contributor Author

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 5cae31f

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 6, 2022
@asddongmen
Copy link
Contributor Author

/run-verify

4 similar comments
@asddongmen
Copy link
Contributor Author

/run-verify

@asddongmen
Copy link
Contributor Author

/run-verify

@asddongmen
Copy link
Contributor Author

/run-verify

@asddongmen
Copy link
Contributor Author

/run-verify

@ti-chi-bot ti-chi-bot merged commit 063b528 into pingcap:master May 6, 2022
ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request May 6, 2022
@asddongmen @ti-chi-bot
This is an automated cherry-pick of pingcap#5332
8c115bf 
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
@ti-chi-bot ti-chi-bot mentioned this pull request May 6, 2022
Merged
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5346.

ti-chi-bot pushed a commit to ti-chi-bot/tiflow that referenced this pull request May 6, 2022
@asddongmen @ti-chi-bot
server (ticdc): add timeout for statusServer. (pingcap#5332)
71573d1 
close pingcap#5303
@ti-chi-bot ti-chi-bot mentioned this pull request May 6, 2022
Closed
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #5347.

@asddongmen asddongmen deleted the fix_http_leak branch May 19, 2022 12:11
asddongmen added a commit to ti-chi-bot/tiflow that referenced this pull request Jun 27, 2022
@asddongmen
This an manual cherry-pick of pingcap#5332
a789081
ti-chi-bot added a commit that referenced this pull request Jun 27, 2022
@ti-chi-bot
server (ticdc): add timeout for statusServer. (#5332) (#5346)
4d7dc3d 
close #5303
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@overvenus overvenus overvenus left review comments

@Rustin170506 Rustin170506 Rustin170506 left review comments

@liuzix liuzix liuzix approved these changes

@CharlesCheung96 CharlesCheung96 CharlesCheung96 approved these changes

Assignees
No one assigned
Labels
component/sink Sink component. component/status-server Status server component. needs-cherry-pick-release-5.4 Should cherry pick this PR to release-5.4 branch. needs-cherry-pick-release-6.0 Should cherry pick this PR to release-6.0 branch. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2. status/ptal Could you please take a look?
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Goroutine leak when usign /api/v1/changefeed to create kafka changefeed
6 participants
@asddongmen @ti-chi-bot @overvenus @liuzix @Rustin170506 @CharlesCheung96