Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

operator: Add automatic stream sharding support #11091

Merged
merged 22 commits into from
Nov 23, 2023
Merged

operator: Add automatic stream sharding support #11091

merged 22 commits into from
Nov 23, 2023

Conversation

periklis
Copy link
Collaborator

What this PR does / why we need it:
The following PR adds automatic stream sharding support for all LokiStack deployment sizes in the Loki Operator. In particular it replaces present defaults like per_stream_rate_limit: 3MB with a desiredRate: 3MB

Which issue(s) this PR fixes:
Fixes #

Special notes for your reviewer:

Checklist

  • Reviewed the CONTRIBUTING.md guide (required)
  • Documentation added
  • Tests updated
  • CHANGELOG.md updated
    • If the change is worth mentioning in the release notes, add add-to-release-notes label
  • Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
  • For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
  • If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

@periklis periklis self-assigned this Oct 31, 2023
@periklis periklis requested review from xperimental and a team as code owners October 31, 2023 12:52
@periklis periklis changed the title operator: Add automatic stream sharding operator: Add automatic stream sharding support Oct 31, 2023
xperimental
xperimental reviewed Oct 31, 2023
View reviewed changes
operator/apis/loki/v1/lokistack_types.go Outdated Show resolved Hide resolved
operator/apis/loki/v1/v1.go Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-config.yaml Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-config.yaml Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-runtime-config.yaml Outdated Show resolved Hide resolved
operator/internal/validation/lokistack.go Outdated Show resolved Hide resolved
@periklis @xperimental
Apply suggestions from code review
092bd15 
Co-authored-by: Robert Jacob <xperimental@solidproject.de>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Nov 20, 2023 •
edited
Loading

Trivy scan found the following vulnerabilities:

xperimental
xperimental reviewed Nov 21, 2023
View reviewed changes
Copy link
Collaborator

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review in code only as I'm on the move and not testing on a cluster.

operator/apis/loki/v1/lokistack_types.go Outdated Show resolved Hide resolved
operator/apis/loki/v1/v1.go Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/build.go Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-config.yaml Outdated Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-config.yaml Show resolved Hide resolved
operator/internal/manifests/internal/config/loki-runtime-config.yaml Outdated Show resolved Hide resolved
xperimental
xperimental reviewed Nov 22, 2023
View reviewed changes
Copy link
Collaborator

@xperimental xperimental left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems to work nicely 👍

Left two more comments.

operator/apis/loki/v1/lokistack_types.go Outdated Show resolved Hide resolved
operator/apis/loki/v1/v1.go Outdated Show resolved Hide resolved
@periklis periklis enabled auto-merge (squash) November 23, 2023 11:36
@periklis periklis merged commit 21a0777 into grafana:main Nov 23, 2023
14 checks passed
rhnasc pushed a commit to inloco/loki that referenced this pull request Apr 12, 2024
@periklis @xperimental @rhnasc
operator: Add automatic stream sharding support (grafana#11091)
0555c0c 
Co-authored-by: Robert Jacob <xperimental@solidproject.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@DylanGuedes DylanGuedes DylanGuedes left review comments

@JoaoBraveCoding JoaoBraveCoding JoaoBraveCoding left review comments

@xperimental xperimental xperimental approved these changes

Assignees

@periklis periklis

Labels
sig/operator size/XL
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@periklis @xperimental @DylanGuedes @JoaoBraveCoding