fix: Ensure Drain patterns are valid for LogQL pattern match filter

[benclive]

What this PR does / why we need it:
This fixes a bug where the Drain generates patterns which don't match within the LogQL engine.
The bug prevented the "add pattern to filter" functionality in the Logs app from working in some cases where the log line ends with whitespace or newlines.
This is a common case for logs generated from the loki-canary app as they all terminate with a new line character.

• Fixed the bug by removing the TrimSpace call when tokenising the input.
• Added tests to ensure that Drain patterns can be matched by the LogQL pattern filter for the input lines.

Checklist

• Reviewed the CONTRIBUTING.md guide (required)
• Documentation added
• Tests updated
• Title matches the required conventional commits format, see here
• Changes that require user attention or interaction to upgrade are documented in docs/sources/setup/upgrade/_index.md
• For Helm chart changes bump the Helm chart version in production/helm/loki/Chart.yaml and update production/helm/loki/CHANGELOG.md and production/helm/loki/README.md. Example PR
• If the change is deprecating or removing a configuration option, update the deprecated-config.yaml and deleted-config.yaml files respectively in the tools/deprecated-config-checker directory. Example PR

[grafanabot]

Hello @JStickler!
Backport pull requests need to be either:

• Pull requests which address bugs,
• Urgent fixes which need product approval, in order to get merged,
• Docs changes.

Please, if the current pull request addresses a bug fix, label it with the type/bug label.
If it already has the product approval, please add the product-approved label. For docs changes, please add the type/docs label.
If the pull request modifies CI behaviour, please add the type/ci label.
If none of the above applies, please consider removing the backport label and target the next major/minor release.
Thanks!

[grafanabot]

This PR must be merged before a backport PR will be created.

[cyriltovena]

LGTM

[grafanabot]

The backport to release-3.0.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-12815-to-release-3.0.x origin/release-3.0.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x fd2301fd62b18eb345bc43868b40343efc1a1f10

When the conflicts are resolved, stage and commit the changes:

git add . && git cherry-pick --continue

If you have the GitHub CLI installed:

# Push the branch to GitHub:
git push --set-upstream origin backport-12815-to-release-3.0.x
# Create the PR body template
PR_BODY=$(gh pr view 12815 --json body --template 'Backport fd2301fd62b18eb345bc43868b40343efc1a1f10 from #12815{{ "\n\n---\n\n" }}{{ index . "body" }}')
# Create the PR on GitHub
echo "${PR_BODY}" | gh pr create --title "chore: [release-3.0.x] fix: Ensure Drain patterns are valid for LogQL pattern match filter" --body-file - --label "size/L" --label "type/bug" --label "backport" --base release-3.0.x --milestone release-3.0.x --web

Or, if you don't have the GitHub CLI installed (we recommend you install it!):

# Push the branch to GitHub:
git push --set-upstream origin backport-12815-to-release-3.0.x

# Create a pull request where the `base` branch is `release-3.0.x` and the `compare`/`head` branch is `backport-12815-to-release-3.0.x`.

# Remove the local backport branch
git switch main
git branch -D backport-12815-to-release-3.0.x