Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

parser: allow drop stats of multiple tables #38042

Merged
merged 4 commits into from
Nov 14, 2022

Conversation

YangKeao
Copy link
Member

Signed-off-by: YangKeao yangkeao@chunibyo.icu

What problem does this PR solve?

Issue Number: close #37872

Problem Summary:

DROP STATS doesn't allow dropping stats of multiple tables now.

What is changed and how it works?

This PR modifies the parser and executors to allow dropping multiple tables without partitions.

The grammar should be discussed, as the DROP STATS t1, t2 GLOBAL and DROP STATS t PARTITION xxx is not consistent now. I'm not sure whether it's expected, or I shouldn't allow dropping multiple tables with GLOBAL at the end?

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Release note

Allow dropping stats of multiple tables

@ti-chi-bot
Copy link
Member

ti-chi-bot commented Sep 21, 2022

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • mjonss
  • xhebox

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Sep 21, 2022
@mjonss
Copy link
Contributor

mjonss commented Sep 21, 2022

The grammar should be discussed, as the DROP STATS t1, t2 GLOBAL and DROP STATS t PARTITION xxx is not consistent now. I'm not sure whether it's expected, or I shouldn't allow dropping multiple tables with GLOBAL at the end?

My short answer: Only allow DROP STATS t[,t2[...]]. No need to handle GLOBAL (always table/global level) or PARTITION (since we want to always have table/global stats and all partitions)

I have a patch for always enabling GLOBAL stats for partitioned tables here, which I also see as a part of this discussion :)

Due to the plans of removing static partition prune mode, I would prefer that we also deprecating the support of partition specific configuration of statistics, and not adding support of dropping stats for a specific partition.

Unless there are plans to change the statistic usage for partitioned tables under dynamic prune mode?

I propose that we only support ANALYZE table (global) with non-default configuration (like buckets, samples etc.) and only saves the configuration on table/global level.

My reasoning is that the partition level statistics is aggregated to table/global level and should not differ in configuration which may result in issues when calculating the table/global statistic.

Meaning the following syntax should be supported:
ANALYZE TABLE <list of tables> ... No change.
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> [INDEX <list of indexes>] No additional configuration allowed or used, but still allowed to update the statistics from a subset of partitions.
ALTER TABLE <partitioned table> ANALYZE PARTITION <list of partitions> [INDEX <list of indexes> No additional configuration allowed or used, but still allowed to update the statistics from a subset of partitions.
DROP STATS <list of tables> (no mentioning of partitions allowed or supported)

And this would be deprecated:
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> AllColumnsOrPredicateColumnsOpt AnalyzeOptionListOpt
ALTER TABLE <partitioned table> ANALYZE PARTITION <list of partitions> AllColumnsOrPredicateColumnsOpt AnalyzeOptionListOpt
ANALYZE INCREMENTAL TABLE <partitioned table> PARTITION...
ANALYZE TABLE <partitioned table> PARTITION <list of partitions> COLUMNS...

executor/simple.go Outdated Show resolved Hide resolved
@@ -1670,15 +1670,14 @@ func (e *SimpleExec) executeAlterInstance(s *ast.AlterInstanceStmt) error {
func (e *SimpleExec) executeDropStats(s *ast.DropStatsStmt) (err error) {
h := domain.GetDomain(e.ctx).StatsHandle()
var statsIDs []int64
if s.IsGlobalStats {
statsIDs = []int64{s.Table.TableInfo.ID}
if s.IsGlobalStats || len(s.PartitionNames) == 0 {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If my proposal that we only support GLOBAL/per table DROP STATS, then I think this should always be done.

@YangKeao YangKeao force-pushed the fix-37872 branch 2 times, most recently from 37e3413 to 356b6f7 Compare September 22, 2022 07:57
@YangKeao
Copy link
Member Author

My short answer: Only allow DROP STATS t[,t2[...]]. No need to handle GLOBAL (always table/global level) or PARTITION (since we want to always have table/global stats and all partitions)

@mjonss Good answer! IIUC, DROP STATS ... GLOBAL and DROP STATS ... PARTITION ... will be deprecated? In the latest update of this patch, I add warnings for these two syntax, and don't touch the function of them (before they are useless). In conclusion, this PR becomes the following change:

  1. Deprecate the DROP STATS ... GLOBAL and DROP STATS ... PARTITION ... syntax
  2. Allow dropping stats for multiple tables through DROP STATS a,b,c,d

PTAL

@YangKeao YangKeao force-pushed the fix-37872 branch 4 times, most recently from 962ea79 to d313663 Compare September 22, 2022 10:13
Copy link
Contributor

@mjonss mjonss left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just a minor suggestion on comment change.

executor/simple.go Outdated Show resolved Hide resolved
@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label Sep 22, 2022
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 1, 2022
Copy link
Contributor

@xhebox xhebox left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@YangKeao Rebase please. Also, don't forget pingcap/doc changes.

@ti-chi-bot ti-chi-bot added status/LGT2 Indicates that a PR has LGTM 2. and removed status/LGT1 Indicates that a PR has LGTM 1. labels Oct 28, 2022
@xhebox
Copy link
Contributor

xhebox commented Nov 11, 2022

@YangKeao rebase please, let us merge it.

Signed-off-by: YangKeao <yangkeao@chunibyo.icu>
@ti-chi-bot ti-chi-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 14, 2022
@xhebox
Copy link
Contributor

xhebox commented Nov 14, 2022

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: bebee8b

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label Nov 14, 2022
@YangKeao
Copy link
Member Author

@YangKeao Rebase please. Also, don't forget pingcap/doc changes.

I have updated the documents in pingcap/docs-cn#11986 and pingcap/docs#11287. Could you please tech-review these documents? @mjonss @xhebox . I only update the syntax (following the DROP TABLE ... style, didn't add more examples about dropping multiple tables). The DROP STATS ... PARTITION ... and DROP STATS ... GLOBAL are not updated there, because they are deprecated (though, have never been on the documents).

@ti-chi-bot ti-chi-bot merged commit b0c3381 into pingcap:master Nov 14, 2022
@sre-bot
Copy link
Contributor

sre-bot commented Nov 14, 2022

TiDB MergeCI notify

🔴 Bad News! New failing [1] after this pr merged.
These new failed integration tests seem to be caused by the current PR, please try to fix these new failed integration tests, thanks!

CI Name Result Duration Compare with Parent commit
idc-jenkins-ci-tidb/mybatis-test 🟥 failed 1, success 0, total 1 11 min New failing
idc-jenkins-ci-tidb/integration-ddl-test 🔴 failed 1, success 5, total 6 44 min Existing failure
idc-jenkins-ci/integration-cdc-test ✅ all 39 tests passed 19 min Fixed
idc-jenkins-ci-tidb/integration-common-test 🟢 all 17 tests passed 13 min Existing passed
idc-jenkins-ci-tidb/common-test 🟢 all 11 tests passed 10 min Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2 🟢 all 28 tests passed 6 min 41 sec Existing passed
idc-jenkins-ci-tidb/tics-test 🟢 all 1 tests passed 6 min 32 sec Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1 🟢 all 26 tests passed 5 min 34 sec Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test 🟢 all 1 tests passed 2 min 43 sec Existing passed
idc-jenkins-ci-tidb/plugin-test 🟢 build success, plugin test success 4min Existing passed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-note Denotes a PR that will be considered when it comes time to generate release notes. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT2 Indicates that a PR has LGTM 2.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add multiple tables to 'DROP STATS'
5 participants