Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Many empty partitions cause statistics.PseudoTable to take up much memory #44308

Closed
pcqz opened this issue May 31, 2023 · 1 comment · Fixed by #44374
Closed

Many empty partitions cause statistics.PseudoTable to take up much memory #44308

pcqz opened this issue May 31, 2023 · 1 comment · Fixed by #44374
Assignees
@hawkingrei
Labels
affects-6.1 affects-6.5 affects-7.1 severity/critical sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@pcqz
Copy link

pcqz commented May 31, 2023

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

Create 20 partitioned tables without data, each table has 60 columns and 2000 partitions.

2. What did you expect to see? (Required)

Since there is no data and statistics, no major change in memory usage.

3. What did you see instead (Required)

TiDB memory usage increases abnormally and continues to increase after each truncate of these tables. Most of memory are taken up by statistics.PseudoTable.

img_v2_63c2495b-5363-4cc1-84cd-49cc11cb813g

image

4. What is your TiDB version? (Required)

v6.5.2
@pcqz pcqz added the type/bug The issue is confirmed as a bug. label May 31, 2023
@pcqz
Copy link
Author

pcqz commented May 31, 2023

Even after dropping all the tables, memory is not released immediately. Heap profile shows that most of memory is still occupied by statistics.PseudoTable.
image

@chrysan chrysan added the sig/planner SIG: Planner label Jun 2, 2023
@ti-chi-bot ti-chi-bot bot added may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-7.1 labels Jun 2, 2023
@hawkingrei hawkingrei mentioned this issue Jun 2, 2023
Merged
12 tasks
@hawkingrei hawkingrei added affects-7.1 and removed may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-7.1 labels Jun 4, 2023
ti-chi-bot bot pushed a commit that referenced this issue Jun 5, 2023
@hawkingrei
statistics: not put pseudo table of partition table into cache (#44374)
9bac155 
close #44308
@ti-chi-bot ti-chi-bot mentioned this issue Jun 5, 2023
Merged
12 tasks
@ti-chi-bot ti-chi-bot mentioned this issue Jun 5, 2023
Merged
12 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Jun 5, 2023
@hawkingrei @ti-chi-bot
statistics: not put pseudo table of partition table into cache (pingc…
ff35c27 
…ap#44374)

close pingcap#44308
@ti-chi-bot ti-chi-bot mentioned this issue Jun 5, 2023
Merged
12 tasks
ti-chi-bot pushed a commit to ti-chi-bot/tidb that referenced this issue Jun 5, 2023
@hawkingrei @ti-chi-bot
statistics: not put pseudo table of partition table into cache (pingc…
32f635c 
…ap#44374)

close pingcap#44308
ti-chi-bot bot pushed a commit that referenced this issue Jun 5, 2023
@ti-chi-bot
statistics: not put pseudo table of partition table into cache (#44374)…
f6e2b2d 
… (#44392)

close #44308
hawkingrei added a commit to ti-chi-bot/tidb that referenced this issue Jun 20, 2023
@hawkingrei
statistics: not put pseudo table of partition table into cache (pingc…
02ba798 
…ap#44374)

close pingcap#44308
ti-chi-bot bot pushed a commit that referenced this issue Jun 29, 2023
@ti-chi-bot
statistics: not put pseudo table of partition table into cache (#44374)…
e4773fd 
… (#44391)

close #44308
ti-chi-bot bot pushed a commit that referenced this issue Jul 11, 2023
@ti-chi-bot
statistics: not put pseudo table of partition table into cache (#44374)…
0bc8b96 
… (#44393)

close #44308
@winoros winoros mentioned this issue Sep 6, 2023
Open
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hawkingrei hawkingrei

Labels
affects-6.1 affects-6.5 affects-7.1 severity/critical sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

statistics: not put pseudo table of partition table into cache hawkingrei/tidb
4 participants
@hawkingrei @jebter @chrysan @pcqz