Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lightning: Table Empty Check Cannot Find Imported Dirty Data on Previous Failed Imports #39477

Closed
dsdashun opened this issue Nov 30, 2022 · 4 comments · Fixed by #39604
Closed
Labels
affects-6.1 affects-6.5 component/lightning This issue is related to Lightning of TiDB. severity/major type/bug The issue is confirmed as a bug.

Comments

@dsdashun
Copy link
Contributor

Bug Report

Please answer these questions before submitting your issue. Thanks!

1. Minimal reproduce step (Required)

  1. Import the data using Lightning, and halt it half-way during importing the data records, but before importing any indices .
  2. Import the data a second time.

2. What did you expect to see? (Required)

The second import should not be allowed (if check-requirements is turned on ), because the table has actually imported some record data in the previous import.

3. What did you see instead (Required)

  • The second import can be executed successfully
  • If checking the record count, we can find inconsistencies on these two statements:
SELECT COUNT(*) FROM t;
SELECT COUNT(*) FROM t USE INDEX();

Seems the dirty data is still inside the table, and it cannot be cleared unless dropping the table and re-import again.

4. What is your TiDB version? (Required)

Latest master version

@dsdashun dsdashun added the type/bug The issue is confirmed as a bug. label Nov 30, 2022
@dsdashun
Copy link
Contributor Author

/component lightning

@ti-chi-bot ti-chi-bot added the component/lightning This issue is related to Lightning of TiDB. label Nov 30, 2022
@dsdashun
Copy link
Contributor Author

/severity moderate

@lance6716
Copy link
Contributor

When lightning failed the first time, it should modify the checkpoint to an invalid state so the second import will report error at the beginning. Does this behaviour still work?

@dsdashun
Copy link
Contributor Author

dsdashun commented Dec 5, 2022

When lightning failed the first time, it should modify the checkpoint to an invalid state so the second import will report error at the beginning. Does this behaviour still work?

It should have no problem. Because it is not related to the the checkpoint implementations. It is just a pre-check refinement. The current mechanism on checkpoints still applies.

@GMHDBJD GMHDBJD removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-6.0 may-affects-6.2 may-affects-6.3 may-affects-6.4 labels Feb 1, 2023
GMHDBJD pushed a commit to GMHDBJD/tidb that referenced this issue Feb 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-6.1 affects-6.5 component/lightning This issue is related to Lightning of TiDB. severity/major type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants