Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

types: always handle overflow error outside the types package #47997

Merged
merged 5 commits into from
Oct 30, 2023

Conversation

lcwangchao
Copy link
Collaborator

@lcwangchao lcwangchao commented Oct 26, 2023

What problem does this PR solve?

Issue Number: close #47517

All functions in types package should respect settings types.Context. However, for most functions, it will ignore overflowAsWarning settings and return this error directly. To make sure all methods follows the same behavior, we unified this behavior to always return overflow error and the overflow handle should be outside the types package

What is changed and how it works?

  1. Remove HandleOverflow in types.ConvertJSONToInt and types.ProduceDecWithSpecifiedTp
  2. Add HandleOverflow in builtinCastXXXAsDecimalSig.evalDecimal and builtinCastJSONAsIntSig.evalInt.
  3. Remove useless FlagIgnoreOverflowError, FlagZeroDateAsWarning, FlagZeroInDateAsWarning and FlagInvalidDateAsWarning flags.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No need to test
    • I checked and no code files have been changed.

Side effects

  • Performance regression: Consumes more CPU
  • Performance regression: Consumes more Memory
  • Breaking backward compatibility

Documentation

  • Affects user behaviors
  • Contains syntax changes
  • Contains variable changes
  • Contains experimental features
  • Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot ti-chi-bot bot added do-not-merge/invalid-title do-not-merge/needs-tests-checked release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 26, 2023
@lcwangchao lcwangchao changed the title Json overflow types: always return overflow error for the functions in types package Oct 26, 2023
@lcwangchao lcwangchao changed the title types: always return overflow error for the functions in types package types: always handle overflow error outside the types package Oct 26, 2023
@codecov
Copy link

codecov bot commented Oct 26, 2023

Codecov Report

Merging #47997 (e2099f9) into master (f135464) will increase coverage by 1.1829%.
The diff coverage is 85.7142%.

Additional details and impacted files
@@               Coverage Diff                @@
##             master     #47997        +/-   ##
================================================
+ Coverage   71.5845%   72.7674%   +1.1829%     
================================================
  Files          1401       1424        +23     
  Lines        405930     412323      +6393     
================================================
+ Hits         290583     300037      +9454     
+ Misses        95533      93370      -2163     
+ Partials      19814      18916       -898     
Flag Coverage Δ
integration 42.3521% <81.4285%> (?)
unit 71.5679% <75.7142%> (-0.0166%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
dumpling 54.0503% <ø> (ø)
parser ∅ <ø> (∅)
br 48.6251% <ø> (-4.2606%) ⬇️

@lcwangchao
Copy link
Collaborator Author

/retest

Copy link
Member

@YangKeao YangKeao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Do we agree that the error handling in types package should follow similar pattern? Which means the function in types package should do its best-effort, and return valid values and all errors, the caller of these functions should take the responsibility to handle these errors.

Also, should we do the same refractoring for HandleTruncate in the future?

The problem is that, some functions will return both TruncateError and OverflowError (in different cases), then we may need to provide a function to handle all possible errors 🤔, but I wonder whether it's too hard/complex to implement this function right considering all these situations. As MySQL itself is not clean, the attitude to warnings in different situations is different. For example, casting a json string to integer can have different warnings compared with casting a normal string to integer:

select cast('-1' as unsigned); -- has warning
select cast(cast('"-1"' as json) as unsigned); -- no warning

(Though, TiDB doesn't do well in warning compatibility now. Maybe it's not a big problem).

@lcwangchao
Copy link
Collaborator Author

@YangKeao

Do we agree that the error handling in types package should follow similar pattern? Which means the function in types package should do its best-effort, and return valid values and all errors, the caller of these functions should take the responsibility to handle these errors.

Yes, I agree with you. The best way is to remove all flags and force return the error. However, I'm not sure it is practice now because too many places rely on the error processing inside the types package. Maybe we should reconsider it after finishing decoupling types and stmtctx.

The problem is that, some functions will return both TruncateError and OverflowError (in different cases), then we may need to provide a function to handle all possible errors 🤔, but I wonder whether it's too hard/complex to implement this function right considering all these situations.

Maybe need serval functions to translate SQLMode and other information such as statement type to ErrorHandleContext. At this time, we can still have one function to handle these error and just make the ErrorHandleContext as the input parameter which has different settings in different scene.

@lcwangchao
Copy link
Collaborator Author

/retest

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 30, 2023

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: XuHuaiyu, YangKeao

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 30, 2023

[LGTM Timeline notifier]

Timeline:

  • 2023-10-26 08:37:56.413029675 +0000 UTC m=+2510274.000139817: ☑️ agreed by YangKeao.
  • 2023-10-30 03:40:01.822676784 +0000 UTC m=+2837999.409786928: ☑️ agreed by XuHuaiyu.

@lcwangchao
Copy link
Collaborator Author

/retest

@lcwangchao
Copy link
Collaborator Author

/retest

@ti-chi-bot ti-chi-bot bot merged commit 5503eb5 into pingcap:master Oct 30, 2023
12 of 16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved lgtm release-note-none Denotes a PR that doesn't merit a release note. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

always return overflow error for the functions in types package
3 participants