Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Planner inferred wrong return type for firstrow agg function #34584

Closed
fixdb opened this issue May 11, 2022 · 1 comment · Fixed by #34585
Closed

Planner inferred wrong return type for firstrow agg function #34584

fixdb opened this issue May 11, 2022 · 1 comment · Fixed by #34585
Assignees
Labels
affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 affects-6.1 severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.

Comments

@fixdb
Copy link
Contributor

fixdb commented May 11, 2022

Smaller repo:

create table foo(a int, b int);
insert into foo values(0, NULL);
alter table foo set tiflash replica 1;


create table bar(a double not null, b decimal(65,0)  not null);
insert into bar values(0, 0);
alter table bar set tiflash  replica 1;

select * from foo where a=all(select a from bar where bar.b=foo.b);
ERROR 1105 (HY000): other error for mpp stream: DB::Exception: Cannot convert NULL value to non-Nullable type

select * from foo where a=all(select 1 from bar where bar.b=foo.b);
ERROR 1105 (HY000): other error for mpp stream: DB::Exception: Cannot convert NULL value to non-Nullable type

explain select * from foo where a=all(select a from bar where bar.b=foo.b);
+------------------------------------------+---------+--------------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                                       | estRows | task         | access object | operator info                                                                                                                                                                                      |
+------------------------------------------+---------+--------------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection_11                            | 1.00    | root         |               | test.foo.a, test.foo.b                                                                                                                                                                             |
| └─Apply_13                               | 1.00    | root         |               | CARTESIAN inner join, other cond:or(and(le(Column#8, 1), and(eq(cast(test.foo.a, double BINARY), Column#7), if(ne(Column#9, 0), NULL, 1))), or(eq(Column#10, 0), if(isnull(test.foo.a), NULL, 0))) |
|   ├─TableReader_15(Build)                | 1.00    | root         |               | data:TableFullScan_14                                                                                                                                                                              |
|   │ └─TableFullScan_14                   | 1.00    | cop[tikv]    | table:foo     | keep order:false, stats:pseudo                                                                                                                                                                     |
|   └─TableReader_34(Probe)                | 1.00    | root         |               | data:ExchangeSender_33                                                                                                                                                                             |
|     └─ExchangeSender_33                  | 1.00    | mpp[tiflash] |               | ExchangeType: PassThrough                                                                                                                                                                          |
|       └─Projection_29                    | 1.00    | mpp[tiflash] |               | Column#7, Column#8, Column#9, Column#10                                                                                                                                                            |
|         └─HashAgg_30                     | 1.00    | mpp[tiflash] |               | funcs:firstrow(Column#13)->Column#7, funcs:count(distinct test.bar.a)->Column#8, funcs:sum(Column#14)->Column#9, funcs:sum(Column#15)->Column#10                                                   |
|           └─ExchangeReceiver_32          | 1.00    | mpp[tiflash] |               |                                                                                                                                                                                                    |
|             └─ExchangeSender_31          | 1.00    | mpp[tiflash] |               | ExchangeType: PassThrough                                                                                                                                                                          |
|               └─HashAgg_19               | 1.00    | mpp[tiflash] |               | group by:test.bar.a, funcs:firstrow(test.bar.a)->Column#13, funcs:sum(0)->Column#14, funcs:count(1)->Column#15                                                                                     |
|                 └─Selection_28           | 0.80    | mpp[tiflash] |               | eq(test.bar.b, cast(test.foo.b, decimal(20,0) BINARY))                                                                                                                                             |
|                   └─TableFullScan_27     | 1.00    | mpp[tiflash] | table:bar     | keep order:false, stats:pseudo                                                                                                                                                                     |
+------------------------------------------+---------+--------------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

But in Tikv, it is getting expected result, same with MySQL:

mysql> set tidb_allow_mpp=0;
Query OK, 0 rows affected (0.19 sec)

mysql> select * from foo where a=all(select a from bar where bar.b=foo.b);
+------+------+
| a    | b    |
+------+------+
|    0 | NULL |
+------+------+
1 row in set (0.19 sec)

mysql> explain select * from foo where a=all(select a from bar where bar.b=foo.b);
+--------------------------------+---------+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| id                             | estRows | task      | access object | operator info                                                                                                                                                                                      |
+--------------------------------+---------+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Projection_11                  | 1.00    | root      |               | test.foo.a, test.foo.b                                                                                                                                                                             |
| └─Apply_13                     | 1.00    | root      |               | CARTESIAN inner join, other cond:or(and(le(Column#8, 1), and(eq(cast(test.foo.a, double BINARY), Column#7), if(ne(Column#9, 0), NULL, 1))), or(eq(Column#10, 0), if(isnull(test.foo.a), NULL, 0))) |
|   ├─TableReader_15(Build)      | 1.00    | root      |               | data:TableFullScan_14                                                                                                                                                                              |
|   │ └─TableFullScan_14         | 1.00    | cop[tikv] | table:foo     | keep order:false, stats:pseudo                                                                                                                                                                     |
|   └─StreamAgg_19(Probe)        | 1.00    | root      |               | funcs:firstrow(test.bar.a)->Column#7, funcs:count(distinct test.bar.a)->Column#8, funcs:sum(0)->Column#9, funcs:count(1)->Column#10                                                                |
|     └─TableReader_22           | 0.80    | root      |               | data:Selection_21                                                                                                                                                                                  |
|       └─Selection_21           | 0.80    | cop[tikv] |               | eq(test.bar.b, cast(test.foo.b, decimal(20,0) BINARY))                                                                                                                                             |
|         └─TableFullScan_20     | 1.00    | cop[tikv] | table:bar     | keep order:false, stats:pseudo                                                                                                                                                                     |
+--------------------------------+---------+-----------+---------------+----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
@fixdb fixdb added the type/bug The issue is confirmed as a bug. label May 11, 2022
@fixdb
Copy link
Contributor Author

fixdb commented May 11, 2022

See also pingcap/tiflash#4556

@seiya-annie seiya-annie added sig/planner SIG: Planner severity/major and removed sig/planner SIG: Planner labels May 12, 2022
@ti-chi-bot ti-chi-bot added may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-6.0 labels May 12, 2022
fixdb added a commit to fixdb/tidb that referenced this issue May 28, 2022
…QAll special case

Before this patch, the fisrtrow agg function infer the return type same with
the argument data type, which is wrong when the arg is not null during the
process planner decorrelate EQAll subqueries, and may cause error on TiFlash
engine.

Currently, firstrow agg function is treated like the exact representation of
aggregate group key, so the data type is the same with group key, even if the
group key is not null.

However, the return type of firstrow should be nullable, we clear the null flag
here instead of during invoking NewAggFuncDesc, in order to keep compatibility
with the existing presumption that the return type firstrow does not change
nullability, whatsoever.

This also can be testified by commit ad7102c, in which it stated that:
```
For all the aggregate functions except `first_row`, if we have an empty table
defined as t(a,b), `select agg(a) from t` would always return one row, while
`select agg(a) from t group by b` would return empty. For `first_row` which is
only used internally by tidb, `first_row(a)` would always return empty for
empty input now.
```

Close pingcap#34584
@qw4990 qw4990 added affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. and removed may-affects-4.0 This bug maybe affects 4.0.x versions. may-affects-5.1 This bug maybe affects 5.1.x versions. may-affects-5.2 This bug maybe affects 5.2.x versions. may-affects-5.3 This bug maybe affects 5.3.x versions. may-affects-5.4 This bug maybe affects 5.4.x versions. may-affects-5.0 This bug maybe affects 5.0.x versions. labels May 30, 2022
@qw4990 qw4990 added affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 and removed may-affects-6.0 labels May 30, 2022
ti-chi-bot pushed a commit that referenced this issue May 30, 2022
ti-chi-bot pushed a commit that referenced this issue Jun 17, 2022
ti-chi-bot pushed a commit that referenced this issue Jun 22, 2022
ti-chi-bot pushed a commit that referenced this issue Jun 22, 2022
ti-chi-bot pushed a commit that referenced this issue Aug 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-5.0 This bug affects 5.0.x versions. affects-5.1 This bug affects 5.1.x versions. affects-5.2 This bug affects 5.2.x versions. affects-5.3 This bug affects 5.3.x versions. affects-5.4 This bug affects 5.4.x versions. affects-6.0 affects-6.1 severity/major sig/planner SIG: Planner type/bug The issue is confirmed as a bug.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants