Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[kvexec] customized operator for count aggregation #8222

Merged
merged 12 commits into from
Aug 9, 2024
Merged

Conversation

max-hoffman
Copy link
Contributor

@max-hoffman max-hoffman commented Aug 7, 2024

Operators that count the number of rows in a relation don't have to deserialize those KV's from storage (ex: select count(y) from xy where x > 1). There are some circumstances where we have to check for field nullability, but otherwise we can just count the KV's returned by the source iterator.

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Aug 7, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
17c017f ok 5937457
version total_tests
17c017f 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 is_faster
tpcc-scale-factor-1 59.99 62.19 0
test_name server_name server_version tps test_name server_name server_version tps is_faster
tpcc-scale-factor-1 dolt b2f0662 39.37 tpcc-scale-factor-1 dolt 17c017f 39.75 0

@coffeegoddd
Copy link
Contributor

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
f8fc91f ok 5937457
version total_tests
f8fc91f 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency_median to_latency_median is_faster
covering_index_scan 2.97 1.18 1
groupby_scan 17.32 17.01 0
index_join 2.66 2.66 0
index_join_scan 2.14 2.14 0
index_scan 54.83 54.83 0
oltp_point_select 0.3 0.29 0
oltp_read_only 5.88 5.88 0
select_random_points 0.65 0.65 0
select_random_ranges 0.83 0.8 0
table_scan 54.83 54.83 0
types_table_scan 142.39 144.97 0
write_tests from_latency_median to_latency_median is_faster
oltp_delete_insert 5.88 5.88 0
oltp_insert 2.97 2.97 0
oltp_read_write 12.08 11.87 0
oltp_update_index 3.02 2.97 0
oltp_update_non_index 2.97 2.91 0
oltp_write_only 6.09 6.09 0
types_delete_insert 6.43 6.43 0

@max-hoffman
Copy link
Contributor Author

#benchmark

Copy link

github-actions bot commented Aug 8, 2024

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

test_name from_latency_p95 to_latency_p95 is_faster
tpcc-scale-factor-1 61.08 61.08 0
test_name server_name server_version tps test_name server_name server_version tps is_faster
tpcc-scale-factor-1 dolt a50b1b7 39.0 tpcc-scale-factor-1 dolt bb31dbd 39.84 0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

read_tests from_latency_median to_latency_median is_faster
covering_index_scan 2.91 1.21 1
groupby_scan 17.01 17.01 0
index_join 2.66 2.61 0
index_join_scan 2.14 2.18 0
index_scan 54.83 54.83 0
oltp_point_select 0.3 0.3 0
oltp_read_only 5.99 5.99 0
select_random_points 0.65 0.65 0
select_random_ranges 0.83 0.8 0
table_scan 55.82 54.83 0
types_table_scan 142.39 142.39 0
write_tests from_latency_median to_latency_median is_faster
oltp_delete_insert 5.99 5.99 0
oltp_insert 3.02 2.97 0
oltp_read_write 12.08 12.08 0
oltp_update_index 3.02 3.02 0
oltp_update_non_index 2.97 2.97 0
oltp_write_only 6.21 6.21 0
types_delete_insert 6.55 6.43 0

@max-hoffman max-hoffman requested a review from zachmu August 8, 2024 19:55
Copy link
Member

@zachmu zachmu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

doRowexec: true,
},
{
name: "reject complex count expressi on",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

spacing

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
ff89161 ok 5937457
version total_tests
ff89161 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
4fc3498 ok 5937457
version total_tests
4fc3498 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@max-hoffman DOLT

comparing_percentages
100.000000 to 100.000000
version result total
d35a051 ok 5937457
version total_tests
d35a051 5937457
correctness_percentage
100.0

@coffeegoddd
Copy link
Contributor

@coffeegoddd DOLT

comparing_percentages
100.000000 to 100.000000
version result total
20c6e26 ok 5937457
version total_tests
20c6e26 5937457
correctness_percentage
100.0

@max-hoffman max-hoffman merged commit 918580a into main Aug 9, 2024
21 checks passed
@max-hoffman max-hoffman deleted the max/kv-count-agg branch August 9, 2024 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants