Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DisposeMany() Rework #755

Merged
merged 6 commits into from
Nov 16, 2023

Conversation

JakenVeina
Copy link
Collaborator

Reworked the .DisposeMany() operator for both caches and lists to perform disposal after downstream operations have processed, rather than before, since the operator cannot guarantee that disposal is safe, when downstream consumers may still need to make use of these disposable items, in order to process removals.

Also fixed a bug within both versions of the .SubscribeMany() operator, where errors were not being properly propagated through the stream, and instead could bubble up to stream inputs.

Also fixed a bug within the cache version of .MergeMany() that allowed an internally-used subject to be disposed, without cleaning up all subscriptions that may later make use of it.

Resolves #732

@JakenVeina
Copy link
Collaborator Author

This also appears to fix #668 based on the initial reproduction.

…erform disposal after downstream operations have processed, rather than before, since the operator cannot guarantee that disposal is safe, when downstream consumers may still need to make use of these disposable items, in order to process removals.

Also fixed a bug within both versions of the `.SubscribeMany()` operator, where errors were not being properly propagated through the stream, and instead could bubble up to stream inputs.

Also fixed a bug within the cache version of `.MergeMany()` that allowed an internally-used subject to be disposed, without cleaning up all subscriptions that may later make use of it.
@JakenVeina
Copy link
Collaborator Author

For reference, performance is ever so slightly improved, with these changes.

DisposeMany_Cache (Before)

| Method                     | addCount | removeCount | Mean       | Error     | StdDev     | Gen0     | Gen1    | Allocated  |
|--------------------------- |--------- |------------ |-----------:|----------:|-----------:|---------:|--------:|-----------:|
| AddsRemovesAndFinalization | 1        | 0           |   2.192 us | 0.0431 us |  0.0890 us |   0.7172 |       - |    2.94 KB |
| AddsRemovesAndFinalization | 1        | 1           |   2.386 us | 0.0464 us |  0.0411 us |   0.8278 |       - |    3.39 KB |
| AddsRemovesAndFinalization | 10       | 0           |   5.805 us | 0.1112 us |  0.2555 us |   2.1362 |       - |    8.74 KB |
| AddsRemovesAndFinalization | 10       | 1           |   5.910 us | 0.1136 us |  0.0948 us |   2.2507 |       - |     9.2 KB |
| AddsRemovesAndFinalization | 10       | 10          |   8.675 us | 0.1635 us |  0.2396 us |   3.2349 |       - |   13.27 KB |
| AddsRemovesAndFinalization | 100      | 0           |  38.162 us | 0.3230 us |  0.2863 us |  17.0288 |       - |    69.6 KB |
| AddsRemovesAndFinalization | 100      | 1           |  39.513 us | 0.7692 us |  1.1275 us |  17.1509 |  0.1221 |   70.05 KB |
| AddsRemovesAndFinalization | 100      | 10          |  40.460 us | 0.6119 us |  0.5110 us |  18.1274 |       - |   74.13 KB |
| AddsRemovesAndFinalization | 100      | 100         |  67.462 us | 0.8737 us |  0.7296 us |  28.0762 |       - |  114.92 KB |
| AddsRemovesAndFinalization | 1000     | 0           | 375.621 us | 7.3458 us |  6.8713 us | 164.0625 | 25.3906 |  678.25 KB |
| AddsRemovesAndFinalization | 1000     | 1           | 374.867 us | 6.1622 us |  5.7641 us | 165.0391 | 34.1797 |   678.7 KB |
| AddsRemovesAndFinalization | 1000     | 10          | 384.597 us | 6.0116 us |  5.6233 us | 166.5039 |  0.4883 |  682.77 KB |
| AddsRemovesAndFinalization | 1000     | 100         | 423.367 us | 8.4002 us | 19.6352 us | 175.7813 |  1.4648 |  723.55 KB |
| AddsRemovesAndFinalization | 1000     | 1000        | 671.497 us | 7.7032 us |  6.0141 us | 276.3672 |  1.9531 | 1131.39 KB |
DisposeMany_Cache (After)

| Method                     | addCount | removeCount | Mean       | Error     | StdDev     | Median     | Gen0     | Gen1    | Allocated  |
|--------------------------- |--------- |------------ |-----------:|----------:|-----------:|-----------:|---------:|--------:|-----------:|
| AddsRemovesAndFinalization | 1        | 0           |   1.667 us | 0.0331 us |  0.0309 us |   1.658 us |   0.6218 |       - |    2.55 KB |
| AddsRemovesAndFinalization | 1        | 1           |   1.995 us | 0.0384 us |  0.0394 us |   1.979 us |   0.7172 |       - |    2.94 KB |
| AddsRemovesAndFinalization | 10       | 0           |   4.812 us | 0.0691 us |  0.0768 us |   4.800 us |   1.8997 |       - |    7.79 KB |
| AddsRemovesAndFinalization | 10       | 1           |   5.382 us | 0.1071 us |  0.2063 us |   5.320 us |   1.9989 |       - |    8.18 KB |
| AddsRemovesAndFinalization | 10       | 10          |   8.560 us | 0.1660 us |  0.3644 us |   8.446 us |   2.8534 |       - |    11.7 KB |
| AddsRemovesAndFinalization | 100      | 0           |  36.279 us | 0.7179 us |  1.3127 us |  35.768 us |  15.3809 |       - |   63.02 KB |
| AddsRemovesAndFinalization | 100      | 1           |  35.727 us | 0.6828 us |  0.7012 us |  35.504 us |  15.5029 |       - |   63.41 KB |
| AddsRemovesAndFinalization | 100      | 10          |  39.987 us | 0.7463 us |  1.6845 us |  39.579 us |  16.3574 |       - |   66.93 KB |
| AddsRemovesAndFinalization | 100      | 100         |  66.018 us | 1.0180 us |  0.9024 us |  66.221 us |  24.9023 |       - |  102.09 KB |
| AddsRemovesAndFinalization | 1000     | 0           | 375.852 us | 7.4697 us | 15.9185 us | 369.812 us | 149.4141 | 10.2539 |  615.41 KB |
| AddsRemovesAndFinalization | 1000     | 1           | 360.066 us | 6.7198 us |  7.1901 us | 361.168 us | 144.0430 | 50.2930 |   615.8 KB |
| AddsRemovesAndFinalization | 1000     | 10          | 366.988 us | 7.3338 us | 10.0386 us | 370.218 us | 145.0195 | 31.7383 |  619.32 KB |
| AddsRemovesAndFinalization | 1000     | 100         | 379.244 us | 3.6844 us |  3.0767 us | 378.370 us | 159.6680 |  0.9766 |  654.48 KB |
| AddsRemovesAndFinalization | 1000     | 1000        | 662.214 us | 8.5587 us |  6.6821 us | 662.860 us | 246.0938 |  1.9531 | 1006.04 KB |
DisposeMany_List (Before)

| Method                     | addCount | removeCount | Mean       | Error      | StdDev     | Median     | Gen0     | Gen1   | Allocated |
|--------------------------- |--------- |------------ |-----------:|-----------:|-----------:|-----------:|---------:|-------:|----------:|
| AddsRemovesAndFinalization | 1        | 0           |   1.439 us |  0.0158 us |  0.0140 us |   1.437 us |   0.5131 |      - |    2.1 KB |
| AddsRemovesAndFinalization | 1        | 1           |   2.136 us |  0.0403 us |  0.0396 us |   2.121 us |   0.6294 |      - |   2.58 KB |
| AddsRemovesAndFinalization | 10       | 0           |   4.520 us |  0.0660 us |  0.0585 us |   4.524 us |   1.3809 |      - |   5.66 KB |
| AddsRemovesAndFinalization | 10       | 1           |   4.954 us |  0.0879 us |  0.0863 us |   4.924 us |   1.4648 |      - |   5.98 KB |
| AddsRemovesAndFinalization | 10       | 10          |   8.228 us |  0.1232 us |  0.1153 us |   8.185 us |   2.1973 |      - |   9.02 KB |
| AddsRemovesAndFinalization | 100      | 0           |  34.546 us |  0.4641 us |  0.3875 us |  34.461 us |   9.8267 |      - |  40.24 KB |
| AddsRemovesAndFinalization | 100      | 1           |  34.638 us |  0.6920 us |  1.2827 us |  34.149 us |   9.8877 |      - |  40.56 KB |
| AddsRemovesAndFinalization | 100      | 10          |  39.286 us |  0.7824 us |  2.1681 us |  38.653 us |  10.6201 |      - |  43.45 KB |
| AddsRemovesAndFinalization | 100      | 100         |  67.139 us |  1.1603 us |  1.8403 us |  66.476 us |  17.7002 |      - |  72.43 KB |
| AddsRemovesAndFinalization | 1000     | 0           | 354.357 us |  7.0484 us | 18.6914 us | 347.633 us |  92.2852 |      - | 377.76 KB |
| AddsRemovesAndFinalization | 1000     | 1           | 336.471 us |  5.8110 us |  5.1513 us | 334.512 us |  92.2852 |      - | 378.08 KB |
| AddsRemovesAndFinalization | 1000     | 10          | 346.435 us |  6.6578 us |  9.3333 us | 344.937 us |  93.2617 | 0.4883 | 380.96 KB |
| AddsRemovesAndFinalization | 1000     | 100         | 367.402 us |  6.3225 us |  5.6047 us | 366.842 us | 100.0977 | 0.4883 | 409.79 KB |
| AddsRemovesAndFinalization | 1000     | 1000        | 666.358 us | 12.8420 us | 12.6125 us | 665.016 us | 170.8984 |      - | 698.23 KB |
DisposeMany_List (After)
| Method                     | addCount | removeCount | Mean       | Error      | StdDev     | Gen0     | Gen1   | Allocated |
|--------------------------- |--------- |------------ |-----------:|-----------:|-----------:|---------:|-------:|----------:|
| AddsRemovesAndFinalization | 1        | 0           |   1.131 us |  0.0222 us |  0.0434 us |   0.4539 |      - |   1.86 KB |
| AddsRemovesAndFinalization | 1        | 1           |   1.740 us |  0.0344 us |  0.0754 us |   0.5703 |      - |   2.34 KB |
| AddsRemovesAndFinalization | 10       | 0           |   3.964 us |  0.0782 us |  0.1450 us |   1.3275 |      - |   5.42 KB |
| AddsRemovesAndFinalization | 10       | 1           |   4.129 us |  0.0450 us |  0.0421 us |   1.4038 |      - |   5.74 KB |
| AddsRemovesAndFinalization | 10       | 10          |   7.583 us |  0.1483 us |  0.2128 us |   2.1439 |      - |   8.78 KB |
| AddsRemovesAndFinalization | 100      | 0           |  31.845 us |  0.6332 us |  1.2499 us |   9.7656 |      - |     40 KB |
| AddsRemovesAndFinalization | 100      | 1           |  31.720 us |  0.6144 us |  1.1389 us |   9.8267 |      - |  40.32 KB |
| AddsRemovesAndFinalization | 100      | 10          |  34.475 us |  0.6895 us |  1.5703 us |  10.5591 |      - |   43.2 KB |
| AddsRemovesAndFinalization | 100      | 100         |  61.352 us |  0.9145 us |  0.7637 us |  17.5781 |      - |  72.19 KB |
| AddsRemovesAndFinalization | 1000     | 0           | 302.338 us |  5.9181 us | 11.4021 us |  92.2852 |      - | 377.52 KB |
| AddsRemovesAndFinalization | 1000     | 1           | 301.170 us |  5.9836 us | 12.6214 us |  92.2852 |      - | 377.84 KB |
| AddsRemovesAndFinalization | 1000     | 10          | 298.594 us |  5.9549 us |  5.5702 us |  92.7734 | 0.4883 | 380.72 KB |
| AddsRemovesAndFinalization | 1000     | 100         | 334.682 us |  6.6404 us | 14.0069 us | 100.0977 | 0.4883 | 409.55 KB |
| AddsRemovesAndFinalization | 1000     | 1000        | 634.405 us | 12.5637 us | 30.8189 us | 170.8984 |      - | 697.99 KB |

Copy link

codecov bot commented Nov 16, 2023

Codecov Report

Attention: 409 lines in your changes are missing coverage. Please review.

Comparison is base (53d5f6d) 64.74% compared to head (c51a8c1) 65.46%.
Report is 1 commits behind head on main.

Files Patch % Lines
src/DynamicData/Cache/ObservableCacheEx.cs 53.48% 19 Missing and 1 partial ⚠️
src/DynamicData/List/ObservableListEx.cs 64.10% 13 Missing and 1 partial ⚠️
src/DynamicData/Aggregation/AvgEx.cs 73.68% 10 Missing ⚠️
src/DynamicData/Aggregation/SumEx.cs 74.35% 10 Missing ⚠️
src/DynamicData/ObservableChangeSet.cs 58.33% 10 Missing ⚠️
src/DynamicData/Platforms/net45/ParallelEx.cs 25.00% 7 Missing and 2 partials ⚠️
src/DynamicData/Aggregation/StdDevEx.cs 76.66% 5 Missing and 2 partials ⚠️
src/DynamicData/List/Internal/GroupOnProperty.cs 0.00% 7 Missing ⚠️
src/DynamicData/Aggregation/CountEx.cs 25.00% 6 Missing ⚠️
src/DynamicData/Cache/Internal/CacheUpdater.cs 53.84% 6 Missing ⚠️
... and 126 more
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #755      +/-   ##
==========================================
+ Coverage   64.74%   65.46%   +0.72%     
==========================================
  Files         226      227       +1     
  Lines       11459    11091     -368     
  Branches     2334     2294      -40     
==========================================
- Hits         7419     7261     -158     
+ Misses       3083     2889     -194     
+ Partials      957      941      -16     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Collaborator

@RolandPheasant RolandPheasant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an excellent contribution, and thanks for taking the pain of getting your head around how it should work.

@ChrisPulman ChrisPulman merged commit 6cfca26 into reactivemarbles:main Nov 16, 2023
3 checks passed
@JakenVeina JakenVeina deleted the dispose-many-rework branch November 21, 2023 10:13
Copy link

github-actions bot commented Dec 6, 2023

This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Dec 6, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants