Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Inaccurate means for float16 TIFFs #176

Merged
merged 2 commits into from
Jun 16, 2021

Conversation

scottclowe
Copy link
Member

When taking the mean of an array of ints, numpy uses float64 for the output.

But when taking the mean of an array of floats, numpy uses the same float precision as the original array.

That leads to the mean being noticably less accurate when taking the mean of TIFFs that are float16 (single precision). We fix this by using float64 for all the means.

This issue wasn't caught by the unit tests before because they were also doing the less accurate rounding to determine the target values.

@scottclowe scottclowe added the bug label Jun 13, 2021
@codecov
Copy link

codecov bot commented Jun 13, 2021

Codecov Report

Merging #176 (e33a06e) into master (c422444) will not change coverage.
The diff coverage is 100.00%.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #176   +/-   ##
=======================================
  Coverage   92.65%   92.65%           
=======================================
  Files           8        8           
  Lines         831      831           
  Branches      165      165           
=======================================
  Hits          770      770           
  Misses         31       31           
  Partials       30       30           
Flag Coverage Δ
unittests 92.53% <100.00%> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
fissa/extraction.py 95.87% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c422444...e33a06e. Read the comment docs.

@scottclowe scottclowe merged commit fd9d282 into rochefort-lab:master Jun 16, 2021
@scottclowe scottclowe deleted the bug_mean-float16-tiff branch June 16, 2021 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant