Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

To Rearrange-Reader branch: Fix 5 tests : calculating number of data columns #369

Conversation

dcslagel
Copy link
Collaborator

@dcslagel dcslagel commented Jul 6, 2020

This pull-request fixes the remaining previously broken tests on the rearrange-reader branch.

It also, removes python 2.7 from Travis-CI. Per #364, Python 2.7 won't be supported on the rearrange-reader branch.

This change adds parsing a data line with regexp_subs which fixes common errors in the data
(~A). It is added to reader.py::inspect_data_section(). inspect_data_section() is a new function
on the rearrange_reader branch that replaces the older "get data line length" steps.
This change is essentially pulling the regex_subs loop from the older steps and adding it to the
new inspect_data_section() function.

This change also accepts the n_columns from insect_data_section() if provisional_wrap is "NO".

API change:
This change adds regex_subs to inspect_data_section's parameter list:
from
inspect_data_section(file_obj, line_nos)
to
inspect_data_section(file_obj, line_nos, regexp_subs)

Here are the tests fixed with this change:

tests/test_null_policy.py::test_null_policy_runon_replaced_1
tests/test_null_policy.py::test_null_policy_runon_replaced_2
tests/test_null_policy.py::test_null_policy_runon_ok_1
tests/test_null_policy.py::test_null_policy_runon_ok_2
tests/test_write.py::test_write_sect_widths_12

Additional-Notes:

def inspect_data_section(file_obj, line_nos, regexp_subs) and def read_data_section_iterative(file_obj, line_nos, regexp_subs, value_null_subs) have a fair amount of code in common. A future issue/pull-request might work to
combine the common code.

Test and Coverage comparison with master branch

rearrange-reader + this change

204 passed, 1 skipped, 8 warnings

Name                   Stmts   Miss  Cover
------------------------------------------
lasio/__init__.py         13      2    85%
lasio/defaults.py         11      0   100%
lasio/examples.py         42     10    76%
lasio/excel.py            88     34    61%
lasio/exceptions.py        6      0   100%
lasio/las.py             364     54    85%
lasio/las_items.py       190     39    79%
lasio/las_version.py      34     22    35%
lasio/reader.py          420     91    78%
lasio/writer.py          160      9    94%
------------------------------------------
TOTAL                   1328    261    80%

master branch

221 passed, 2 skipped, 5 warning

Name                       Stmts   Miss  Cover
----------------------------------------------
lasio/__init__.py             13      2    85%
lasio/convert_version.py      20     20     0%
lasio/defaults.py             11      0   100%
lasio/examples.py             42     10    76%
lasio/excel.py                88     34    61%
lasio/exceptions.py            6      0   100%
lasio/las.py                 388     46    88%
lasio/las_items.py           190     31    84%
lasio/las_version.py          34     22    35%
lasio/reader.py              365     36    90%
lasio/writer.py              160      9    94%
----------------------------------------------
TOTAL                       1317    210    84%

Let me know if this change could be accepted (or rejected) or
needs some additional changes before being approved and merged.

Thank you,
DC

Edit note 2020-07-08: update coverage stats to remove my development files. This moves both total coverages from percents in the 60s to percents in the 80s.

Parsing a data line with regexp_subs fixes common errors in the data
(~A) section. Adding the regexp_subs to inspect_data_section() fixes
the following tests by standardizing the line data resulting in
consistent date array lengths:
- test_null_policy_runon_replaced_1
- test_null_policy_runon_replaced_2
- test_null_policy_runon_ok_1
- test_null_policy_runon_ok_2
- test_write_sect_widths_12
@dcslagel dcslagel changed the title Add regexp_sub parsing to inspect_data_section() Merge to Rearrange-Reader branch: Add regexp_sub parsing to inspect_data_section() Jul 6, 2020
@dcslagel dcslagel changed the title Merge to Rearrange-Reader branch: Add regexp_sub parsing to inspect_data_section() To Rearrange-Reader branch: Add regexp_sub parsing to inspect_data_section() Jul 6, 2020
@dcslagel dcslagel changed the title To Rearrange-Reader branch: Add regexp_sub parsing to inspect_data_section() To Rearrange-Reader branch: Fix 5 test_null_policy tests Jul 6, 2020
@dcslagel dcslagel changed the title To Rearrange-Reader branch: Fix 5 test_null_policy tests To Rearrange-Reader branch: Fix 5 tests : calculating number of data columns Jul 6, 2020
Copy link
Owner

@kinverarity1 kinverarity1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thank you!

lasio/reader.py Show resolved Hide resolved
@kinverarity1 kinverarity1 merged commit 9f16eba into kinverarity1:rearrange-reader Jul 7, 2020
@dcslagel
Copy link
Collaborator Author

dcslagel commented Jul 8, 2020

Thank you for the review and merge! :-)

@dcslagel dcslagel deleted the rr-fix-null-policy-2-tests branch September 8, 2020 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants