NPI-4067 Improvements to SP3 validation and unit tests #93
Conversation
…ct mode and adding optional overrides for some sections
…eck-sp3-column-alignment
…much practical difference they make yet
…F, as not doing so would cause crashes
… compariing mutual properties. Filename property extraction now returns empty not None by default, for consistency.
…offending lines of any given file, and to follow strict_mode setting
…rt as this is now off by default
…on when to raise vs warn
…for new orphan property output option
|
Previous test issue fixed.
|
ronaldmaj
left a comment
ronaldmaj
left a comment
There was a problem hiding this comment.
A few minor edits suggested here and a couple comments. I think it's basically good enough to go in as-is, but if you wanted to address the edits, I can re-approve again.
| import io as _io | ||
| import os as _os | ||
| import re as _re | ||
| from typing import Callable, Literal, Mapping, Optional, Union, List, Tuple, overload |
There was a problem hiding this comment.
Do we still need List or Tuple to be imported?
I guess for compatibility right?
There was a problem hiding this comment.
Good catch. These were deprecated in Python3.9, so we can clean them up.
https://docs.python.org/3/library/typing.html#typing.List
This PEP gives broader info: https://peps.python.org/pep-0585/
There was a problem hiding this comment.
There are a lot of these across the codebase, so I'll put them in a separate PR to keep things tidy.
New PR here: #97
gnssanalysis/gn_io/sp3.py
Outdated
|
|
||
| # Columns in SP3 we expect (by SP3 d spec) to be unused (contain a space). | ||
| # Deviation from this can be used to detect column misalignment | ||
| _SP3_UNUSED_COLUMN_INDEXES_EPOCH_HEADER: List[int] = [3, 8, 11, 14, 17, 20] |
There was a problem hiding this comment.
The List[int] could be list[int] but I assume for back compatibility we still need the one from Typing right?
There was a problem hiding this comment.
Not needed since Python3.9 (thanks to PEP 585). Will be addressed in a subsequent PR.
There was a problem hiding this comment.
Fixes for new code implemented. Fixes for the rest of the code base are in #97 (to be merged after this PR goes in)
…de will come in a separate PR.
…ompare_clk which is now deprecated, and its replacement diff_clk
This PR improves content validity / consistency tests in SP3 reading, tidies up unit tests, and addresses some Pandas deprecation and performance warnings.
strict_modelogic and structure of options, including optional overrides for specific use cases (e.g. comments, duplicate epochs, content vs header discrepancy).assertRaises()ContextManager).strict_modefor things a test is not trying to actively check..view()->.astype()