Added ignore kwarg to traverse() in common, allowing directory pruning

[Cobertos]

This adds an optional kwarg to traverse() in common.py, allowing directories that are ignored to be pruned by fdfind and find, instead of iterating over and excluding later.

There's also two tests to make sure that this is working with both fdfind and find. Passes on my machine

[Cobertos]

Hmm, I forgot about the Windows implementation

[karlicoss]

Thanks, ill review later when i get to my computer. I think there are only a few Windows users, so unless you have easy access to windows computer, no need to worry for now! (as long as it doesn't crash)

…

On Sun, Feb 28, 2021, 00:34 Cobertos ***@***.***> wrote: Hmm, I forgot about the Windows implementation — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#209 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACHEBKDGBX2R7GUO74CCQTTBGFSTANCNFSM4YKOUCVA> .

[Cobertos]

Oh okay, I already fixed x3

[Cobertos]

Oh, I also forgot to mention that this requires more_itertools for the disperse function. I did not add the dependency, let me know the best way to handle that, considering all the different sort ways there are to install.

[karlicoss]

Ah, it's already a dependency:

promnesia/setup.py

Line 34 in 20a07c2

'more_itertools',

so no problem with that. more_itertools is valuable enough to have as dependency (plus it's one of the most common dependencies anyway, so usually already installed)

[karlicoss]

Sorry, need to figure out how to run github CI against other's people's PRs -- at the moment it's only triggered by my own pushes only, so I need to checkout PR and push it to trigger.

---

Looks like it's failing ignore_find test (CI=true tox -e tests -- -k ignore_find)

E         Full diff:
E           [
E         +  PosixPath('/adhoc-project/tests/testdata/traverse/imhere.txt'),
E            PosixPath('/adhoc-project/tests/testdata/traverse/imhere2/real.txt'),
E         -  PosixPath('/adhoc-project/tests/testdata/traverse/imhere.txt'),
E           ]

guess find doesn't guarantee order.

---

mypy also is going to complain CI=true tox -e mypy-core:

src/promnesia/common.py: note: In function "find_args":
src/promnesia/common.py:401: error: Argument 1 to "chain" has incompatible type "*Iterator[List[str]]"; expected
"Iterable[List[str]]"  [arg-type]
            ignore_names = list(itertools.chain(*intersperse(['-o'], ignore_names)))
                                                 ^
src/promnesia/common.py:411: error: List item 2 has incompatible type "List[Sequence[str]]"; expected "str"  [list-item]
            *prune_dir_args,
             ^
src/promnesia/common.py:413: error: List item 5 has incompatible type "List[Sequence[str]]"; expected "str"  [list-item]
            *ignore_file_args
             ^
src/promnesia/common.py: note: In function "fdfind_args":
src/promnesia/common.py:425: error: Argument 1 to "chain" has incompatible type "*List[List[str]]"; expected
"Iterable[List[str]]"  [arg-type]
            ignore_args = list(itertools.chain(*ignore_args))
                                                ^
src/promnesia/common.py:429: error: List item 1 has incompatible type "List[List[str]]"; expected "str"  [list-item]
            *ignore_args,
             ^

If you allow edits from maintainers, I don't mind quickly fixing it.

[Cobertos]

Hmm, it should allow maintainers to edit. The checkbox was checked? Oh yeah, the order, I usually use pythons unittest asserts. Probably just needs a Set comparison What is the Python typing stuff, how do I run it? Never seen that before.

…

On Sun, Feb 28, 2021, 9:01 AM karlicoss ***@***.***> wrote: Sorry, need to figure out how to run github CI against other's people's PRs -- at the moment it's only triggered by my own pushes only, so I need to checkout PR and push it to trigger. ------------------------------ Looks like it's failing ignore_find test (CI=true tox -e tests -- -k ignore_find) E Full diff: E [ E + PosixPath('/adhoc-project/tests/testdata/traverse/imhere.txt'), E PosixPath('/adhoc-project/tests/testdata/traverse/imhere2/real.txt'), E - PosixPath('/adhoc-project/tests/testdata/traverse/imhere.txt'), E ] guess find doesn't guarantee order. ------------------------------ mypy also is going to complain CI=true tox -e mypy-core: src/promnesia/common.py: note: In function "find_args": src/promnesia/common.py:401: error: Argument 1 to "chain" has incompatible type "*Iterator[List[str]]"; expected "Iterable[List[str]]" [arg-type] ignore_names = list(itertools.chain(*intersperse(['-o'], ignore_names))) ^ src/promnesia/common.py:411: error: List item 2 has incompatible type "List[Sequence[str]]"; expected "str" [list-item] *prune_dir_args, ^ src/promnesia/common.py:413: error: List item 5 has incompatible type "List[Sequence[str]]"; expected "str" [list-item] *ignore_file_args ^ src/promnesia/common.py: note: In function "fdfind_args": src/promnesia/common.py:425: error: Argument 1 to "chain" has incompatible type "*List[List[str]]"; expected "Iterable[List[str]]" [arg-type] ignore_args = list(itertools.chain(*ignore_args)) ^ src/promnesia/common.py:429: error: List item 1 has incompatible type "List[List[str]]"; expected "str" [list-item] *ignore_args, ^ If you allow edits from maintainers, I don't mind quickly fixing it. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#209 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABTSGCH52YDLZFDKRRR6JVTTBJEERANCNFSM4YKOUCVA> .

[karlicoss]

Ah, indeed, pushing onto the branch works, not sure why it didn't first time I tried!

Python typing stuff is mypy. In principle, possible to run it as mypy -p promnesia, but due to third party modules and other complexities, the most reliable is to do it via tox (e.g. CI=true tox -e mypy-core)

promnesia/tox.ini

Lines 32 to 39 in 708d073

	[testenv:mypy-core]
	commands =
	pip install -e .[linting]
	python -m mypy -p promnesia --exclude 'sources/*' \
	# txt report is a bit more convenient to view on CI
	--txt-report .coverage.mypy-core \
	--html-report .coverage.mypy-core \
	{posargs}

Either way, it was nothing major, just mypy being a bit picky about reusing the name with a different type, I quickly fixed it.

[Cobertos]

Oh I see, I haven't used tox at all. I will have to look into how that's all set. Looks like you got the PR CI stuff working too.

[karlicoss]

Thanks for the contribution!