Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve performance of find_similar_names #173

Merged
merged 1 commit into from
Jul 19, 2022
Merged

improve performance of find_similar_names #173

merged 1 commit into from
Jul 19, 2022

Conversation

maxbachmann
Copy link
Contributor

@maxbachmann maxbachmann commented Jul 9, 2022

on most platforms this improves the performance by 30-40x by using a highly optimized C extension.
On unsupported platforms falls back to a pure Python implementation, which still improves performance by 2x.

I am not really familiar with poetry. I used:

poetry add rapidfuzz
poetry remove pylev

However this removed files for a lot of dependencies, which seems wrong. What is the correct process to replace a dependency?

@maxbachmann

This comment was marked as outdated.

@radoering
Copy link
Member

I am not really familiar with poetry. I used:

poetry add rapidfuzz
poetry remove pylev

However this removed files for a lot of dependencies, which seems wrong. What is the correct process to replace a dependency?

Seems you have experienced python-poetry/poetry#5970. Update to poetry 1.1.14, clear your cache (poetry cache clear pypi --all), restore the lock file and do the steps again (add and remove should be fine).

@maxbachmann
Copy link
Contributor Author

Seems you have experienced python-poetry/poetry#5970. Update to poetry 1.1.14, clear your cache (poetry cache clear pypi --all), restore the lock file and do the steps again (add and remove should be fine).

thanks this worked

@maxbachmann
Copy link
Contributor Author

maxbachmann commented Jul 9, 2022

Not sure whats up with the Python3.11 failures, but I don't think they are caused by the change.

@maxbachmann
Copy link
Contributor Author

restarting the CI today solved the build issues, so they were unrelated.

@maxbachmann maxbachmann marked this pull request as draft July 13, 2022 18:29
@maxbachmann maxbachmann marked this pull request as ready for review July 19, 2022 18:23
@maxbachmann
Copy link
Contributor Author

This is now ready for review.

Secrus
Secrus previously approved these changes Jul 19, 2022
Copy link
Member

@Secrus Secrus left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@maxbachmann would you mind writing a simple test for that? I see there are no tests for _utils module at the moment and this could be a good start.

@maxbachmann
Copy link
Contributor Author

Sure I can add a test

on most platforms this improves the performance by 30-40x
by using a highly optimized C extension. On unsupported platforms
falls back to a pure Python implementation, which still improves
performance by 2x.
@maxbachmann
Copy link
Contributor Author

I added a simple test for find_similar_names

@Secrus Secrus merged commit b5dc7a8 into python-poetry:master Jul 19, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants