Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEV - Refactor CI and environment setup #1759

Merged
merged 81 commits into from
May 24, 2024
Merged

DEV - Refactor CI and environment setup #1759

merged 81 commits into from
May 24, 2024

Conversation

trallard
Copy link
Collaborator

@trallard trallard commented Apr 5, 2024

This PR addresses some of the concerns/issues raised in #1292, more details, caveats, and TODOs below.

What is in this PR

  1. A refactored CI (currently in a tests_tox.yml file; if accepted, this should completely replace the existing tests.yml, but I thought I would leave it here in the interim so folks could compare as the CI gets executed. Some notable items:
    • There was a lot of duplication as we set Python and other things multiple times, so I created an action that performs that essential setup task and avoids duplication
    • The current CI workflow involves calling nox using bash scripts to install things, calling sphinx-build directly, etc. Here, we use tox for all the jobs and steps, providing syntax and tool consistency and removing cognitive overload for maintainers and contributors.
    • I made some style improvements to make all the workflow files more readable and easier to follow and added some nice emojis, which are great when looking at the actions CI over and over.
    • Added macos-14 to our testing matrices; even with that and other combinations of OSes, python and Sphinx versions, even with these changes this CI suite is between 3.5 and 4 minutes faster than the existing one
    • Gets rid of codecov upload, which has been a burden and instead uploads results to GH artifacts and prints a summary
  2. tox.ini, which is the base of this refactor - this would mean tox will effectively replace nox; therefore, the noxfile.py would be removed. The reasons why I am proposing this replacement are:
    • tox is faster than nox, and I have found that the should_install step in the noxfile is flaky, and I have had to nuke my env multiple times.
    • tox is also very good at handling rebuilds and keeping track of dependencies changes which is something we have been struggling with using nox
    • We need to test against multiple versions of Python and Sphinx, and tox is very flexible and designed for this kind of complex setups
    • With the current tox.ini configuration, we can install PST in development mode for tasks like testing, profiling, etc. and perform other functions by installing a packaged version of PST; this is mostly for building our docs, which more closely mimics how any other user would use PST, thus helping with testing and validation on that end too
    • We can use tox for both CI, local development, docs, compiling, lining, and all (actually, we drop the pre-commit action dependency here and do our lint)

What it does not do

  1. We still use stb and sphinx-build as they serve different purposes - sbt helps with the compilation of assets and all node-related things as well as packaging, while sphinx-build only generates our docs in HTML format. So, we are keeping these two dependencies as we need both.
  2. It Does not address figure out how to control node version in CIs #998 but makes it easier for me or someone else to extend and test against different versions of node

Still to do

  • Delete obsolete files github_actions_install.sh
  • Remove obsolete dependencies nox -> deferred to other PR
  • Update documentation as needed to replace use of nox

Deferred to other PR

To move this PR and facilitate the merge, I have decided to move this other significant amount of work to another PR.
Note that this means this PR does not get rid of nox, so until this follow-up PR, we have nox and tox, but I have started to move the documentation here to include the use of tox for development tasks and the like.

Questions

  1. We use Python 3.10 in Readthedocs (see readthedocs.yml) and Python 3.12 by default for some of our docs building tasks. Do we want to update the version in RTD for consistency?

If you made it all the way here you deserve a cookie 🍪, thank you 💜

@trallard
Copy link
Collaborator Author

trallard commented May 20, 2024

Thanks @Carreau I'd be happy to get this in. I also updated the description to better reflect the content and status of the work.

If @drammock gives a thumbs up let's merge.

Copy link
Collaborator

@drammock drammock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm on board with this. I've left a few comments/questions on things that looked odd to me; not sure if they're mistakes or just showing my lack of familiarity with tox / actions / etc.

.github/actions/set-dev-env/action.yml Outdated Show resolved Hide resolved
# oldest Python version with the oldest Sphinx version
- os: ubuntu-latest
python-version: "3.9"
sphinx-version: "61"
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bit confusing, expected 6.1. why not use the tr -d . trick on sphinx version too?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly I cannot remember why I did this but can change to match the rest

.github/workflows/CI.yml Show resolved Hide resolved
.github/workflows/CI.yml Outdated Show resolved Hide resolved
tox.ini Outdated Show resolved Hide resolved
tox.ini Show resolved Hide resolved
pyproject.toml Outdated Show resolved Hide resolved
pyproject.toml Show resolved Hide resolved
@Carreau
Copy link
Collaborator

Carreau commented May 23, 2024

Minor conflict with #1826 that added timeouts. I merge with main, but timeouts may needed to be readded to actions in the future.

@trallard
Copy link
Collaborator Author

trallard commented May 23, 2024

I have addressed all the review comments, so if CI is happy, we could merge @drammock

Edit I broke stuff so will fix it

@drammock
Copy link
Collaborator

drammock commented May 23, 2024 via email

.github/workflows/CI.yml Outdated Show resolved Hide resolved
.github/workflows/CI.yml Outdated Show resolved Hide resolved
tox.ini Outdated
# keep this in sync across all docs environments
extras = {[testenv:docs-no-checks]extras}
deps =
py39-sphinx61-docs: {py39-sphinx61-tests[deps]}
Copy link
Collaborator

@Carreau Carreau May 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
py39-sphinx61-docs: {py39-sphinx61-tests[deps]}
py39-sphinx61-docs: py39-sphinx61-tests[deps]

I think this is the issue as the error message is

/opt/hostedtoolcache/Python/3.9.19/x64/bin/uv pip install './{py39-sphinx61-tests[deps]}'
error: Failed to read from the distribution cache
  Caused by: failed to query metadata of file `/home/runner/work/pydata-sphinx-theme/pydata-sphinx-theme/{py39-sphinx61-tests[deps]}`

tox.ini Outdated Show resolved Hide resolved
@trallard
Copy link
Collaborator Author

Thanks for all your help, @Carreau 🟣 - I fixed the issue with the deps, and the CI worked well.
I just pushed a change to add the new ubuntu-24.4, so if this works, I will merge it as is. If it fails, I will revert and merge without the latest commit.

@trallard trallard merged commit 41c4cf5 into pydata:main May 24, 2024
30 checks passed
@trallard trallard deleted the trallard/update-ci branch May 24, 2024 15:41
ivanov pushed a commit to ivanov/pydata-sphinx-theme that referenced this pull request Jun 5, 2024
…#1800)

* Add env variable to control test-them build location.

I needed this to debug some things, so why not send it as a PR.

See pydata#1798, maybe this should be folded in pydata#1759 ?

* Update tests/conftest.py

Co-authored-by: Daniel McCloy <dan@mccloy.info>

---------

Co-authored-by: Daniel McCloy <dan@mccloy.info>
ivanov pushed a commit to ivanov/pydata-sphinx-theme that referenced this pull request Jun 5, 2024
This PR addresses some of the concerns/issues raised in pydata#1292
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind: enhancement New feature or request tag: CI Pull requests that update GitHub Actions code tag: dependencies Pull requests that update a dependency file
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants