Add configuration in pyproject.toml

[rth]

A number of things could be done with different levels of reliability when bundling applications and it becomes cumbersome to configure them all via CLI arguments. This adds support for a config file for Pyodide pack. It will look for a pyproject.toml file in the current dir or parent folders, and load the [tool.pyodide_pack] section. Somewhat inspired by what briefcase does

The full config currently (with default options) is as follows,

[tool.pyodide_pack]
requires = []  # list of dependencies to load
include_paths =  []

[tool.pyodide_pack.py]
strip_module_docstrings = true 
strip_docstrings = true
py_compile = false  # currently not implemented for `pyodide pack`
 
[tool.pyodide_pack.so]
drop_unused_so = true   # currently always enabled

This goes a bit more in the direction of pyodide pack being a pipeline of transformations and allows to combine for instance removing unused modules with AST transformations of the source code (and stripping docstrings by default).

For instance when taking the sympy example,

pyodide pack examples/sympy/app.py

before we had,

Total output size (stdlib + packages): 3.59 MB (45.1% reduction)

with this PR (and stripping docstrings) we have,

Total output size (stdlib + packages): 1.93 MB (70.6% reduction)

This also includes an unrelated bug fix for stripping docstring from a function of a form

def f():
    """Some docstring"""

which previously resulted in an invalid syntax and now produces,

def f():
    pass

TODO:

• fix ci
• check test coverage
• document this

There are still a few rough edges (e.g. py-compile support in pyodide pack) I'll address them in follow up PRs.

[rth]

@ryanking13 @hoodmane if you have any thoughts on this let me know, in particular as to how we configure CLI tools in the pyodide ecosystem. Though maybe no other tool currently needs a user modifiable configuration, for instance in pyodide build it's hardcoded on purpose depending on the pyodide version.

[ryanking13]

Cool! I think it is a good idea to add Pyodide-related configs to pyproject.toml. As you mentioned, there are currently no special settings for the other tools (pyodide-lock, pyodide-build), but I think that could be added in the future.

About the config hierarchy, which one do you think is better? [tool.pyodide.pack] or [tool.pyodide_pack]? We are already using [tool.pyodide] in pyodide-build, but I guess pyodide-build and pyodide-pack are completely different things, so I'm not sure it makes sense to group them under the same level.

[rth]

About the config hierarchy, which one do you think is better? [tool.pyodide.pack] or [tool.pyodide_pack]?

I was also wondering about it, but in the end, went for [tool.pyodide_pack] as doing e.g. [tool.pyodide.pack.py] seemed a bit too nested to me, and as you are saying probably better to separate them in general. Otherwise, technically pyodide should have document config options in child config nodes which won't be practical in this case.