Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-32248 - Implement importlib.resources #4911

Merged
merged 18 commits into from
Dec 30, 2017
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
162 changes: 162 additions & 0 deletions Doc/library/importlib.rst
Original file line number Diff line number Diff line change
Expand Up @@ -775,6 +775,168 @@ ABC hierarchy::
itself does not end in ``__init__``.


:mod:`importlib.resources` -- Resources
---------------------------------------

.. module:: importlib.resources
:synopsis: Package resource reading, opening, and access
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Block contents in reST directives should start with three spaces, not four.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we have some inconsistency in this file. I just copied it from every other section in that file except the top level .. module:: importlib.


**Source code:** :source:`Lib/importlib/resources.py`

--------------

.. versionadded:: 3.7

This module leverages Python's import system to provide access to *resources*
within *packages*. If you can import a package, you can access resources
within that package. Resources can be opened or read, in either binary or
text mode.

Resources are roughly akin to files inside directories, though it's important
to keep in mind that this is just a metaphor. Resources and packages **do
not** have to exist as physical files and directories on the file system.

Loaders can support resources by implementing the :class:`ResourceReader`
abstract base class.

The following types are defined.

.. class:: Package
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks misleading to describe it as a class, no?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any suggestions on what directive to use?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know. I think it's the first time a stdlib module exposes typing declarations. I'm not sure it's a good idea (see current python-dev discussion).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, typing defines types as classes, so with the lack of a better alternative, I think it's fine.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes... technically it's correct. What I mean is that a class is supposed to specify concrete behaviour, but being a type, Package doesn't have any methods or attributes of its own. So I don't see the point of mentioning it in the docs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It describes the expected interfaces for arguments in the API, so it seems valuable to me. We'd still have to describe that somewhere and I don't want to have to repeat it for every method.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah... It might be worth discussing on python-dev to see what a recommended practice can look like.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ivan said that Guido prefers data:: and that makes more sense in the face of PEP 560/563. So I've made that switch in my branch for consistency. I do still think it makes sense to document them because user code may want to use those types in their own annotations.


``Package`` types are defined as ``Union[str, ModuleType]``. This means
that where the function describes accepting a ``Package``, you can pass in
either a string or a module. Module objects must have a resolvable
``__spec__.submodule_search_locations`` that is not ``None``.

.. class:: Resource

This type describes the resource names passed into the various functions
in this package. This is defined as ``Union[str, os.PathLike]``.


The following functions are available.

.. function:: open_binary(package, resource)

Open for binary reading the *resource* within *package*.

:param package: A package name or module object. See above for the API
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This kind of markup seems unusual in sdtlib documentation.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unusual, so I vote to drop it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even I do not like this markup style in my own code and drop it.

Copy link
Member Author

@warsaw warsaw Dec 27, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was copied from the standalone version, where it makes more sense and doesn't have the same consistency problems. I am rewriting it without the markup for this branch.

that such module objects must support.
:type package: ``Package``
:param resource: The name of the resource to open within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type resource: ``Resource``
:returns: a binary I/O stream open for reading.
:rtype: ``typing.io.BinaryIO``


.. function:: open_text(package, resource, encoding='utf-8', errors='strict')

Open for text reading the *resource* within *package*. By default, the
resource is opened for reading as UTF-8.

:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:param resource: The name of the resource to open within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type resource: ``Resource``
:param encoding: The encoding to open the resource in. *encoding* has
the same meaning as with :func:`open`.
:type encoding: str
:param errors: This parameter has the same meaning as with :func:`open`.
:type errors: str
:returns: an I/O stream open for reading.
:rtype: ``typing.TextIO``

.. function:: read_binary(package, resource)

Read and return the contents of the *resource* within *package* as
``bytes``.

:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:param resource: The name of the resource to read within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type resource: ``Resource``
:returns: the contents of the resource.
:rtype: ``bytes``

.. function:: read_text(package, resource, encoding='utf-8', errors='strict')

Read and return the contents of *resource* within *package* as a ``str``.
By default, the contents are read as strict UTF-8.

:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:param resource: The name of the resource to read within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type resource: ``Resource``
:param encoding: The encoding to read the contents of the resource in.
*encoding* has the same meaning as with :func:`open`.
:type encoding: str
:param errors: This parameter has the same meaning as with :func:`open`.
:type errors: str
:returns: the contents of the resource.
:rtype: ``str``

.. function:: path(package, resource)

Return the path to the *resource* as an actual file system path. This
function returns a context manager for use in a :keyword:`with` statement.
The context manager provides a :class:`pathlib.Path` object.

Exiting the context manager cleans up any temporary file created when the
resource needs to be extracted from e.g. a zip file.

:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:param resource: The name of the resource to read within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type resource: ``Resource``
:returns: A context manager for use in a :keyword`with` statement.
Copy link
Contributor

@AraHaan AraHaan Dec 18, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this line read :returns: A context manager for use in an :keyword:`with` statement.?

Entering the context manager provides a :class:`pathlib.Path`
object.
:rtype: context manager providing a :class:`pathlib.Path` object


.. function:: is_resource(package, name)

Return ``True`` if there is a resource named *name* in the package,
otherwise ``False``. Remember that directories are *not* resources!

:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:param name: The name of the resource to read within *package*.
*resource* may not contain path separators and it may
not have sub-resources (i.e. it cannot be a directory).
:type name: ``str``
:returns: A flag indicating whether the resource exists or not.
:rtype: ``bool``


.. function:: contents(package)

Return an iterator over the contents of the package. The iterator can
return resources (e.g. files) and non-resources (e.g. directories). The
iterator does not recurse into subdirectories.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is one supposed to do with directories?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It depends on how much you want to ignore the API. Basically this is an escape hatch for users if they know they are on a file system and so can actually use the subdirectory results. For others they should filter them out.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think what's a little confusing may be that contents() returns str names (i.e. entries) so you can do with them whatever you want. Anyway, if that's what's confusing here, I'm rewriting this section a bit to make that clearer.


:param package: A package name or module object. See above for the API
that such module objects must support.
:type package: ``Package``
:returns: The contents of the package, both resources and non-resources.
:rtype: An iterator over ``str``


:mod:`importlib.machinery` -- Importers and path hooks
------------------------------------------------------

Expand Down
9 changes: 8 additions & 1 deletion Doc/whatsnew/3.7.rst
Original file line number Diff line number Diff line change
Expand Up @@ -282,7 +282,14 @@ Other Language Changes
New Modules
===========

* None yet.
importlib.resource
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

importlib.resources*

------------------

This module provides several new APIs and one new ABC for access to, opening,
and reading *resources* inside packages. Resources are roughly akin to files
inside of packages, but they needn't be actual files on the physical file
system. Module loaders can implement the
:class:`importlib.abc.ResourceReader` ABC to support this new module's API.


Improved Modules
Expand Down
Loading