Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gh-101688: Implement types.get_original_bases #101827

Merged
merged 45 commits into from
Apr 23, 2023
Merged
Show file tree
Hide file tree
Changes from 32 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
50707ba
Implement typing.get_orig_class and get_orig_bases
Gobot1234 Feb 11, 2023
4fef9f8
📜🤖 Added by blurb_it.
blurb-it[bot] Feb 11, 2023
4cb193d
Apparently my lsp wasn't working
Gobot1234 Feb 11, 2023
44d3ae9
Removing trailing whitespace
Gobot1234 Feb 11, 2023
59e9ac2
Fix test failure
Gobot1234 Feb 11, 2023
20024e9
Merge branch 'main' into orig_class-and-bases
Gobot1234 Feb 11, 2023
769fdbd
Fix typo
Gobot1234 Feb 11, 2023
7194d3e
Respond to initial review
Gobot1234 Feb 12, 2023
eeb4475
Remove trailing ws
Gobot1234 Feb 17, 2023
198dc08
Respond to second round of review
Gobot1234 Feb 25, 2023
695cf93
Fix typo
Gobot1234 Feb 25, 2023
cc11033
Fix last few comments
Gobot1234 Apr 5, 2023
a66282b
Merge remote-tracking branch 'upstream/main' into orig_class-and-bases
Gobot1234 Apr 5, 2023
d3768ba
Merge branch 'main' into orig_class-and-bases
Gobot1234 Apr 5, 2023
581a91a
Apply suggestions from code review
Gobot1234 Apr 6, 2023
fb15427
Remove typing.get_orig_class
Gobot1234 Apr 6, 2023
96b9536
Merge remote-tracking branch 'origin/orig_class-and-bases' into orig_…
Gobot1234 Apr 6, 2023
6c38e4d
Remove old test
Gobot1234 Apr 6, 2023
372bd6b
Rename to get_original_bases
Gobot1234 Apr 6, 2023
a6fe8ac
Autoformatting is fun
Gobot1234 Apr 6, 2023
2b71b74
Fix more suggestions
Gobot1234 Apr 6, 2023
e7635c6
Remove rawstring
Gobot1234 Apr 6, 2023
42e1668
Apply suggestions from code review
Gobot1234 Apr 7, 2023
1040478
Fix `test_types` failures
AlexWaygood Apr 8, 2023
8085258
Apply suggestions from code review
Gobot1234 Apr 8, 2023
b9bf4fd
Add a whatsnew entry
Gobot1234 Apr 8, 2023
ee59cbd
Update Lib/types.py
Gobot1234 Apr 8, 2023
e5a91a5
Update Doc/library/types.rst
Gobot1234 Apr 8, 2023
7bad429
Merge branch 'main' into orig_class-and-bases
AlexWaygood Apr 8, 2023
2a39055
Merge branch 'main' into orig_class-and-bases
Gobot1234 Apr 9, 2023
1ea82be
Remove trailing WS
Gobot1234 Apr 9, 2023
1cb14c2
Fix more trailing WS
Gobot1234 Apr 9, 2023
6f06c52
Fallback to __bases__
Gobot1234 Apr 9, 2023
689267a
Update missing docs
Gobot1234 Apr 9, 2023
1ae16ea
Apply suggestions from code review
Gobot1234 Apr 9, 2023
3a8619a
Update Doc/library/types.rst
Gobot1234 Apr 9, 2023
e1d55d6
Merge remote-tracking branch 'upstream/main' into orig_class-and-bases
AlexWaygood Apr 11, 2023
bab8cb3
Small tweaks to docs
AlexWaygood Apr 11, 2023
fb9ef70
Use `assert` in docs examples
AlexWaygood Apr 12, 2023
dc433c4
Document type.__orig_bases__
Gobot1234 Apr 12, 2023
4268b74
Revert "Document type.__orig_bases__"
Gobot1234 Apr 12, 2023
9f54ac1
Update Doc/library/types.rst
Gobot1234 Apr 19, 2023
c122b23
Hone docs more
AlexWaygood Apr 22, 2023
2134053
Merge remote-tracking branch 'upstream/main' into orig_class-and-bases
AlexWaygood Apr 23, 2023
cb21a65
More tests and docs following #103698
AlexWaygood Apr 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 28 additions & 0 deletions Doc/library/types.rst
Original file line number Diff line number Diff line change
Expand Up @@ -86,6 +86,34 @@ Dynamic Type Creation

:pep:`560` - Core support for typing module and generic types

.. function:: get_original_bases(cls, /)

Return the objects in the bases list of the class's definition as they
existed prior to any modification by :meth:`~object.__mro_entries__`. This
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does "modification by __mro_entries__" refer to? In my mind that is a read-only function that doesn't modify anything.

But I'm probably missing another subtle distinction. :-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't modify anything itself I guess, but it does cause the __bases__ to be changed when creating the type.

Suggested change
existed prior to any modification by :meth:`~object.__mro_entries__`. This
existed before :meth:`~object.__mro_entries__` caused them to change. This

Does this reflect the nature of the internals here better do you think?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't follow. Can you point me to the code where __mro_entries__ modifies something?

FWIW In general it's better not to describe things in terms of "internals", but instead refer to the specification of an object or operation in the documentation. (And once something is documented it's no longer internal, even if it has a dunder name.)

Copy link
Member

@AlexWaygood AlexWaygood Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider the following class:

from typing import Generic, TypeVar

T = TypeVar("T")

class Foo(Generic[T]): ...

Naively, from looking at the code, you might expect the __bases__ attribute of Foo to be equal to (Generic[T],), if you're not familiar with how typing generics and __mro_entries__ work. But it's not:

>>> from typing import TypeVar, Generic
>>> T = TypeVar("T")
>>> class Foo(Generic[T]): ...
...
>>> Foo.__bases__
(<class 'typing.Generic'>,)
>>> Foo.__bases__[0] == Generic[T]
False

This is because Python looks at Generic[T] (an instance of typing._GenericAlias), sees it's not an instance of type, looks for an __mro_entries__ method on Generic[T], finds that it does have an __mro_entries__ method, and calls Generic[T].__mro_entries__((Generic[T],)) to determine what the Foo class's "real" bases should be:

>>> Generic[T].__mro_entries__((Generic[T],))
(<class 'typing.Generic'>,)

So whereas the Foo class's "original bases" are (Generic[T],), the class's "actual bases, after __mro_entries__ has been called on all bases that aren't instances of type and have __mro_entries__ methods on them" are (Generic,).

That's the information we're trying to get across here. The trouble is getting it across in a way that's both concise and clear!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I have to admit I was again fooled by misreading. :-( I might as well come clear, I didn't remember the ugly PEP 560 mechanism and thought you were referencing __mro__. :-(

Maybe we can work a PEP 560 reference in the text, e.g. "Return the tuple of base classes originally passed to class creation, before they were replaced the PEP 560 mechanism using __mro_entries__."

I'm really sorry for being so dense.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you did ask me for a review. :-) PEP 560 is one of the most obscure mechanisms in all of typing, so deserves a mention whenever it is used. Also, perhaps a mention that this essentially just returns __orig_bases__, if it exists, might be helpful.

Copy link
Member

@AlexWaygood AlexWaygood Apr 12, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, let's add the PEP-560 link then. Though I don't know how helpful mentioning __orig_bases__ would be for most users, since the attribute is currently completely undocumented AFAIK (other than an offhand reference in PEP-560, shortly above a paragraph that says that the attribute should remain an implementation detail of CPython that shouldn't be used outside the stdlib)?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, you did ask me for a review. :-)

And I very much value your feedback — this is a tough function to describe!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Though I don't know how helpful mentioning __orig_bases__ would be for most users, since the attribute is currently completely undocumented AFAIK

But the use cases for the new addition are all people who are trying to use __orig_bases__! They should be able to find this through searching.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hah, fair point :-)

is useful for introspecting :ref:`Generics <user-defined-generics>`.
Gobot1234 marked this conversation as resolved.
Show resolved Hide resolved

Examples::

from typing import TypeVar, Generic

T = TypeVar("T")
class Foo(Generic[T]): ...
class Bar(Foo[int], float): ...
class Baz(list[str]): ...

Foo.__bases__ == (Generic,)
get_original_bases(Foo) == (Generic[T],)

Bar.__bases__ == (Foo, float)
AlexWaygood marked this conversation as resolved.
Show resolved Hide resolved
get_original_bases(Bar) == (Foo[int], float)

Baz.__bases__ == (list,)
get_original_bases(Baz) == (list[str],)

get_original_bases(int) == None

.. versionadded:: 3.12


Standard Interpreter Types
--------------------------
Expand Down
7 changes: 7 additions & 0 deletions Doc/whatsnew/3.12.rst
Original file line number Diff line number Diff line change
Expand Up @@ -388,6 +388,13 @@ threading
profiling functions in all running threads in addition to the calling one.
(Contributed by Pablo Galindo in :gh:`93503`.)

types
-----

* Add :func:`types.get_original_bases` to allow for further introspection of
:ref:`user-defined-generics` when subclassed. (Contributed by
James Hilton-Balfe and Alex Waygood in :gh:`101827`.)

unicodedata
-----------

Expand Down
23 changes: 23 additions & 0 deletions Lib/test/test_types.py
Original file line number Diff line number Diff line change
Expand Up @@ -1360,6 +1360,29 @@ class C: pass
D = types.new_class('D', (A(), C, B()), {})
self.assertEqual(D.__bases__, (A1, A2, A3, C, B1, B2))

def test_get_original_bases(self):
T = typing.TypeVar('T')
class A: pass
class B(typing.Generic[T]): pass
class C(B[int]): pass
class D(B[str], float): pass
self.assertIsNone(types.get_original_bases(A))
self.assertEqual(types.get_original_bases(B), (typing.Generic[T],))
self.assertEqual(types.get_original_bases(C), (B[int],))
self.assertIsNone(types.get_original_bases(int))
self.assertEqual(types.get_original_bases(D), (B[str], float))

class E(list[T]): pass
class F(list[int]): pass

self.assertEqual(types.get_original_bases(E), (list[T],))
self.assertEqual(types.get_original_bases(F), (list[int],))

Gobot1234 marked this conversation as resolved.
Show resolved Hide resolved
class G(typing.NamedTuple):
x: int

self.assertIs(types.get_original_bases(G)[0], typing.NamedTuple)

# Many of the following tests are derived from test_descr.py
def test_prepare_class(self):
# Basic test of metaclass derivation
Expand Down
28 changes: 28 additions & 0 deletions Lib/types.py
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,34 @@ def _calculate_meta(meta, bases):
"of the metaclasses of all its bases")
return winner


def get_original_bases(cls, /):
"""Return the class's "original" bases prior to modification by `__mro_entries__`.

This is useful for introspecting Generics.

Gobot1234 marked this conversation as resolved.
Show resolved Hide resolved
Examples::

from typing import TypeVar, Generic

T = TypeVar("T")
class Foo(Generic[T]): ...
class Bar(Foo[int], float): ...
class Baz(list[str]): ...

get_original_bases(Foo) == (Generic[T],)
get_original_bases(Bar) == (Foo[int], float)
get_original_bases(Baz) == (list[str],)
get_original_bases(int) == None
"""
if isinstance(cls, type):
try:
return cls.__orig_bases__
except AttributeError:
pass
return None
Gobot1234 marked this conversation as resolved.
Show resolved Hide resolved
Gobot1234 marked this conversation as resolved.
Show resolved Hide resolved


class DynamicClassAttribute:
"""Route attribute access on a class to __getattr__.

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Implement :func:`types.get_original_bases` to provide further introspection
for types.