Repository navigation
Typing guide: add code snippets for common patterns and more resources - #1177
Conversation
Add a 'Common patterns, with examples' section with snippets trimmed from napari's source (TYPE_CHECKING imports, X | None and narrowing, numpy.typing arrays, Protocols such as LayerDataProtocol), and extend 'Where to read more' with the typing, mypy, NumPy typing and pyrefly error-kind docs. Closes napari#1166
| needs `Dims` only to annotate a parameter, so the import sits under | ||
| `TYPE_CHECKING` and is never executed at runtime: | ||
|
|
||
| ```python | ||
| from __future__ import annotations | ||
|
|
||
| from typing import TYPE_CHECKING, Any, Protocol | ||
|
|
||
| if TYPE_CHECKING: | ||
| from napari.components import Dims |
There was a problem hiding this comment.
I feel repetition, as we have covered in (lines 51-60), so maybe we can reduce this section
| On Python versions before 3.14, without the `__future__` import, | ||
| `dims: Dims` would be evaluated when the class is defined and raise a | ||
| `NameError`, because `Dims` only exists for the checker. |
There was a problem hiding this comment.
this is already covered (line 46-50) , so i think we can get rid of this
| Write `X | None` for an optional value and `X | Y` for a union. They mean the | ||
| same as `Optional[X]` and `Union[X, Y]`, which you will still see in older | ||
| modules. Ruff's automatic rewrite (`UP007`, `UP045`) is currently disabled in | ||
| napari's `pyproject.toml`, so leave existing annotations as they are unless you | ||
| are changing that code anyway. From | ||
| [`utils/status_messages.py`](https://github.com/napari/napari/blob/main/src/napari/utils/status_messages.py): |
There was a problem hiding this comment.
how about keeping only this
Write `X | None` for an optional value and `X | Y` for a union. They mean the
same as `Optional[X]` and `Union[X, Y]`, which you will still see in older
modules while napari's ruff config disables the automatic rewrite (`UP007`,
`UP045`). From
[`utils/status_messages.py`](https://github.com/napari/napari/blob/main/src/napari/utils/status_messages.py):
| The `is None` check is the kind of narrowing that should be preferred to | ||
| `cast()`. Because napari requires Python 3.11 or newer, `int | None` also works |
There was a problem hiding this comment.
we have discussed about cast in line 85-88, so maybe we can keep in one place
| — the standard library documentation for `TYPE_CHECKING`, `Protocol`, | ||
| `cast` and the other names used above. | ||
| - [Static Typing with Python](https://typing.python.org/en/latest/) — the | ||
| checker-neutral guides and the typing specification, including a |
There was a problem hiding this comment.
we already have reference for static typing, do we need another one or keep one.
| - [mypy documentation](https://mypy.readthedocs.io/en/stable/) — its | ||
| [protocols](https://mypy.readthedocs.io/en/stable/protocols.html) and |
There was a problem hiding this comment.
i think we can drop this 🤔
| [kinds of types](https://mypy.readthedocs.io/en/stable/kinds_of_types.html) | ||
| pages explain unions, `None` and narrowing with examples. | ||
| - [NumPy typing](https://numpy.org/doc/stable/reference/typing.html) — | ||
| `npt.NDArray`, `npt.ArrayLike` and `npt.DTypeLike`. |
| `npt.NDArray`, `npt.ArrayLike` and `npt.DTypeLike`. | ||
| - [pyrefly error kinds](https://pyrefly.org/en/docs/error-kinds/) — what each | ||
| `[error-kind]` in a pyrefly message means, for writing targeted | ||
| suppressions. |
There was a problem hiding this comment.
don't think we need explicit reference for the errors, as we have reference in line 300 ( typing for developer, i think thats enough )
Aniketsy
left a comment
There was a problem hiding this comment.
@zelihaguven thanks for the work!
I've left some comments, please have a look and let me know if anything is unclear :)
|
@zelihaguven first off thanks for picking this up! I must admit I feel this is a bit overkill for a gentle introduction to typing and it would cause more confusion than clarity. The example snippets shouldn't refer to the real napari code base and should instead show small examples that focus on typing. Additionally it is not clearly stated |
This is mostly a driveby, but what are your thoughts on a |
That would probably fit better in the context of these changes and would have been the natural follow-up step, indeed. I think probably something like a "typing cheat sheet" kind of document. Still they should be generalized, not tied to the specifics of the napari internals. |
|
Thanks @jacopoabramo and @TimMonko, that makes sense, and I agree the current version is too much for an intro page. My plan for this PR:
Does that structure work for you, or would you prefer the cheat sheet as a separate follow-up PR? |
|
@zelihaguven yes this plan's outlook looks reasonable, thank you! I would encourage you to have a look at this review because it also pinpoints where to put specific mentions as additions in the documentation page. Some additional guidelines:
Just overall look at that review and address it as best as you can, including your plan. |
Keep the type checking page light and move the longer examples into a new typing cheat sheet with small, generic snippets instead of napari internals. - typing.md: add a tip on why annotations help, tabs showing code without and with annotations, and a short self-contained TYPE_CHECKING example with one sentence on why those imports never run at runtime. - typing.md: remove the "Common patterns" section and the duplicated links in "Where to read more". - typing_patterns.md: new cheat sheet covering the __future__ import (with tabs), optional values, collections.abc inputs, TypeVar generics, NumPy arrays and protocols. - Add the new page to the TOC and the developers index, and cross-link the two pages.
|
Thanks @jacopoabramo, @TimMonko and @Aniketsy! I pushed a new commit that follows the plan:
Happy to adjust anything, including splitting the cheat sheet into a follow-up PR if you'd prefer. |
| # Typing cheat sheet | ||
|
|
||
| Short, self-contained examples of the rules in [Type checking](napari-typing). | ||
| Each snippet focuses on one typing pattern rather than on napari's own code. |
|
|
||
| Short, self-contained examples of the rules in [Type checking](napari-typing). | ||
| Each snippet focuses on one typing pattern rather than on napari's own code. | ||
| For how to run the checker and how to handle its errors, see the main page. |
There was a problem hiding this comment.
Add a link directly to the main page
| Tools that read annotations at runtime, such as magicgui and Pydantic, need | ||
| extra care here, as explained in |
There was a problem hiding this comment.
Add links to both magicgui and pydantic
| DEFAULT_PRECISION = 3 | ||
|
|
||
|
|
||
| def format_float(value: float, precision: int | None = None) -> str: | ||
| if precision is None: | ||
| precision = DEFAULT_PRECISION | ||
| # from here on, the checker knows that `precision` is an `int` | ||
| return f'{value:.{precision}f}' |
There was a problem hiding this comment.
| DEFAULT_PRECISION = 3 | |
| def format_float(value: float, precision: int | None = None) -> str: | |
| if precision is None: | |
| precision = DEFAULT_PRECISION | |
| # from here on, the checker knows that `precision` is an `int` | |
| return f'{value:.{precision}f}' | |
| def format_float(value: float, precision: int | None = None) -> str: | |
| if precision is None: | |
| # since the input `precision` is `None`, set it to a default value | |
| precision = 3 | |
| # from here on, the checker knows that `precision` is an `int` | |
| return f'{value:.{precision}f}' |
| Annotate parameters with the abstract types from `collections.abc` and return | ||
| the concrete type you built: |
There was a problem hiding this comment.
| Annotate parameters with the abstract types from `collections.abc` and return | |
| the concrete type you built: | |
| `collections.abc` is a python module enstablishing abstract, reusable and pythonic-like interfaces for customizeable classes that implement common patterns. | |
| It's best to annotate the input types of a function or method through these types. Instead, output types should return clearly defined types. In general the idea is "flexible inputs, strict outputs". |
There was a problem hiding this comment.
Also add a small example of what actually one of these types look like. This is tied to the follow-up text anyway.
|
done |
| ## Protocols for "anything with these members" | ||
|
|
||
| A `Protocol` describes what a function uses instead of which class it expects: | ||
|
|
||
| ```python | ||
| from typing import Protocol | ||
|
|
||
|
|
||
| class HasShape(Protocol): | ||
| @property | ||
| def shape(self) -> tuple[int, ...]: ... | ||
|
|
||
|
|
||
| def ndim(data: HasShape) -> int: | ||
| return len(data.shape) | ||
| ``` |
There was a problem hiding this comment.
This section is incomplete, it should present what happens with read-write attributes (property is a read-only attribute) and methods
Keep the read-only HasShape example and show a writable attribute and a method on a second protocol, with a class that matches it.
TimMonko
left a comment
There was a problem hiding this comment.
Thanks @zelihaguven for continuing to work on this! I overall am quite pleased! I find myself surprised that a "generic" typing guide seems so helpful to put in napari, but because it directly addresses our patterns I think it works quite nice.
I think it's really almost there! Would you like to be added to the napari/napari Citation file? If so, please either open a pull request on napari/napari or reply here with your information
- given-names:
family-names:
affiliation: <optional>
orcid: https://orcid.org/<orcid> <optional>
alias: zelihaguvenThanks again for this great work! It brought clarity to me on some topics :)
There was a problem hiding this comment.
| concrete type you built.** When possible, take `Iterable`, `Sequence` or |
Interesting typo I saw when reading -- no idea how that got past the spell checker previously! I'll just push it here.
| - file: developers/contributing/dev_install | ||
| - file: developers/contributing/testing | ||
| - file: developers/contributing/typing | ||
| - file: developers/contributing/typing_patterns |
There was a problem hiding this comment.
| - file: developers/contributing/typing_patterns | |
| subtrees: | |
| - entries: | |
| - file: developers/contributing/typing_patterns |
I quite like how its nest in docs/developers/index.md. What do you think about also making it a child for the toctree / sidebar?
There was a problem hiding this comment.
I wasn't sure because it was so many lines here, but then reading through I quite like these minimal explanations and comparisons. The changes to this file are great!
| signature. If a function modifies its argument in place, ask for | ||
| `MutableSequence` or `MutableMapping` instead. |
There was a problem hiding this comment.
this reads as a jump in logic to me, as the example above only uses Mapping -- so my immediate thought jumps to -- are there ImmutableMapping? Is Mapping by default mutable or immutable? Indeed, what is Mapping as Iterable has been explained. Oh, and there is Sequence here now too? Why Sequence. I know the answer to some (but not all) of these questions, but as a reader it feels abrupt. You could either remove such mentions or expand on them to make it more clear. Especially because Sequence is used in the next section.
There was a problem hiding this comment.
Probably after this line it's better to add a note marker stating in a few words the difference between a mutable vs. immutable data structure.
Thing is, I don't want to make this guide a thorough walkthrough of the typing specification.
For example, Mapping is not mutable, while a MutableMapping is. Same for Sequence and MutableSequence. The difference is mainly in the supported methods.
I wouldn't know how to explain this section better without pulling a comparison between the supported methods...
Best we could do is point into the note to the collections.abc documentation. It provides exhaustive comparison of the supported methods.
| The checker now knows that `first([1, 2])` is an `int` and `first(["a"])` is a | ||
| `str`. napari still supports Python 3.11, so use `TypeVar` rather than the | ||
| `def first[T](...)` syntax added in Python 3.12. |
There was a problem hiding this comment.
cool TIL and finally understand! Thank you :)
| read-only, so assigning to it is an error. `label` is a method: the object | ||
| must define it, and the checker checks the call and the `str` return. | ||
|
|
||
| Add `@runtime_checkable` only if you also need `isinstance()` checks. |
There was a problem hiding this comment.
| Add `@runtime_checkable` only if you also need `isinstance()` checks. | |
| Decorate the `Protocol` with `@runtime_checkable` only if you also need `isinstance()` checks. |
To give clarity on where for people that need it
| sample.label() | ||
| ``` | ||
|
|
||
| The checker treats `name` as a read-write attribute, so assigning a `str` is |
There was a problem hiding this comment.
what do you mean by a read-write attribute? Is that something baked into Protocol or because of the type annotation on name?
I think I'm wondering if the HasShape and the Named block can be combined, there is much overlap in logic and by splitting like this I'm having a harder time following what is important.
There was a problem hiding this comment.
what do you mean by a
read-writeattribute? Is that something baked into Protocol or because of the type annotation onname?
Consider the following:
class MyProto(Protocol):
name: str
class MyOtherProto(Protocol):
@property
def name(self) -> str:
...
class MyClass:
def __init__(self) -> None:
self.name = "MyClass"
my_obj = MyClass()
isinstance(my_obj, MyProto) # prints True
isinstance(my_obj, MyOtherProto) # prints TrueSo yes, the distinction between read-write vs read-only attributes it's baked into the definition of a protocol. In this case the two protocols differ on the ability to write into name, and in the latter it's explicitly marked as @property which makes it read-only.
I think I'm wondering if the
HasShapeand theNamedblock can be combined, there is much overlap in logic and by splitting like this I'm having a harder time following what is important.
At a glance they can, and it would probably help to do so from a readability perspective but we could lose the explanation of the distinction above.
There was a problem hiding this comment.
excellent explanation, than you!
- Fix the "possible" typo in typing.md. - Nest the typing cheat sheet under the typing page in the sidebar. - Add a short note on Mapping vs MutableMapping and Sequence vs MutableSequence, linking the collections.abc docs. - Say where to put @runtime_checkable.
|
Thanks @jacopoabramo, that makes sense. I added a short note after the Mapping/Sequence paragraph on mutable vs. immutable ABCs, with a link to the collections.abc docs, and kept HasShape and Named separate so the read-only vs. read-write distinction stays clear. I also fixed the typo, nested the cheat sheet under the typing page in _toc.yml, and used the suggested runtime_checkable wording. Thanks @TimMonko for the review! |
for more information, see https://pre-commit.ci
@jacopoabramo hmm i think you and @TimMonko have already reviewed this in depth ❤️ soo we're good 👍 and if you guys want me to have one final look, please let me know 😉 ( actually i'm dealing with my typing PRs 😿 ) |
There was a problem hiding this comment.
Awesome work @zelihaguven. Very happy with this now! Thank you for your dedication to this PR :) I'm grateful for all the work that you've done.
Yes, @jacopoabramo, in the docs we require milestoning before merge. This is the way napari used to work as well, but I can't remember why we changed it and why not here 🤔 I think because managing the milestone is much less "informative" in napari/napari with 200+ PRs, but here its a much smaller scope. (I'll let you do it and merge -- whenever you want!)
Closes #1166. Adds a "Common patterns, with examples" section to the typing guide, with short snippets trimmed from napari's source (TYPE_CHECKING imports,
X | Noneand narrowing,numpy.typingarrays, Protocols such asLayerDataProtocol), each linked to the file it comes from.Extends "Where to read more" with the Python
typingdocs, typing.python.org, the mypy docs (protocols, kinds of types), NumPy typing and pyrefly's error kinds; all links checked.Checked locally:
rumdl check(incl. ruff format of code blocks) and codespell pass, the snippets run and passpyrefly check, and the page builds without warnings in a minimal Sphinx + MyST build (the full docs build was not run).