Skip to main content

Building blocks for precise & flexible Python type hints

Project description

optype

One protocol, one method.

Building blocks for precise & flexible type hints.

Continuous Integration PyPI Python Versions License Ruff Checked with pyright


Installation

Optype is available as optype on PyPI:

pip install optype

Overview

The API of optype is flat; a single import optype is all you need. There are four flavors of things that live within optype,

  • optype.Can{} types describe what can be done with it. For instance, any CanAbs[T] type can be used as argument to the abs() builtin function with return type T. Most Can{} implement a single special method, whose name directly matched that of the type. CanAbs implements __abs__, CanAdd implements __add__, etc.
  • optype.Has{} is the analogue of Can{}, but for special attributes. HasName has the __name__: str attribute, HasDict has a __dict__, etc.
  • optype.Does{} describe the type of operators. So DoesAbs is the type of the abs({}) builtin function, and DoesPos the type of the +{} prefix operator.
  • optype.do_{} are the correctly-typed implementations of Does{}. For each do_{} there is a Does{}, and vice-versa. So do_abs: DoesAbs is the typed alias of abs({}), and do_pos: DoesPos is a typed version of operator.pos. The optype.do_ operators are more complete than operators, has runtime-accessible type annotations, and uses a fully predictable naming scheme.

Reference

All typing protocols here live in the root optype namespace. They are runtime-checkable so that you can do e.g. isinstance('snail', optype.CanAdd), in case you want to check whether snail implements __add__.

[!NOTE] It is bad practice to use a typing.Protocol as base class for your implementation. Because of @typing.runtime_checkable, you can use isinstance either way.

Unlikecollections.abc, optype's protocols aren't abstract base classes, i.e. they don't extend abc.ABC, only typing.Protocol. This allows the optype protocols to be used as building blocks for .pyi type stubs.

Type conversion

The return type of these special methods is invariant. Python will raise an error if some other (sub)type is returned. This is why these optype interfaces don't accept generic type arguments.

operator operand
expression method type
bool(_) __bool__ CanBool
int(_) __int__ CanInt
float(_) __float__ CanFloat
complex(_) __complex__ CanComplex
bytes(_) __bytes__ CanBytes
str(_) __str__ CanStr

These formatting methods are allowed to return instances that are a subtype of the str builtin. The same holds for the __format__ argument. So if you're a 10x developer that wants to hack Python's f-strings, but only if your type hints are spot-on; optype is you friend.

operator operand
expression function type method type
repr(_) do_repr DoesRepr __repr__ CanRepr[Y: str]
format(_, x) do_format(_, x) DoesFormat __format__ CanFormat[X: str, Y: str]

"Rich comparison" operators

These special methods generally a bool. However, instances of any type can be returned.

operator operand
expression reflected function type method type
_ < x x > _ do_lt DoesLt __lt__ CanLt[X, Y]
_ <= x x >= _ do_le DoesLe __le__ CanLe[X, Y]
_ == x x == _ do_eq DoesEq __eq__ CanEq[X, Y]
_ != x x != _ do_ne DoesNe __ne__ CanNe[X, Y]
_ > x x < _ do_gt DoesGt __gt__ CanGt[X, Y]
_ >= x x <= _ do_ge DoesGe __ge__ CanGe[X, Y]

Callable objects

Unlike operator, optype provides the operator for callable objects: optype.do_call(f, *args. **kwargs).

CanCall is similar to collections.abc.Callable, but is runtime-checkable, and doesn't use esoteric hacks.

operator operand
expression function type method type
_(*args, **kwargs) do_call DoesCall __call__ CanCall[**Xs, Y]

Numeric operations

For describing things that act like numbers. See the Python docs for more info.

operator operand
expression function type method type
_ + x do_add DoesAdd __add__ CanAdd[X, Y]
_ - x do_sub DoesSub __sub__ CanSub[X, Y]
_ * x do_mul DoesMul __mul__ CanMul[X, Y]
_ @ x do_matmul DoesMatmul __matmul__ CanMatmul[X, Y]
_ / x do_truediv DoesTruediv __truediv__ CanTruediv[X, Y]
_ // x do_floordiv DoesFloordiv __floordiv__ CanFloordiv[X, Y]
_ % x do_mod DoesMod __mod__ CanMod[X, Y]
divmod(_, x) do_divmod DoesDivmod __divmod__ CanDivmod[X, Y]
_ ** x
pow(_, x)
do_pow DoesPow __pow__ CanPow2[X, Y2]
CanPow[X, None, Y2, Any]
pow(_, x, m) do_pow DoesPow __pow__ CanPow3[X, M, Y3]
CanPow[X, M, Any, Y3]
_ << x do_lshift DoesLshift __lshift__ CanLshift[X, Y]
_ >> x do_rshift DoesRshift __rshift__ CanRshift[X, Y]
_ & x do_and DoesAnd __and__ CanAnd[X, Y]
_ ^ x do_xor DoesXor __xor__ CanXor[X, Y]
_ | x do_or DoesOr __or__ CanOr[X, Y]

Note that because pow() can take an optional third argument, optype provides separate interfaces for pow() with two and three arguments. Additionally, there is the overloaded intersection type CanPow[X, M, Y2, Y3] =: CanPow2[X, Y2] & CanPow3[X, M, Y3], as interface for types that can take an optional third argument.

For the binary infix operators above, optype additionally provides interfaces with reflected (swapped) operands:

operator operand
expression function type method type
x + _ do_radd DoesRAdd __radd__ CanRAdd[X, Y]
x - _ do_rsub DoesRSub __rsub__ CanRSub[X, Y]
x * _ do_rmul DoesRMul __rmul__ CanRMul[X, Y]
x @ _ do_rmatmul DoesRMatmul __rmatmul__ CanRMatmul[X, Y]
x / _ do_rtruediv DoesRTruediv __rtruediv__ CanRTruediv[X, Y]
x // _ do_rfloordiv DoesRFloordiv __rfloordiv__ CanRFloordiv[X, Y]
x % _ do_rmod DoesRMod __rmod__ CanRMod[X, Y]
divmod(x, _) do_rdivmod DoesRDivmod __rdivmod__ CanRDivmod[X, Y]
x ** _
pow(x, _)
do_rpow DoesRPow __rpow__ CanRPow[X, Y]
x << _ do_rlshift DoesRLshift __rlshift__ CanRLshift[X, Y]
x >> _ do_rrshift DoesRRshift __rrshift__ CanRRshift[X, Y]
x & _ do_rand DoesRAnd __rand__ CanRAnd[X, Y]
x ^ _ do_rxor DoesRXor __rxor__ CanRXor[X, Y]
x | _ do_ror DoesROr __ror__ CanROr[X, Y]

Note that CanRPow corresponds to CanPow2; the 3-parameter "modulo" pow does not reflect in Python.

Similarly, the augmented assignment operators are described by the following optype interfaces:

operator operand
expression function type method type
_ += x do_iadd DoesIAdd __iadd__ CanIAdd[X, Y]
_ -= x do_isub DoesISub __isub__ CanISub[X, Y]
_ *= x do_imul DoesIMul __imul__ CanIMul[X, Y]
_ @= x do_imatmul DoesIMatmul __imatmul__ CanIMatmul[X, Y]
_ /= x do_itruediv DoesITruediv __itruediv__ CanITruediv[X, Y]
_ //= x do_ifloordiv DoesIFloordiv __ifloordiv__ CanIFloordiv[X, Y]
_ %= x do_imod DoesIMod __imod__ CanIMod[X, Y]
_ **= x do_ipow DoesIPow __ipow__ CanIPow[X, Y]
_ <<= x do_ilshift DoesILshift __ilshift__ CanILshift[X, Y]
_ >>= x do_irshift DoesIRshift __irshift__ CanIRshift[X, Y]
_ &= x do_iand DoesIAnd __iand__ CanIAnd[X, Y]
_ ^= x do_ixor DoesIXor __ixor__ CanIXor[X, Y]
_ |= x do_ior DoesIOr __ior__ CanIOr[X, Y]

Additionally, there are the unary arithmetic operators:

operator operand
expression function type method type
+_ do_pos DoesPos __pos__ CanPos[Y]
-_ do_neg DoesNeg __neg__ CanNeg[Y]
~_ do_invert DoesInvert __invert__ CanInvert[Y]
abs(_) do_abs DoesAbs __abs__ CanAbs[Y]

The round() built-in function takes an optional second argument. From a typing perspective, round() has two overloads, one with 1 parameter, and one with two. For both overloads, optype provides separate operand interfaces: CanRound1[Y] and CanRound2[N, Y]. Additionally, optype also provides their (overloaded) intersection type: CanRound[N, Y1, Y2] = CanRound1[Y1] & CanRound2[N, Y2].

operator operand
expression function type method type
round(_) do_round DoesRound __round__/1 CanRound1[Y1]
CanRound[None, Y1, Any]
round(_, n) do_round DoesRound __round__/2 CanRound2[N, Y2]
CanRound[N, Any, Y2]
round(_, n=None) do_round(_, n=None) DoesRound __round__ CanRound[N, Y1, Y2]

For example, type-checkers will mark the following code as valid (tested with pyright in strict mode):

x: float = 3.14
x1: CanRound1[int] = x
x2: CanRound2[int, float] = x
x3: CanRound[int, int, float] = x

Furthermore, there are the alternative rounding functions from the math standard library:

operator operand
expression function type method type
math.trunc(_) do_trunc DoesTrunc __trunc__ CanTrunc[Y]
math.floor(_) do_floor DoesFloor __floor__ CanFloor[Y]
math.ceil(_) do_ceil DoesCeil __ceil__ CanCeil[Y]

Note that the type parameter Y has no upper type bound, because technically these methods can return any type. However, in practise, it is very common to have them return an int.

Async objects

The optype variant of collections.abc.Awaitable[V]. The only difference is that optype.CanAwait[V] is a pure interface, whereas Awaitable is also an abstract base class.

operator operand
expression method type
await _ __await__ CanAwait[V]

Iteration

The operand x of iter(_) is within Python known as an iterable, which is what collections.abc.Iterable[K] is often used for (e.g. as base class, or for instance checking).

The optype analogue is CanIter[Ks], which as the name suggests, also implements __iter__. But unlike Iterable[K], its type parameter Ks binds to the return type of iter(_). This makes it possible to annotate the specific type of the iterable that iter(_) returns. Iterable[K] is only able to annotate the type of the iterated value. To see why that isn't possible, see python/typing#548.

The collections.abc.Iterator[K] is even more awkward; it is a subtype of Iterable[K]. For those familiar with collections.abc this might come as a surprise, but an iterator only needs to implement __next__, __iter__ isn't needed. This means that the Iterator[K] is unnecessarily restrictive. Apart from that being theoretically "ugly", it has significant performance implications, because the time-complexity of isinstance on a typing.Protocol is $O(n)$, with the $n$ referring to the amount of members. So even if the overhead of the inheritance and the abc.ABC usage is ignored, collections.abc.Iterator is twice as slow as it needs to be.

That's one of the (many) reasons that optype.CanNext[V] and optype.CanNext[V] are the better alternatives to Iterable and Iterator from the abracadabra collections. This is how they are defined:

operator operand
expression function type method type
next(_) do_next DoesNext __next__ CanNext[V]
iter(_) do_iter DoesIter __iter__ CanIter[Vs: CanNext]

Async Iteration

Yes, you guessed it right; the abracadabra collections made the exact same mistakes for the async iterablors (or was it "iteramblers"...?).

But fret not; the optype alternatives are right here:

operator operand
expression function type method type
anext(_) do_anext DoesANext __anext__ CanANext[V]
aiter(_) do_aiter DoesAIter __aiter__ CanANext[Vs: CanAnext]

But wait, shouldn't V be a CanAwait? Well, only if you don't want to get fired... Technically speaking, __anext__ can return any type, and anext will pass it along without nagging (instance checks are slow, now stop bothering that liberal). For details, see the discussion at python/typeshed#7491. Just because something is legal, doesn't mean it's a good idea (don't eat the yellow snow).

Containers

operator operand
expression function type method type
len(_) __len__ CanLen
_.__length_hint__() (docs) __length_hint__ CanLengthHint
_[k] do_getitem DoesGetitem __getitem__ CanGetitem[K, V]
_.__missing__() (docs) __missing__ CanMissing[K, V]
_[k] = v do_setitem DoesSetitem __setitem__ CanSetitem[K, V]
del _[k] do_delitem DoesDelitem __delitem__ CanDelitem[K]
k in _ do_contains DoesContains __contains__ CanContains[K]
reversed(_) __reversed__ CanReversed[Y]

Because CanMissing[K, M] generally doesn't show itself without CanGetitem[K, V] there to hold its hand, optype conveniently stitched them together as optype.CanGetMissing[K, V, M].

Additionally, optype provides protocols for types with (custom) hash or index methods:

operator operand
expression function type method type
hash(_) do_hash DoesHash __hash__ CanHash[V]
_.__index__() (docs) do_index DoesIndex __index__ CanIndex[V]

Attribute access

operator operand
expression function type method type
v = _.k or
v = getattr(_, k)
do_getattr DoesGetattr __getattr__ CanGetattr[K: str, V]
_.k = v or
setattr(_, k, v)
do_setattr DoesSetattr __setattr__ CanSetattr[K: str, V]
del _.k or
delattr(_, k)
do_delattr DoesDelattr __delattr__ CanDelattr[K: str]
dir(_) do_dir DoesDir __dir__ CanDir[Vs: CanIter]

Descriptors

Interfaces for descriptors.

operator operand
expression method type
class T: d = _ __set_name__ CanSetName[T]
u = T.d
v = T().d
__get__ CanGet[T: object, U, V]
T().k = v __set__ CanSet[T, V]
del T().k __delete__ CanDelete[T]

Context managers

Support for the with statement.

operator operand
expression method(s) type
with _ as v __enter__, __exit__ CanWith[V, R]
__enter__ CanEnter[V]
__exit__ CanExit[R]

For the async with statement the interfaces look very similar:

operator operand
expression method(s) type
async with _ as v __aenter__, __aexit__ CanAsyncWith[V, R]
__aenter__ CanAEnter[V]
__aexit__ CanAExit[R]

Buffer types

Interfaces for emulating buffer types using the buffer protocol.

operator operand
expression method type
v = memoryview(_) __buffer__ CanBuffer[B: int]
del v __release_buffer__ CanReleaseBuffer

Future plans

  • Support for Python versions before 3.12.
  • More standard library protocols, e.g. copy, dataclasses, pickle.
  • Typed mixins for DRY implementation of operator, e.g. for comparison ops GeFromLt, GtFromLe, etc as a typed alternative for functools.total_ordering. Similarly for numeric types, with e.g. __add__ and __neg__ a mixin could generate __pos__ and __sub__.
  • Dependency-free third-party type support, e.g. protocols for numpy's array interface.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

optype-0.2.0.tar.gz (24.8 kB view details)

Uploaded Source

Built Distribution

optype-0.2.0-py3-none-any.whl (17.9 kB view details)

Uploaded Python 3

File details

Details for the file optype-0.2.0.tar.gz.

File metadata

  • Download URL: optype-0.2.0.tar.gz
  • Upload date:
  • Size: 24.8 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.11.5 Linux/6.6.10-76060610-generic

File hashes

Hashes for optype-0.2.0.tar.gz
Algorithm Hash digest
SHA256 57d62a4996d838029a7c1b3e795cdf4b86c6a19819bb8aa7aa388d35316852a1
MD5 b9841ee7302e91755c390a37eb9f5a24
BLAKE2b-256 34d3f1430aa7f67a35b950519d0a128ba6ead83439d77e2a61e9f0fe09210a9a

See more details on using hashes here.

File details

Details for the file optype-0.2.0-py3-none-any.whl.

File metadata

  • Download URL: optype-0.2.0-py3-none-any.whl
  • Upload date:
  • Size: 17.9 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: poetry/1.8.2 CPython/3.11.5 Linux/6.6.10-76060610-generic

File hashes

Hashes for optype-0.2.0-py3-none-any.whl
Algorithm Hash digest
SHA256 3f0cbba199702fe314f16c88301c1190840d67d86a676068b3c2174950d6ba79
MD5 4bb04752ab834a0fe42c11aa2200b747
BLAKE2b-256 f33e39d036d9db53fb15c648833a330cc9e7345e43b8c74c2a5edde5b11c5d9b

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page