nti.webhooks 0.0.6

Webhooks

Webhooks are HTTPS requests from one party — the source — to another party, the destination. These requests are one-way: the source sends the request to the destination, and aside from conforming that the request was received, takes no further action (the request’s response from the destination is irrelevant). These requests are sent from the source to let the destination know that something has happened: a new entity (or resource, in the REST sense) has been created, an old one updated or deleted, etc. Such requests typically carry a payload in the body providing information about the action (usually the representation of the affected resource). Destinations are identified via complete URL; destinations may expect to be informed of events affecting one, several, or all possible types of entities handled by the source.

This Package

This package is installed in a source server and manages the registration and sending of webhooks. The registrations may be either static, or they may be dynamic, as in the case of REST Hooks, where individual “subscriptions” may be started and stopped.

This package is intended to integrate with highly event-driven applications using zope.event, that define their resources using zope.interface, manage event delivery, resource adaptation, and dependency injection using zope.component, and (optionally) implement a hierarchy of component registries using zope.site and nti.site. Data persistence is provided through persistent objects, typically with ZODB.

Out Of Scope

Certain concerns are out of scope for this package (but other packages built upon this package my provide them). These include, but are not limited to:

• Providing a user interface for managing subscriptions.

• Providing an HTTPS API for managing subscriptions. This package provides the underlying data storage, but accepting parameters, etc, and marshaling them into the correct Python calls, is not a concern here.

• Providing a user interface or HTTPS API for viewing webhook audit logs.

• Enabling webhooks to fire only for specific objects. This package deals with scopes (sites) and kinds of objects, not individual instances.

In Scope/Features

Certain concerns are very much in scope for this package, and this package should provide a complete, easy to use solution that addresses these concerns. Where necessary, if a concern cannot be addressed directly by this package, extension points (interfaces and zope.component utilities) may be defined. These include, but are not limited to:

• Resource Representation

  The on-the-wire form of the resources is built using nti.externalization.

  To allow customization of the external forms, a named externalizer is used; nti.externalization will fall back to the default externalizer if no externalizer of the given name is available. The default externalizer is named “webhook-delivery”, but dialects may use something different.

• Alternate Webhook Dialects

  Webhooks are a general protocol and mostly interoperable. But to support cases where particular destinations have specific requirements, “dialects” are used. There is a default dialect and then there may be specializations of it. Each webhook subscription may have associated with it the name of a dialect to use. These dialects are found in the component registry. For example, a dialect may choose to use a different externalizer name such as “zapier-webhook-delivery”.

• Transactional

  Webhooks should not be delivered if the ultimate creation or persistence of a resource failed. To this end, webhook delivery in this package is integrated with the transaction package.

  Resources are externalized during a late phase of the transaction commit process; the details about the delivery are recorded and persisted, and only after the transaction is successfully committed does the HTTP request get made.

• Concurrency

  Webhook delivery and record keeping should be lightweight, and all actual network IO should proceed in a non-blocking fashion. This means that this package will spawn threads (or greenlets, using gevent.

• Error Handling/Failure Retry

  A limited amount of retry logic is provided by this package, but that does not extend to process boundaries. If the process hosting this package is killed while a delivery is pending, no automatic provision is made to resume delivery attempts in any other process.

  The API is present to allow that to be implemented, though.

• Auditing/Delivery History

  For each subscription, delivery attempts, status, and responses are stored in a ring-buffer like structure. This can be inspected to see if deliveries succeeded, failed, or never completed.

• Access Control on Deliveries

  Each subscription is associated with an IPrincipal that owns it. A request is only delivered to a subscription if the IPrincipal that owns the subscription can access the entity, as determined by zope.security.

• Access Control on Subscriptions

  While not enforced by this package, the above owner relationship will be used to provide role managers that grant read and read/write access to remove subscriptions only to the owner of the subscription.

  TODO: Make sure client packages can extend that to provide for admin access. So long as we don’t DENY it should be fine.

• Hierarchy of Subscriptions

  Subscriptions are made within a particular Zope site (the closest enclosing site to a resource when a resource is subscribed to, or the currently active site otherwise). These sites may have parents.

  TODO: Work out the details of that.

  When an event is received that might result in webhook delivery, active subscriptions are checked for in the currently active site, as well as in the sites up the hierarchy of the resource itself. All applicable subscribers will get a delivery.

  For example, if the president of the company (an administrator) subscribes to “new user created” events at the global (root, base or “/”) level, and a department head subscribes to “new user created” for their department (“/NOAA”), while a local office manager subscribes to events for their office (“/NOAA/NWS/OUN”), then creating a new user in the OKC office may send three deliveries, one to the manager, one to the secretary, and one to the president.

• Converting From Object Events to Webhook Events

  TODO: Write me.

  This package needs to have a clear way to have client packages specify what events should produce webhook deliveries. The exact mechanism is TBD. Possibly clients are expected to use <classImplements> ZCML directives to apply marker interfaces? Or they might register a subscriber provided by this package for their own existing interfaces?

  We want this process, and the process of finding all active subscriptions, to be fast. I’m imagining something like view lookup, keeping active subscriptions in the various component registries? That doesn’t work non-persistently.

0.0.6 (2021-09-07)

• Make subscriptions, delivery attempts, delivery attempt requests, and delivery attempt responses have a mimeType value when externalized.

0.0.5 (2020-12-04)

• Add support for Python 3.9.

• Principal IDs are no longer required to be URIs or dotted names. See issue 21.

0.0.4 (2020-09-16)

• Use a custom ITraverser when finding sites to install persistent ZCML subscriptions in. This traverser fires IBeforeTraverseEvent notifications, letting subscribers to that (such as nti.site.subscribers.threadSiteSubscriber) take action (such as making sites current when they’re about to be traversed). This can help when the site path contains namespaces.

0.0.3 (2020-08-24)

• Move permission definition to a separate file, permissions.zcml, that is included by default. Use the ZCML <exclude> directive before including this package’s configuration if you were experiencing configuration conflicts.

0.0.2 (2020-08-06)

• Add a subscriber and methods to remove subscriptions when principals are deleted. See PR 17.

0.0.1 (2020-08-05)

• Initial PyPI release.