pyhorn 0.8.0

Install

pip install pyhorn

Overview

Data available from the REST endpoints is accessible through methods of the MHClient object. In many cases the response data is encapsulated in additional classes to make accessing various attributes of the data possible using object notation. For instance, data about Matterhorn workflows can be accessed using the client.workflows() method. The return value will be a list of endpoints.workflow.Workflow objects. The operations attribute of those objects will get you a list of endpoints.workflow.WorkflowOperation objects. And so forth.

Currently there are only a handfull of endpoints wrapped in this way, and only a few convenience classes and methods defined for each one. The idea is to continue adding more as I (or you) need them. Pull requests welcome!

MHClient Method List

In the case of methods that accept a set of keyword arguments, the list of expected kwargs is mapped directly from the Matterhorn endpoint. In other words, if you want to know what kwargs to use for MHClient.workflows(), check the corresponding entry in the REST API docs at http://matterhorn.example.edu/docs.html?path=/workflow.

• endpoints() - /info/components.json

• me() - /info/me.json

• workflows(**kwargs) - /workflow/instances.json

• workflow(instance_id) - /workflow/instance/{id}.json

• episodes(**kwargs) - /episode/episode.json

• episode(episode_id) - /episode/episode.json

• user_actions(**kwargs) - /usertracking/actions.json

• agents() - /capture-admin/agents.json

• agent(agent_name) - /capture-admin/agents/{agent_name}.json

• hosts() - /services/hosts.json

• job(job_id) - /services/job/{job_id}.json

• search_episodes(**kwargs) - /search/episode.json

• search_episode(episode_id) - /search/episode.json

• statistics() - /services/statistics.json

Example Usage

Create the client interface…

>>> from pyhorn import MHClient
>>> client = MHClient('http://matterhorn.example.edu', 'user', 'passwd')

User/pass combo should be for your REST API user, typically the “matterhorn_system_account” user. It is possible to create and use a client object without a user/pass combo (omit the constructor args), but requests will only work for endpoints that do not require auth (e.g., episode search)

The default request timeout is 5 seconds. Pass timeout=n to the MHClient constructor to use something else.

Get a list of available endpoints…

>>> client.endpoints()
[{u'description': u'Capture Agent Admin REST Endpoint',
    u'docs': u'http://matterhorn.example.edu80/capture-admin/docs',
    u'path': u'/capture-admin',
    u'type': u'org.opencastproject.capture.admin',
    u'version': u'1.4.4',
    u'wadl': u'http://matterhorn.example.edu80/capture-admin/?_wadl&_type=xml'},
    {u'description': u'EpisodeService REST Endpoint',
    u'docs': u'http://matterhorn.example.edu80/episode/docs',
    u'path': u'/episode',
    u'type': u'org.opencastproject.episode',
    u'version': u'1.4.4',
    u'wadl': u'http://matterhorn.example.edu80/episode/?_wadl&_type=xml'},
...

Get list of current workflow instances…

>>> wfs = client.workflows()
>>> for wf in wfs:
        print wf.id + ": " + wf.state
1646: STOPPED
1649: STOPPED
1651: STOPPED
1655: STOPPED
4211: SUCCEEDED
14479: SUCCEEDED
14486: SUCCEEDED
441: STOPPED
445: STOPPED
...

… or just the successful ones…

>>> wfs = client.workflows(state="SUCCEEDED")

… or the operations for a particular instance…

>>> wf = client.workflow(instance_id=1646)
>>> ops = wf.operations
>>> for op in ops:
    print op.id + ": " + op.state
apply-acl: SUCCEEDED
tag: SUCCEEDED
tag: SUCCEEDED
inspect: SUCCEEDED
prepare-av: SUCCEEDED
prepare-av: SUCCEEDED
compose: SUCCEEDED
compose: SUCCEEDED
...

Get the list of currently configured capture agents

>>> cas = client.agents()
>>> for ca in cas:
    print ca.name + ": " + ca.state
epiphan001: unknown
epiphan002: unknown
ewave001: idle
ewave002: idle
ncast001: idle
ncast002: shutting_down

Endpoint Object Wrappers

pyhorn attempts to make the Matterhorn API responses more convenient to work with by wrapping the json response data in a set of classes that provide easy access via object attributes and automatic “dereferencing” of associated data.

The following endpoint data classes are defined:

• Workflow

• WorkflowOperation

• ServiceJob

• Episode

• Mediapackage

• MediaTrack

• CaptureAgent

• UserAction

• Search

These are just the initial set because they represent the data I needed to deal with in the other projects that prompted the creation of pyhorn. It is trivial to add additional wrapper classes. Pull requests welcome!

Attribute access

Endpoint data classes inherit from pyhorn.endpoints.base.EndpointObj. The json response data is stored in a _raw attribute and made accessible via dot-notation by overriding __getattr__. A simple illustration:

>>> from pyhorn.endpoints.base import EndpointObj
>>> obj = EndpointObj({"foo": "bar", "baz": [1,2,3]}, client)
>>> obj.foo
bar
>>> obj.baz
[1, 2, 3]
>>> obj.abc
Traceback ...
...
AttributeError: response data for <class 'pyhorn.endpoints.base.EndpointObj'> has no key 'abc'

At this point the dot-notation access only works for top-level values. There is a EndpointObj.raw_get method that accepts a path_key argument if you need to access something deeper in the response structure.

>>> obj = EndpointObj({"foo": {"bar": {"baz": 1}}})
>>> obj.raw_get("foo.bar.baz")
1

Dereferencing

In a handful of cases accessing certain attributes (@property, actually) of an endpoint data wrapper object will return an instance or instances of a different wrapper class. For example, Workflow.operations will extract the operation data from the raw json and return a list of WorkflowOperation objects that wrap the individual operation data structures contained in the original response.

This works also for dereferencing data that requires an additional request to the Matterhorn API. For instance, Accessing the WorkflowOperation.job property triggers a request to the /services/job/{job_id}.json, with the response being wrapped in a ServiceJob object and returned.

The current list of these dereferencing relationships is:

• Workflow.operations -> list of WorkflowOperation objects

• Workflow.job -> ServiceJob

• Workflow.episode -> Episode

• Workflow.mediapackage -> Mediapackage

• WorkflowOperation.job -> ServiceJob

• ServiceJob.parent -> ServiceJob

• ServiceJob.children -> list of ServiceJob objects

• Episode.mediapackage -> Mediapackage

• Mediapackage.tracks -> list of MediaTrack objects

• UserAction.episode -> SearchEpisode

Setting Maintenance Mode

As of v0.4.0 you can toggle the maintenance mode on a host.

>>> hosts = client.hosts()
>>> for host in hosts:
        host.set_maintenance(True)

Caching

As of v0.7.0 the use of requests_cache has been dropped in favor of an internal cache that stores the responses from a subset of the single-item endpoint methods. A future goal is to allow more granular control over the caching policy, but for now the following endpoint responses are cached:

• CaptureEndpoint.agent

• EpisodeEndpoint.episode

• SearchEndpoint.episode

• Workflow.instance

Caching works via a decorator function on the endpoint methods. The JSON response data from the Matterhorn API is cached in-memory with each entry assigned a time-to-live (ttl) value to control expiration. Each cached method also has a configured max_entries value. If/when the number of entries reaches that limit a cull operation will prune 1/3 of the existing entries.

To disable caching altogether pass cache_enabled=False to the MHClient constructor.

To clear the cache call client.clear_cache().

Testing

During development pyhorn tests are executed using pytest.

To run the tests from a local git clone:

pip install -r tests/requirements.txt

then

python setup.py test

License

pyhorn is licensed under the Apache 2.0 license

Copyright

2015 President and Fellows of Harvard College

Release History