globus-action-provider-tools 0.7

Action Provider Tools Introduction

This is an experimental and unsupported toolkit to help developers build Action Providers for use in Globus Automate including for invocation via Globus Flows.

As this is experimental, no support is implied or provided for any sort of use of this package. It is published for ease of distribution among those planning to use it for its intended, experimental, purpose.

This README is intended to introduce the concepts, requirements and overview of this toolkit to be used when implementing an Action for use with Globus Automate. The three main sections are 1) an introduction to the interface required of all Actions, 2) A description of how to prepare your service to use Globus Auth, and 3) A description of the content of this toolkit and how it can be used in a Python implementation. This overview does not provide any ;guidance on how to host or operate the Action so that it is accessible via the internet.

Actions Overview

The fundamental purpose of the Globus Automate platform is to tie together multiple operations or units of work into a coordinated orchestration. We refer to each of these operations as an Action. In particular, the Flows service provides a means of coordinating multiple actions across potentially long periods of time to perform some aggregate function larger than any single Action provides. The Triggers service ties Events, or occurrences within a managed environment, to Actions such that each occurrence of the Event automatically invokes the Action associated with it.

In both the Flows and the Triggers cases, the Actions require a uniform interface for invocation, monitoring and possibly termination so that new Actions may be introduced without requiring customization or re-implementation of the invoking services. We refer to the service endpoints which can be invoked in this manner as Action Providers and the uniform interface for interacting with the Action Providers as the Action Provider Interface. We provide here an overview of the Action Provider Interface as a guide for use when implementing an Action Provider.

Setting Up an Action Provider in Globus Auth

The Action Provider Interface makes use of and is bound closely with authentication via the Globus Auth system. To authenticate RESTful requests using Globus Auth, a service must register as a “resource server”. This is a multi-step process involving use of both the Globus Auth developer portal, and the Globus Auth API for configuring various access control state. To help with this process, we provide a step-by-step guide to using Globus Auth for this purpose:

1. Register a new App on https://developers.globus.org using a browser. After insuring that you are logged in to the developer portal in a browser at this URL, perform the following steps:

   • Click Add another project

     • Provide a name, contact email and select which of your own Globus Auth linked identities are permitted to administer the project. In future interactions with the Globus Developer Portal to manipulate the resource server, you will be required to login with this identity.

   • Find your new, empty project, and select Add drop down and “new app”

     • Provide a name for the specific app within the project. This will be a common name displayed to users when they make use of the Action Provider.

     • When creating a resource server, the other fields on the app creation page are not used.

       • “Redirects” is not used, but a value must be provided. You can use a URL associated with your service or a placeholder value like “https://localhost”.

       • “Scopes” are not relevant so any entered values and make no difference, so should be left blank. The “Privacy Policy” and “Terms and Conditions” may be displayed to users making use of your action provider, but they are not required.

• Make note of the “Client Id” in the expanded description of your app. This value will be used elsewhere in the creation of the service and is often referenced as client_id.

• In the section “Client Secrets” click “Create a new secret”

  • Provide a name which is meaningful to you. It will not be displayed to other users.

  • Make note of the generated secret. Like the client_id this will be used later in development. Be sure not to lose it as it can only be displayed once. However, new client secrets can be created and old ones deleted at any time should the need for a replacement secret arise.

2. Use the Globus Auth REST API to introspect your Action Provider Resource Server and create required Scopes.

curl --user <client_id>:<client_secret> https://auth.globus.org/v2/api/clients/<client_id>

Note the use of the <client_id> and <client_secret> values generated during your registration on the Globus Developer Portal. A successful return from this command is a JSON representation of the Globus Auth client similar to:

{
  "client": {
    "scopes": [],
    "redirect_uris": [
      "https://localhost"
    ],
    "name": "My Action Provider",
    "links": {
      "privacy_policy": null,
      "terms_and_conditions": null
    },
    "grant_types": [
      "authorization_code",
      "client_credentials",
      "refresh_token",
      "urn:globus:auth:grant_type:dependent_token"
    ],
    "fqdns": [],
    "visibility": "private",
    "project": "a47b9014-9250-4e21-9de5-b4aac81d464b",
    "required_idp": null,
    "preselect_idp": null,
    "id": "8e98ba5a-21a9-4bef-ab6a-0fcdbed36405",
    "public_client": false,
    "parent_client": null
  }
}

Of note are the fields fqdns and scopes. fqdns stands for “Fully Qualified Domain Names” and these are DNS host names which have been associated with your client. These are not required, but if an fqdn is registered, then when scopes are created (see below), they will reflect the fqdn and will be easier to read and understand than the default scope representation (which will also be generated) which are simply opaque UUIDs. We briefly describe registering an FQDN below.

scopes are created to identify operations on the Action Provider. Typically, an Action Provide defines just one scope and it is exposed to users in the Action Provider’s introspection (GET /) information in the field globus_auth_scope. After the discussion of (optional) registration an fqdn we discuss required Scope creation below.

• Registering an FQDN for your Action Provider:

  • Create a TXT record in DNS for your hostname containing the <client_id> generated

    • This may take some time to propagate through DNS. You can shorten this time by setting the TTL value on the DNS record to a smaller value (such as 60 seconds)

  • Use the Globus Auth API to register the FQDN which it will do by insuring that the TXT record is in DNS for the client_id:

  curl -XPOST --user <client_id>:<client_secret> https://auth.globus.org/v2/api/clients/<client_id>/fqdns -d '{"fqdn": "<your FQDN string>"}'


- *Creating a Scope*: Creation of a scope is required as the scope will be used in authenticating REST calls on the Action Provider.

  - Start by creating a "scope definition" JSON document in the following format replacing the ``name``, ``description`` and optionally the ``scope_suffix``:

{
  "scope": {
    "name": "Action Provider Operations",
    "description": "All Operations on My Action Provider",
    "scope_suffix": "action_all",
    "dependent_scopes": [],
    "advertised": true,
    "allow_refresh_tokens": true
  }
}

The name and description fields are purely informative and will be presented to other users who use the Globus Auth API to lookup the scope. The scope_suffix will be placed at the end of the generated “scope string” which is a URL identifier for the scope. It provides the context for the operations this scope covers among all operations your service provides. For Action Providers, we commonly use action_all to indicate all operations defined by the Action Provider API, but any string is acceptable.

dependent_scopes define scopes of other Globus Auth resource servers that your Action Provider will invoke to perform its work. For example, if your Action Provider uses Globus Transfer to first move some data to compute upon, the scope for the Globus Transfer service would be placed in the dependent_scopes list. In the most common case in which the Action Provider does not rely on other services, this list should be empty. Enumerating scopes which may be useful, such as the example Globus Transfer used here, or how authentication is performed using dependent scopes is beyond the scope of this document. We encourage you to consult the Globus Auth Documentation for more information.

The advertised property indicates whether the scope will be visible to all users who do scope look ups on Globus Auth. You may select either true or false for this depending on your own policy. allow_refresh_tokens should generally be set to true, indicating that a client of the Action Provider who has authenticated the user via Globus Auth is allowed to refresh that authentication without further interactions from the user. Especially in the case where an Action may be long running and is monitored by an automated system like Globus Flows, it is important that token refresh is permitted.

With the scope creation JSON document complete, the scope will be created in Globus Auth with the following REST interaction via the curl command:

curl --user <client_id>:<client_secret> -H Content-Type: application/json -XPOST     https://auth.globus.org/v2/api/clients/<client_id>/scopes -d '<Insert Scope creation document from above>'

This should return the definition of the new scope matching the values provided in your scope creation document. As an example:

{
  "scopes": [
    {
      "dependent_scopes": [],
      "description": "<your description>",
      "allows_refresh_token": true,
      "client": "<client_id>",
      "advertised": true,
      "scope_string": "https://auth.globus.org/scopes/<client_id>/action_all",
      "id": "<A UUID for this scope>",
      "name": "<your scope name>"
    }
  ]
}

The returned scope_string, which always takes the form of a URL, will be the value exposed to users who wish to authenticate with Globus Auth to use your Action Provider. It will be part of the Action Provider description document, returned on the Action Provider Introspection operation (GET /) with the key globus_auth_scope.

Note that the returned value is an array of scopes. That is, more than one scope definition may be generated from the single scope creation request. This happens in the case where an FQDN has been registered for your client_id. In this case, a similar scope definition will also be generated, but the scope_string will contain the FQDN value(s). The scope_string values may be used interchangeably both by users requesting authentication to the Action Provider and in the globus_auth_scope value of the Action Provider Description.

Using the Toolkit

This toolkit provides the following components:

1. Authentication helpers that make it easier to validate Globus Auth tokens and determine if a given request should be authorized

2. An OpenAPI v3 specification and associated helpers that can be used to validate incoming requests and verify the responses your Action Provider generates. This document also defines the interface which must be supported by your REST API to have it function as an Action Provider.

3. Simple bindings for the document types “Action Request” and “Action Status” to Python NamedTuple representations and helper functions for serializing and deserializing these structures to/from JSON.

4. Helper methods for binding the REST API calls defined by the Action Interface to a Flask application. These helpers will perform the Authentication and Validation steps (as provided by components 1 and 2) and communicate with an Action Provider implementation using the structures defined in 3. For those users building an Action Provider using Flask, this provides a simplified method of getting the REST API implemented and removing common requirements so the focus can be on the logic of the Action provided.

pip install globus-action-provider-tools

from globus_action_provider_tools.authentication import TokenChecker
# You will need to register a client and scope(s) in Globus Auth
# Then initialize a TokenChecker instance for your provider:
checker = TokenChecker(
    client_id='YOUR_CLIENT_ID',
    client_secret='YOUR_CLIENT_SECRET',
    expected_scopes=['https://auth.globus.org/scopes/YOUR_SCOPES_HERE'],
    expected_audience='YOUR_CLIENT_NAME',
)

access_token = request.headers['Authorization'].replace('Bearer ', '')
auth_state = checker.check_token(access_token)

# This user's Globus identities:
auth_state.identities
# frozenset({'urn:globus:auth:identity:9d437146-f150-42c2-be88-9d625d9e7cf9',
#           'urn:globus:auth:identity:c38f015b-8ad9-4004-9160-754b309b5b33',
#           'urn:globus:auth:identity:ffb5652b-d418-4849-9b57-556656706970'})
# Groups this user is a member of:
auth_state.groups
# frozenset({'urn:globus:groups:id:606dbaa9-3d57-44b8-a33e-422a9de0c712',
#           'urn:globus:groups:id:d2ff42bc-c708-460f-9e9b-b535c3776bdd'})

all_principals = auth_state.identities.union(auth_state.groups)

resource_allows = ['urn:globus:auth:identity:c38f015b-8ad9-4004-9160-754b309b5b33']
auth_state.check_authorization(resource_allows)
# True

resource_allows = ['public']
auth_state.check_authorization(resource_allows, allow_public=True)
# True
resource_allows = ['all_authenticated_users']
auth_state.check_authorization(resource_allows, allow_all_authenticated_users=True)
# True

from globus_action_provider_tools.validation import request_validator, response_validator
# Validating a request
result = request_validator.validate(request)
# Or a response:
result = response_validator.validate(request, response)
# raise errors if invalid
result.raise_for_errors()
# or get list of errors
errors = result.errors

from globus_action_provider_tools.data_types import (
    ActionProviderDescription,
    ActionProviderJsonEncoder,
    ActionRequest,
    ActionStatus,
    ActionStatusValue,
)
status = ActionStatus(
    action_id=str(uuid.uuid4()),
    status=ActionStatusValue.SUCCEEDED,
    creator_id=caller_id,
    monitor_by=request.monitor_by,
    manage_by=request.manage_by,
    start_time=str(datetime.datetime.now().isoformat()),
    completion_time=str(datetime.datetime.now().isoformat()),
    release_after=60 * 60 * 24 * 30,  # 30-days in seconds
    display_status=ActionStatusValue.SUCCEEDED.name,
    details=result_details,
)
json_string = json.dumps(action_status, cls=ActionProviderJsonEncoder)

from globus_action_provider_tools.data_types import (
    ActionProviderDescription,
    ActionRequest,
    ActionStatus,
    ActionStatusValue,
)
from globus_action_provider_tools.flask import (
    ActionStatusReturn,
    add_action_routes_to_blueprint,
)
action_blueprint = Blueprint("action", __name__, url_prefix="/action")
provider_description = ActionProviderDescription(
    globus_auth_scope="<scope created in Globus Auth>",
    title="My Action Provider",
    admin_contact="support@example.com",
    synchronous=True,
    input_schema={}, # JSON Schema representation of the input on the request
    log_supported=False
)
add_action_routes_to_blueprint(
    action_blueprint,
    CLIENT_ID,
    CLIENT_SECRET,
    CLIENT_NAME,
    provider_description,
    action_run,
    action_status,
    action_cancel,
    action_release,
)

Appendix: Example Action Provider

skeleton_blueprint = Blueprint("skeleton", __name__, url_prefix="/skeleton")

input_schema = {
    "$id": "https://automate.globus.org/skeleton_action_provider.input.schema.json",
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "Skeleton Action Provider Input Schema",
    "type": "object",
    "properties": {"input_string": {"type": "string"}},
    "additionalProperties": False,
    "required": ["input_string"],
}

provider_description = ActionProviderDescription(  globus_auth_scope="https://auth.globus.org/scopes/16e16447-209a-4825-ae19-25e279d91642/action_all_with_groups",
    title="skeleton_action_provider",
    admin_contact="support@globus.org",
    synchronous=True,
    input_schema=input_schema,
    log_supported=False,  # This provider doesn't implement the log callback
)

# A simulated database mapping input user action requests identifiers to a previously
# seen request id and the corresponding action id
_fake_request_db: Dict[str, Tuple[ActionRequest, str]] = {}

_fake_action_db: Dict[str, ActionStatus] = {}

def _authorize_action_access(
    action_id: str, auth_state: AuthState, policy_type: str = "manage_by") -> ActionStatus:
    status = _fake_action_db.get(action_id)
    if status is None:
        raise NotFound(f"No Action with id {action_id}")
    if policy_type == "manage_by":
        policy_group = status.manage_by
    elif policy_type == "monitor_by":
        policy_group = status.monitor_by
    if policy_group is None:
        policy_group = []
    allowed_set = set([status.creator_id] + [status.monitor_by])
    if auth_state.check_authorization(allowed_set, allow_all_authenticated_users=True):
        return status
    else:
        raise NotFound(f"No Action with id {action_id}")

def action_run(request: ActionRequest, auth: AuthState) -> ActionStatusReturn:
    caller_id = auth.effective_identity
    request_id = request.request_id
    full_request_id = f"{caller_id}:{request_id}"
    prev_request = _fake_request_db.get(full_request_id)
    if prev_request is not None:
        if prev_request[0] == request:
            return action_status(prev_request[1], auth)
        else:
            raise Conflict(
                f"Request with id {request_id} already present with different parameters "
            )
    # Local processing would happen here
    result_details = {
        # This is safe because the input has been validated against the input schema
        "you_said": request.body["input_string"]
    }
    status = ActionStatus(
        action_id=str(uuid.uuid4()),
        status=ActionStatusValue.SUCCEEDED,
        creator_id=caller_id,
        monitor_by=request.monitor_by,
        manage_by=request.manage_by,
        start_time=str(datetime.datetime.now().isoformat()),
        completion_time=str(datetime.datetime.now().isoformat()),
        release_after=request.release_after or "P30D",
        display_status=ActionStatusValue.SUCCEEDED.name,
        details=result_details,
    )
    _fake_request_db[full_request_id] = (request, status.action_id)
    _fake_action_db[status.action_id] = status
    return status

def action_status(action_id: str, auth: AuthState) -> ActionStatusReturn:
    return _authorize_action_access(action_id, auth, "monitor_by"), 200

def action_cancel(action_id: str, auth: AuthState) -> ActionStatusReturn:
    status = _authorize_action_access(action_id, auth, "manage_by")
    if status.status in (ActionStatusValue.SUCCEEDED, ActionStatusValue.FAILED):
        return status
    # Process Action cancellation
    status.status = ActionStatusValue.FAILED
    status.display_status = "Canceled by user request"
    return status

def action_release(action_id: str, auth: AuthState) -> ActionStatusReturn:
    status = _authorize_action_access(action_id, auth, "manage_by")
    if status.status not in (ActionStatusValue.SUCCEEDED, ActionStatusValue.FAILED):
        raise Conflict("Action is not complete")
    _fake_action_db.pop(action_id)
    # Both fake and badly inefficient
    remove_req_id: Optional[str] = None
    for req_id, req_and_action_id in _fake_request_db.items():
        if req_and_action_id[1] == action_id:
            remove_req_id = req_id
            break
    if remove_req_id is not None:
        _fake_request_db.pop(remove_req_id)
    return status, 200

def main():
    app = Flask(__name__)
    app.url_map.strict_slashes = False
    add_action_routes_to_blueprint(
        skeleton_blueprint,
        CLIENT_ID,
        CLIENT_SECRET,
        CLIENT_NAME,
        provider_description,
        action_run,
        action_status,
        action_cancel,
        action_release,
    )
    app.register_blueprint(skeleton_blueprint)
    app.run(debug=True)

if __name__ == "__main__":
    main()