Skip to main content

Firecloud API bindings and FISS CLI

Project description

====================
1. Description
====================

FireCloud API bindings and FireCloud Service Selector (FISS)
Command-Line interface.

FireCloud:
https://software.broadinstitute.org/firecloud/

Github:
https://github.com/broadinstitute/fiss

====================
2. Contents
====================
1. Description
2. Contents
3. Requirements
4. Release Notes
5. FISS
6. Low-Level API
7. High-Level Classes
8. Yapsy Plugin System

This package contains low-level api bindings to FireCloud services,
high-level class abstractions for workspaces and entities, and the FISS
command-line tool for scripting FireCloud Tasks.

====================
3. Requirements
====================

Install this package via pip:
% pip install firecloud

Or download the source and install with setup.py:
% python setup.py install

This package depends on yapsy, oauth2client, and requests.

FireCloud uses the Google Cloud SDK (https://cloud.google.com/sdk/) to manage
authorization. To use the firecloud CLI or API, you must install the SDK and
login locally with

gcloud auth login

**Note: FireCloud encryts all web traffic with TLSv1.2. Some versions of
python will not function correctly, and must be recompiled with openssl
v1.02 or later. If recompiling is not an option, there is also a
dockerized version of the tool available on docker hub
(https://hub.docker.com/r/broadgdac/fiss/).

% docker pull broadgdac/fiss

====================
4. Release Notes
====================
v0.11.0 - New endpoint for paginated entity queries, also used by attr_get
to more reliably return attributes on large workspaces.

Also added new FISS command `attr_fill_null`, which will assign null
sentinel files to entities which do not have a valid attribute. This,
in a sense, "fills in" the sparse matrix of data for a given list of
attributes.

v0.10.1 - Fix Py3 compatibility issues.

v0.10.0 - Remove `fiss` entrypoint, to not overwrite fiss 1.0 installs. New fiss
"mop" command, to clean files that are not referenced in a workspace
attribute from the underlying google bucket.

v0.9.6 - Fix expected response code for entity_import

v0.9.5 - Fix urllib compatibility issue in upload_entities

v0.9.4 - Add ability to change inputs and other configuration details in a
workspace configuration

v0.9.3 - Improved documentation. Fixed url for method configurations in a workspace.

v0.9.2 - Remove file-based check for getting application default credentials,
improving support for Jupyter-based sessions.

v0.9.1 - Fix expected return code for flow_new.

v0.9.0 - Replace Httplib2 with Requests, which resolves the SSL handshake issue
for python3. Added six to required packages, making FireCloud fully
compatible with python 2 & 3. Add links to FireCloud's swagger
documentation.

v0.8.1 - Improved test coverage. Enhanced interactive classes. Fixed error logic
to match new server response.

v0.8.0 - firecloud.api now boasts 100% coverage of existing (useful) API endpoints.

Several api endpoints for manipulating NIH credentials and profile info
will deliberately remain unimplemented unless a good reason arises. Any
discrepancies are noted in the api module's docstring; see
help(firecloud.api) for a lengthy overview.

New Method and Configuration objects.
>>> m = Method("broad-firecloud-gdac", "gistic2", 1)


v0.7.4 - FISS functions can now display their source code with the -F option.

v0.7.3 - Improved documentation, with docstrings for every public function.
This enables python's builtin help system to be used from the REPL.
To see all the available firecloud api calls:
% python
>>> import firecloud
>>> help(firecloud.api)

v0.7.2 - Fix some commands that broke with the API changes

v0.7.1 - Add workspace functions for importing entities and creating entity sets.

v0.7.0 - Renamed package "firecloud", replacing "fissfc"
- firecloud_cli.py renamed fiss.py
- firecloud_api.py renamed api.py
Added workspace class for more flexible python interactivity. Example:
>>> w = Workspace.new("broad-firecloud-testing", "my_workspace")
>>> w.bucket()
"fc-92e4ic0a-60fb-454a-8ue0-d0efea803421"
>>> w.lock()
Added better error messaging for unexpected FireCloud Errors

v0.6.2 - Add '--protected' option for workspace creation.

v0.6.1 - Firecloud production API is now https://api.firecloud.org/api

v0.6.0 - Refactored the argument structure for inferring a default project.
Instead of passing the namespace as an optional input to a
subcommand, the project can be overriden explicitly with the -p
top-level flag. This cleans up the syntax for many commands, and
makes the help clearer.

Add --all option to space_list
attr_get:
* attributes now given a a list of positional arguments,
instead of comma-separated
* entity type limited to firecloud supported entities
Fixed help messages for many commands


v0.5.0 - Add python 3 support, so that fissfc should be installable to
either version. A TLS issue may still prevent fissfc from
working on some versions of python.

v0.4.1 - Add attr_get subcommand, as a replacement for fiss annot_get.
Clarify error message.

v0.4.0 - Implement default value for google project into most subcommands

v0.3.2 - Add check in firecloud_api for the existence of gcloud credentials

v0.3.1 - Add confirmation prompt for delete commands, and a
Flag to override them.

v0.3.0 - Added support for Yapsy plugins.

v0.0.1 - Initial Release

=====================================
5. FireCloud Service Selector (FISS)
=====================================

FISS enables high-level commands to be executed directly from the command line,
and is installed locally to /usr/local/bin/fiss.

FISS includes several forms of self-documentation. Type

% fiss --help

to get an overview. To list available commands, run

% fiss -l

Each command includes it's own help:

% fiss space_list -h

and as a bonus, to see how a command is implemented (including the underlying api calls),
use the "-F" option:

% fiss -F space_list

===========================
6. FireCloud Low-Level API
===========================
The following is a list of available endpoints as of v0.9.3. Swagger
documentation for the FireCloud orchestration layer is available at
https://api.firecloud.org/. See help(firecloud.api) for more details.

abort_sumbission()
clone_workspace()
copy_config_from_repo()
copy_config_to_repo()
copy_entities()
create_submission()
create_workspace()
create_workspace_config()
delete_entity()
delete_pair()
delete_pair_set()
delete_participant()
delete_participant_set()
delete_repository_method()
delete_sample()
delete_sample_set()
delete_workspace()
delete_workspace_config()
get_config_template()
get_entities()
get_entities_tsv()
get_entities_with_type()
get_entity()
get_inputs_outputs()
get_repository_config()
get_repository_config_acl()
get_repository_method()
get_repository_method_acl()
get_status()
get_submission()
get_submission_queue()
get_workflow_outputs()
get_workspace()
get_workspace_acl()
get_workspace_config()
list_billing_projects()
list_entity_types()
list_repository_configs()
list_repository_methods()
list_submissions()
list_workspace_configs()
list_workspaces()
lock_workspace()
ping()
rename_workspace_config()
unlock_workspace()
update_repository_config_acl()
update_repository_method()
update_repository_method_acl()
update_workspace_acl()
update_workspace_attributes()
update_workspace_config()
upload_entities()
upload_entities_tsv()
validate_config()

=======================
7. High-Level Classes
=======================

Workspace (firecloud.workspace.Workspace)
Entity (firecloud.entity.Entity)
Submisssion (firecloud.submission.Submission)
Config (firecloud.config.Config)

==========================
8. Yapsy plugins for FISS
==========================

The FISS CLI includes a yapsy-based plugin system for defining additional
command line options. Documentation coming soon.

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

firecloud-0.11.0.tar.gz (22.6 kB view details)

Uploaded Source

File details

Details for the file firecloud-0.11.0.tar.gz.

File metadata

  • Download URL: firecloud-0.11.0.tar.gz
  • Upload date:
  • Size: 22.6 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No

File hashes

Hashes for firecloud-0.11.0.tar.gz
Algorithm Hash digest
SHA256 f124467fed3436a587dd894c8154c79ff7677d547dab4eb83e87f76508b2caae
MD5 4eba2c2b13a9530bac514bf39e6ecf86
BLAKE2b-256 c5a21f065add7de1b45585b271ab8ab2da34a59eb301a4d2de78faca0c459ec7

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page