Upload accounting records to an eudat accounting server
Project description
Upload accounting records to an eudat accounting server
Administrators of (storage) resources provided through the EUDAT Common Data Infrastructure can use this tool to conveniently report current resource consumption per registered resource. Accounting records are submitted per individual resource identified by its (P)ID which is available as soon as the resource has been registered at EUDAT’s Data Project Management Tool (DPMT).
Default settings are such that only the resource id and the consumed value need to be provided. The default unit is byte and the default resource type is set to storage. The full documentation of options supported is described in the next section.
If the tools provided here do not address your use case you can either file a feature request on the tracker or use a more low-level approach as outlined in the server documentation.
Full documentation and API
Installation
The easiest way to install the tool is via pip or easy_install. It is usually best to do this in a virtualenv:
$ pip install eudat.accounting.client
Command line interface
As a result of the above there are now two console script called addRecord and iRODScollector. Invoke it with -h to see its usage pattern and options.
addRecord
$ bin/addRecord -h
usage: addRecord [-h] [--version] [-b BASE_URL] [-u USER] [-p PASSWORD]
[-d DOMAIN] [-s SERVICE] [-n NUMBER] [-o OBJECT_TYPE]
[-k KEY] [-T TYPE] [-m MEASURE_TIME] [-C COMMENT] [-t] [-v]
account value [unit]
positional arguments:
account account to be used. Typically the (P)ID of the
resource to be accounted
value The value to be recorded
unit The unit of measurement for the value provided.
Default: "byte"
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-b BASE_URL, --base_url BASE_URL
base URL of the accounting server to use. Default:
https://accounting.eudat.eu
-u USER, --user USER user id used for logging into the server. If not
provided it is looked up in the environment variable
"ACCOUNTING_USER". Default: "" - aka not set
-p PASSWORD, --password PASSWORD
password used for logging into the server. If not
provided it is looked up in the environment variable
"ACCOUNTING_PW". Default: "" - aka not set
-d DOMAIN, --domain DOMAIN
name of the domain holding the account. Default: eudat
-s SERVICE, --service SERVICE
UID (or PID) of the registered service component
reporting the record. Default: "" - not set
-n NUMBER, --number NUMBER
number of objects associated with this accounting
record. This is EUDAT specific. Default: "" - not set
-o OBJECT_TYPE, --object_type OBJECT_TYPE
object type for the number of objects specified with
"-n". This is EUDAT specific. Default: "registered
objects"
-k KEY, --key KEY key used to refer to the record. If not set the
accounting server will create the key. Specifying an
existing key will overwrite the existing record.
Default: "" - not set
-T TYPE, --type TYPE type of the resource accounted. Default: storage
-m MEASURE_TIME, --measure_time MEASURE_TIME
measurement time of the accounting record if different
from the current time. Default: "" - not set
-C COMMENT, --comment COMMENT
arbitrary comment (goes into the meta dictionary).
Default: "" - not set
-t, --test Dry run. Don't push data to server - run only locally
Default: off
-v, --verbose return the key of the accounting record created.
Default: off
iRODScollector
$ bin/iRODScollector -h
usage: iRODScollector [-h] [--version] [-c CONFIGPATH] [-k KEY] [-T TYPE]
[-m MEASURE_TIME] [-C COMMENT] [-t] [-v]
optional arguments:
-h, --help show this help message and exit
--version show program's version number and exit
-c CONFIGPATH, --configpath CONFIGPATH
path to configuration file. Default:
"./irodscollector.cfg" (in the current working
directory)
-k KEY, --key KEY key used to refer to the record. If not set the
accounting server will create the key. Specifying an
existing key will overwrite the existing record.
Default: "" - not set
-T TYPE, --type TYPE type of the resource accounted. Default: storage
-m MEASURE_TIME, --measure_time MEASURE_TIME
measurement time of the accounting record if different
from the current time. Default: "" - not set
-C COMMENT, --comment COMMENT
arbitrary comment (goes into the meta dictionary).
Default: "" - not set
-t, --test Dry run. Don't push data to server - run only locally
Default: off
-v, --verbose return the key of the accounting record created.
Default: off
A template configuration file is included in the distribution and looks like this:
$ cat irodscollector.ini
#
# template of a configuration file for EUDAT's irodscollector
#
# section containing the logging options
[Logging]
log_file=eudatacct.log
# section containing the properties to access the accounting server
# to get statistical data and report them
[Report]
# base URL of the accounting server to be used
base_url=https://accounting.eudat.eu
# domain: either eudat or test or demo
domain=eudat
# uid of the corresponding registered storage resource on DPMT
# (same as storage_space_uuid on RCT)
account=<insert uid here>
# username of the provider on the accouniting server
# owning the account specified above
# contact dp-admin@mpcdf.mpg.de if you need one
user=<username of provider>
# if you have an access token from RCT already reuse that here
password=<password or access token>
service_uuid=<unsuported at the moment>
# section contains the list of collections to be accounted together, replace
# the examples with your collections, the script sums the values of all
# collections and sends it to EUDAT's accounting service.
[Collections]
clist=
/zone/some/path
/zone/other/path
Copy this to irodscollector.cfg and adapt it to your site.
Most of this should be self-explaining. Note that you need to provide credentails for the accounting service. If you do not have any contact the EUDAT accounting manager.
In addition, you need to make sure that the user invoking this script has a suitable iRODS environment set up.
Basic usage information as well as error messages are logged to a file named .accounting.log in the current working directory from where addRecord has been invoked.
Developer notes
Please use a virtualenv to maintain this package, but I should not need to say that.
The package can be installed directly from GitHub:
$ pip install git+git://github.com/EUDAT-DPMT/eudat.accounting.client
The code is organized in a nested namespace package, i.e., the real action is happening in the subdirectory
$ cd src/eudat/accounting/client
Start looking around there.
Run the tests (not really that meaningful so far):
$ python setup.py test
$ python run_tests.py
Links
Project home page
Source code
Issues tracker
https://github.com/EUDAT-DPMT/eudat.accounting.client/issues
1.0.1 (2017-08-25)
addRecord now also supports the dry run option ‘-t’ This addresses issue #3 [raphael-ritz]
1.0.0 (2017-03-15)
Adding iRODScollector based on Claudio’s and Pavel’s work [raphael-ritz]
Python 3 compatibility [raphael-ritz]
1.0.0rc1 (2016-07-22)
Initial release [raphael-ritz]
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Hashes for eudat.accounting.client-1.0.1.tar.gz
Algorithm | Hash digest | |
---|---|---|
SHA256 | 1214ab1106b0928ffdbbba9c13cc46bfb6adad0f7cb932e67ff99770b475f94b |
|
MD5 | 383b5358e674eecab0ca73b91e7f946a |
|
BLAKE2b-256 | 54e9815414be33c94e76ce71ba8e614442ab3666df235f38c470970a55a77799 |