Skip to main content

The server library for the GWDataFind service

Project description

This module defines a Flask App that serves URLs based on the contents of a diskcache.

Requirements

  • Python >= 3.4

  • Flask >= 1.0.0

Apache + Gunicorn configuration

This service runs under Gunicorn behind an Apache proxy, which is used to verify X.509 certificates. The certificate information is then passed to the application in the header where the subject is authenticated against a grid-mapfile.

Add the following to the Apache system config file, e.g. /etc/sysconfig/httpd on CentOS 7

OPENSSL_ALLOW_PROXY_CERTS=1

Edit the appropriate 443 virtual host configuration to add

# Using Apache as a proxy to gunicorn
  ProxyPass "/LDR" "http://127.0.0.1:8080/" retry=0
  ProxyPassReverse "/LDR" "http://127.0.0.1:8080/"

  <Location "/LDR/">
       SSLRequireSSL
       SSLVerifyClient optional
       SSLVerifyDepth 10
       SSLOptions +ExportCertData +StrictRequire +LegacyDNStringFormat
       SSLOptions +StdEnvVars
       RequestHeader set SSL_CLIENT_S_DN "%{SSL_CLIENT_S_DN}s"
       RequestHeader set SSL_CLIENT_I_DN "%{SSL_CLIENT_I_DN}s"
  </Location>

Start a gunicron instance with

/usr/bin/gunicorn-3.6 --bind=127.0.0.1:8080 --workers=5 "gwdatafind_server:create_app()"

Sample configuration file can be put into /etc/gwdatafind-server.ini

[LDRDataFindServer]
# frame cache file
framecachefile = <path_to>/ascii_cache.dat
framecachetimeout = 10
# grid-mapfile
gridmapcachefile = <path_to>/grid-mapfile
gridmapcachetimeout = 60
# optional parameters
site_exclude_pattern = ^X$
frametype_exclude_pattern = .+EXCLUDE.+
frametype_include_pattern = .+_TEST_\d+
filter_preference = """{'^file': ['preferred', 'hdfs']}"""

Client API

The GWDataFind Server runs as a daemon providing a RESTful interface via Gunicorn + Apache. There is also a client application gw_data_find for command line usage.

The URLs supported areliwted below. They all support HTTP GET,HEAD, and OPTIONS methods.

URL (add HTTP[S]://<host>[:<port>]/LDR to beginning

Function

/services/data/v1/

Show URLs (debugging)

/services/data/v1/<ext>.json

Show observatories

/services/data/v1/<ext>/<site>.json

Show tags (frame types)

/services/data/v1/<ext>/<site>/<tag>/latest.json

Show latest

/services/data/v1/<ext>/<site>/<tag>/<start>,<end>.json

Show URLs

/services/data/v1/<ext>/<site>/<tag>/<start>,<end>/<urltype>.json

Show URLs of one type (file,URL)

/services/data/v1/<ext>/<site>/<tag>/segments.json

Show all available segments

/services/data/v1/<ext>/<site>/<tag>/segments/<start>,<end>.json

Show avalable segments in a time interval

/services/data/v1/<ext>/<site>/<tag>/latest.json

Show latest URL

/services/data/v1/<ext>/<site>/<tag>/<filename>.json

Show a single URL

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

gwdatafind-server-1.0.1.tar.gz (25.5 kB view details)

Uploaded Source

Built Distribution

gwdatafind_server-1.0.1-py3-none-any.whl (24.1 kB view details)

Uploaded Python 3

File details

Details for the file gwdatafind-server-1.0.1.tar.gz.

File metadata

  • Download URL: gwdatafind-server-1.0.1.tar.gz
  • Upload date:
  • Size: 25.5 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/50.3.1 requests-toolbelt/0.9.1 tqdm/4.50.2 CPython/3.8.5

File hashes

Hashes for gwdatafind-server-1.0.1.tar.gz
Algorithm Hash digest
SHA256 fb73ca7f0435550b15096abb7417ae3877a23ac0c015cb69826ea63c226e958d
MD5 21bf00c8e396c3f72affc4cb67fc3428
BLAKE2b-256 e6ab2efa3baf96454fef439ff6c43af3e1091c31cc11e432a63ef785fed5cbee

See more details on using hashes here.

Provenance

File details

Details for the file gwdatafind_server-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: gwdatafind_server-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 24.1 kB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/3.2.0 pkginfo/1.5.0.1 requests/2.22.0 setuptools/50.3.1 requests-toolbelt/0.9.1 tqdm/4.50.2 CPython/3.8.5

File hashes

Hashes for gwdatafind_server-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 28be19670c7050e9f82696055deb269820d3fdfc517e36abf3f69916fad9e61d
MD5 7a408c392764cd7dbb98d6eec16d3966
BLAKE2b-256 ec1c5db676466bfb814bad216f208c5203dccb0ed42482198012ec243f5bdb05

See more details on using hashes here.

Provenance

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page