Deploy ephemeral namespaces for cloud.redhat.com app testing
Project description
bonfire
A CLI tool used to deploy ephemeral environments for testing cloud.redhat.com applications
bonfire interacts with a running instance of qontract-server to obtain namespace and application configurations defined in the AppSRE team's internal app-interface repository.
It also interacts with OpenShift to manage the reservation of ephemeral namespaces for testing.
It is meant to be partnered with the Clowder operator to spin up an ephemeral environment for testing.
Installation
pip install -r requirements.txt
pip install .
Deploying applications
The bonfire config get command can be used to print processed app configs to stdout.
The bonfire config deploy command can be used to deploy app configs into a namespace.
Using a local config
To get up and running without needing to contact app-interface's qontract-server, you can utilize
a local config file.
See the (example config)[example_config.yaml] which specifies each app you wish to deploy along with the location for that app's template.
For example, to use this approach to get the config for the ingress app along with all of its
dependencies, you can create your own config.yaml and run:
bonfire config get --app ingress --local-config --get-dependencies
Loading an app's ephemeral config from app-interface
You'll first need to set proper env variables to interface with your instance of qontract-server:
export QONTRACT_BASE_URL="https://myserver/graphql"
export QONTRACT_USERNAME=myUsername
export QONTRACT_PASSWORD=myPassword
If these env vars are not specified, bonfire will attempt to access a local qontract-server (see "Setting up a local qontract-server" below)
You can then use the bonfire config get command to obtain the kubernetes configuration of an application defined according to the app-interface schema.
bonfire will query the qontract GraphQL API and read the desired application's deploy configuration.
bonfire config get relies on a few key pieces of info to process an app's config:
- The application name. This is the name of the
appinapp-interface - a 'src env' -- the name of the
app-interfaceenvironment that it should pull application configs for. An app's config will only be processed if it has a deploy target set up for this environment (default: "ephemeral") - a 'ref env' -- the name of the
app-interfaceenvironment that we want the application's IMAGE_TAG and deploy template to come from. We will use the IMAGE_TAG/template defined on the app's deploy target that matches this environment name. - Any template refs you wish to override -- in other words, if you want to download a different git hash of an application component's template.
- Any image tags you wish to override
- Whether or not you want to dynamically load dependencies that all components of
apprelies on. This requires theappto be using the Clowder operator.
For example, let's say that we are running a PR check against the insights-puptoo service. This service:
- is a member of the
ingressapplication. - the kubernetes deploy manifest for this service resides in the same repo as the code
- every time a PR is opened in this repo, a docker image is built and pushed to
quay.io/myorg/insights-puptoowith the tagpr-<git hash>. The PR opened against the app has commit hashabc1234
If we intend to deploy the ingress application group into namespace mynamespace, using the new template/image of the insights-puptoo PR, but using the production template/image for all other components, we could run:
APP_NAME=ingress
COMPONENT_NAME=insights-puptoo
GIT_COMMIT=pr-abc1234
IMAGE=quay.io/myorg/insights-puptoo
IMAGE_TAG=abc1234
NAMESPACE=mynamespace
bonfire config get \
--ref-env insights-prod \
--app $APP_NAME \
--set-template-ref $COMPONENT_NAME=$GIT_COMMIT \
--set-image-tag $IMAGE=$IMAGE_TAG \
--get-dependencies \
--namespace $NAMESPACE \
> k8s_resources.json
oc apply -f k8s_resources.json -n $NAMESPACE
Running a local qontract-server
For testing/debug purposes, instead of committing changes directly to app-interface, you can run your own local copy of the app-interface API server.
- Clone https://github.com/app-sre/qontract-server
- Clone the internal
app-interfacerepo
In qontract-server, run:
npm install yarn
make bundle APP_INTERFACE_PATH=/path/to/app-interface
LOAD_METHOD=fs DATAFILES_FILE=bundle/bundle.json yarn run server
Namespace reservation
bonfire is also used to reserve, release, and reconcile ephemeral namespaces running on our test OpenShift clusters.
The list of ephemeral namespaces is stored in app-interface.
The service account that bonfire logs in to the cluster with has a custom role bound to it which allows it to edit namespace labels:
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: namespace-editor
rules:
- apiGroups:
- ""
resources:
- namespaces
verbs:
- get
- list
- patch
- update
- watch
This role is bound to the service account in each ephemeral namespace.
Bonfire uses labels to keep track of which namespaces are reserved AND ready. A "ready" namespace is one which has been "wiped clean" and then had a fresh set of base test configurations copied into it.
When a tester is logged in using the proper account, namespace commands can be used such as:
bonfire namespace reserve -- find an available namespace and reserve it. By default the TTL is 1 hr.
bonfire namespace release <namespace> -- release a namespace reservation
Use bonfire namespace -h to see a list of all available namespace commands.
Namespace reconciler
A separate cron job runs the bonfire namespace reconcile command every 2 minutes. This command does the following:
- Checks for any namespaces that are released, but not ready, and "prepares" them by wiping them and copying base test resources into them. After being prepared, the namespace is marked "ready". A namespace is prepared by:
- creating an ephemeral
ClowdEnvironmentresource for it, and - copying any secrets defined in the
ephemeral-basenamespace into it
- creating an ephemeral
- Checks for any namespaces that are reserved, but do not have an "expires" time set on them yet. This would be a newly-reserved namespace. The reconciler is responsible for applying the "expires time"
- Checks the "expires time" on all reserved namespaces. If any have expired, bonfire will release them and re-prepare them.
Interactions with Clowder
-
For every namespace that
bonfireprepares, it creates a ClowderClowdEnvironmentresource following this template. The name of the environment matches this format. So, if bonfire prepared a namespace calledephemeral-01, then the name of theClowdEnvironmentwould beenv-ephemeral-01. -
When
bonfire get configis executed for a certain namespace, it will dynamically populate anENV_NAMEparameter and pass this to all templates it processes. Therefore, all templates that define aClowdAppresource should set theenvironmentmapping in their spec using an${ENV_NAME}parameter. -
When
bonfire namespace wait-on-resourcesis executed, it follows this logic:
- Wait for all resources owned by a 'ClowdEnvironment' to appear in the namespace
- Wait for all the deployments in the namespace to reach 'active' state.
- Wait for resources owned by a 'ClowdApp' to appear in the namespace
- Wait for all the deployments in the namespace to reach 'active' state (deployments we already waited on in step 2 are not waited on again)
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
File details
Details for the file crc-bonfire-0.0.0.tar.gz.
File metadata
- Download URL: crc-bonfire-0.0.0.tar.gz
- Upload date:
- Size: 36.2 kB
- Tags: Source
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.24.0 setuptools/52.0.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | c8c025d765746e6d186e703ec8522e0b1f242340c9f4c7f7265c0cb789accc20 |
|
| MD5 | 2de4aa940988ce9e05de077bf15c42d1 |
|
| BLAKE2b-256 | 438181250fda66dd936ab933f1fde3695eb4570f02b0a80644174479515668c0 |
Provenance
File details
Details for the file crc_bonfire-0.0.0-py3-none-any.whl.
File metadata
- Download URL: crc_bonfire-0.0.0-py3-none-any.whl
- Upload date:
- Size: 27.1 kB
- Tags: Python 3
- Uploaded using Trusted Publishing? No
- Uploaded via: twine/3.3.0 pkginfo/1.7.0 requests/2.24.0 setuptools/52.0.0 requests-toolbelt/0.9.1 tqdm/4.56.0 CPython/3.9.1
File hashes
| Algorithm | Hash digest | |
|---|---|---|
| SHA256 | d4247db3f06b8c9ab1f3ab5f7f83641ccd3a1295030881076f3e5b655e076744 |
|
| MD5 | d726ecc37c6fd23054bc9d617f64f7ed |
|
| BLAKE2b-256 | 9a1026d150538e51032ba2edfa29225f6e9aff3c3b3089247a868733b516905a |
