Skip to main content

Utilities for setting up a Sentry development environment

Project description

devenv

managing dev environments since '24

devenv is an extensible execution framework and library for authoring a simple set of high level commands - bootstrap, sync, doctor, nuke - that manage a repository's dev environment.

prerequisites

Are you a Sentry employee? Make sure your GitHub account has been added to a getsentry/engineering team. If not, open an IT Ticket before continuing.

Otherwise, set the SENTRY_EXTERNAL_CONTRIBUTOR environment variable.

install

Download this and run it:

bash install-devenv.sh

This "global" devenv is installed to ~/.local/share/sentry-devenv/bin/devenv.

To update this installation, run devenv update.

user guide

devenv bootstrap

This is intended for initial setup of a new machine.

devenv fetch [repository name]

Any repository on github in the form of [org]/[reponame]

Repositories are cloned to a "coderoot" directory which is specified in the global configuration.

Note: sentry and ops are currently special names which perform more complicated installations (e.g., sentry will set up both sentry and getsentry)

devenv sync

This runs a user-supplied [reporoot]/devenv/sync.py which should:

  • make sure any desired tools are installed
  • bring the dev environment up-to-date, or create it if it doesn't exist

This script runs within devenv's runtime, which has access to many useful high-level routines. There are currently no api docs, but referring to the examples should get you 90% of the way there.

If you have a feature request, please open an issue!

In general, our library is designed to isolate, as much as possible, a repo's dev environment within [reporoot]/.devenv. For example, gcloud is installed to [reporoot]/.devenv/bin/gcloud (with the gcloud sdk at [reporoot]/.devenv/bin/google-cloud-sdk). An exception to this would be python virtualenvs, which was implemented before the idea of [reporoot]/.devenv.

devenv doctor

When you're inside a repository, this diagnoses and tries to fix common issues. Checks and fixes are defined in [reporoot]/devenv/checks.

devenv nuke|uninstall (wip)

When you're inside a repository, this completely removes the dev environment.

technical overview

Everything devenv needs is in ~/.local/share/sentry-devenv.

  • ~/.local/share/sentry-devenv/bin contains devenv and direnv

runtime

  • devenv is installed exclusively in a virtualenv at ~/.local/share/sentry-devenv/venv
    • this venv exclusively uses a python at ~/.local/share/sentry-devenv/python

global configuration

~/.config/sentry-devenv/config.ini

[devenv]
# the parent directory of all devenv-managed repos
coderoot = ~/code

repository configuration

[reporoot]/devenv/config.ini

[devenv]
# optionally require a minimum version to run sync.py
minimum_version = 1.11.0

There are plenty more sections, their use is best seen in the examples.

examples

Skip to:

direnv

A minimum viable [reporoot]/.envrc is currently needed:

if [[ -f "${PWD}/.env" ]]; then
    dotenv
fi

PATH_add "${HOME}/.local/share/sentry-devenv/bin"

if ! command -v devenv >/dev/null; then
    echo "install devenv: https://github.com/getsentry/devenv#install"
    return 1
fi

PATH_add "${PWD}/.devenv/bin"

python

You can have multiple virtualenvs, which is useful if you rely on a python tool that has a bunch of dependencies that may conflict with others.

[reporoot]/.envrc

export VIRTUAL_ENV="${PWD}/.exampleproject"

PATH_add "${PWD}/.venv-exampleproject/bin"
PATH_add "${PWD}/.venv-inhouse-tool/bin"

[reporoot]/devenv/sync.py

from devenv.lib import config, venv

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]

    for name in ("exampleproject", "inhouse-tool"):
        venv_dir, python_version, requirements, editable_paths, bins = venv.get(reporoot, name)
        url, sha256 = config.get_python(reporoot, python_version)
        print(f"ensuring {name} venv at {venv_dir}...")
        venv.ensure(venv_dir, python_version, url, sha256)

        print(f"syncing {name} with {requirements}...")
        venv.sync(reporoot, venv_dir, requirements, editable_paths, bins)

    return 0

[reporoot]/devenv/config.ini

[venv.exampleproject]
python = 3.12.3
requirements = requirements-dev.txt
editable =
  .

[venv.inhouse-tool]
python = 3.12.3
requirements = inhouse-tool/requirements-dev.txt

[python3.12.3]
darwin_x86_64 = https://github.com/indygreg/python-build-standalone/releases/download/20240415/cpython-3.12.3+20240415-x86_64-apple-darwin-install_only.tar.gz
darwin_x86_64_sha256 = c37a22fca8f57d4471e3708de6d13097668c5f160067f264bb2b18f524c890c8
darwin_arm64 = https://github.com/indygreg/python-build-standalone/releases/download/20240415/cpython-3.12.3+20240415-aarch64-apple-darwin-install_only.tar.gz
darwin_arm64_sha256 = ccc40e5af329ef2af81350db2a88bbd6c17b56676e82d62048c15d548401519e
linux_x86_64 = https://github.com/indygreg/python-build-standalone/releases/download/20240415/cpython-3.12.3+20240415-x86_64-unknown-linux-gnu-install_only.tar.gz
linux_x86_64_sha256 = a73ba777b5d55ca89edef709e6b8521e3f3d4289581f174c8699adfb608d09d6
linux_arm64 = https://github.com/indygreg/python-build-standalone/releases/download/20240415/cpython-3.12.3+20240415-aarch64-unknown-linux-gnu-install_only.tar.gz
linux_arm64_sha256 = ec8126de97945e629cca9aedc80a29c4ae2992c9d69f2655e27ae73906ba187d

node

[reporoot]/.envrc

PATH_add "${PWD}/node_modules/.bin"

[reporoot]/devenv/sync.py

from devenv import constants
from devenv.lib import config, node, proc

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]
    cfg = config.get_repo(reporoot)

    node.install(
        cfg["node"]["version"],
        cfg["node"][constants.SYSTEM_MACHINE],
        cfg["node"][f"{constants.SYSTEM_MACHINE}_sha256"],
        reporoot,
    )
    node.install_yarn(cfg["node"]["yarn_version"], reporoot)

    print("installing node dependencies...")
    proc.run(
        (
            ".devenv/bin/yarn",
            "install",
            "--frozen-lockfile",
            "--no-progress",
            "--non-interactive",
        ),
    )

    return 0

If you'd like a different node version, fill in the appropriate urls https://nodejs.org/dist/ first in config.ini, then reach out to dev-infra and we can mirror it to GCS.

[reporoot]/devenv/config.ini

[node]
# upstream (https://nodejs.org/dist/) is not reliable enough so we've mirrored it to GCS
darwin_x86_64 = https://storage.googleapis.com/sentry-dev-infra-assets/node/node-v20.13.1-darwin-x64.tar.xz
darwin_x86_64_sha256 = c83bffeb4eb793da6cb61a44c422b399048a73d7a9c5eb735d9c7f5b0e8659b6
darwin_arm64 = https://storage.googleapis.com/sentry-dev-infra-assets/node/node-v20.13.1-darwin-arm64.tar.xz
darwin_arm64_sha256 = e8a8e78b91485bc95d20f2aa86201485593685c828ee609245ce21c5680d07ce
linux_x86_64 = https://storage.googleapis.com/sentry-dev-infra-assets/node/node-v20.13.1-linux-x64.tar.xz
linux_x86_64_sha256 = efc0f295dd878e510ab12ea36bbadc3db03c687ab30c07e86c7cdba7eed879a9
# used for autoupdate
version = v20.13.1
yarn_version = 1.22.22

brew

[reporoot]/devenv/sync.py

from devenv import constants
from devenv.lib import brew

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]

    brew.install()

    proc.run(
        (f"{constants.homebrew_bin}/brew", "bundle"),
        cwd=reporoot,
    )

    return 0

[reporoot]/Brewfile

# whatever you want, but we generally discourage installing
# things via brew as it's very difficult to pin a particular
# version of something

colima

[reporoot]/devenv/sync.py

from devenv import constants
from devenv.lib import brew, config, colima, limactl, proc

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]
    cfg = config.get_repo(reporoot)

    brew.install()

    proc.run(
        (f"{constants.homebrew_bin}/brew", "bundle"),
        cwd=reporoot,
    )

    colima.install(
        cfg["colima"]["version"],
        cfg["colima"][constants.SYSTEM_MACHINE],
        cfg["colima"][f"{constants.SYSTEM_MACHINE}_sha256"],
        reporoot,
    )
    limactl.install(reporoot)

    # start colima if it's not already running
    colima.start(reporoot)

    return 0

[reporoot]/Brewfile

# this is docker cli (not desktop) which is needed for interacting with colima
brew 'docker'

# QEMU is needed if you are on an intel mac
# brew 'qemu'

[reporoot]/devenv/config.ini

[colima]
darwin_x86_64 = https://github.com/abiosoft/colima/releases/download/v0.6.6/colima-Darwin-x86_64
darwin_x86_64_sha256 = 84e72678945aacba5805fe363f6c7c87dc73e05cbbfdfc09f9b57cedf110865d
darwin_arm64 = https://github.com/abiosoft/colima/releases/download/v0.6.6/colima-Darwin-arm64
darwin_arm64_sha256 = b2729edcf99470071240ab6986349346211e25944a5dc317bba8fa27ed0f25e5
linux_x86_64 = https://github.com/abiosoft/colima/releases/download/v0.6.6/colima-Linux-x86_64
linux_x86_64_sha256 = bf9e370c4bacbbebdfaa46de04d0e01fe2649a8e366f282cf35ae7dd84559a25
linux_arm64 = https://github.com/abiosoft/colima/releases/download/v0.6.6/colima-Linux-aarch64
linux_arm64_sha256 = 6ecba675e90d154f22e20200fa5684f20ad1495b73c0462f1bd7da4e9d0beaf8
# used for autoupdate
version = v0.6.6

gcloud

[reporoot]/devenv/sync.py

from devenv import constants
from devenv.lib import config, gcloud

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]
    cfg = config.get_repo(reporoot)

    gcloud.install(
        cfg["gcloud"]["version"],
        cfg["gcloud"][SYSTEM_MACHINE],
        cfg["gcloud"][f"{SYSTEM_MACHINE}_sha256"],
        reporoot,
    )

    return 0

[reporoot]/devenv/config.ini

[gcloud]
# custom python version not supported yet, it just uses
# devenv's internal python 3.11
darwin_x86_64 = https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-490.0.0-darwin-x86_64.tar.gz
darwin_x86_64_sha256 = fa396909acc763cf831dd5d89e778999debf37ceadccb3c1bdec606e59ba2694
darwin_arm64 = https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-490.0.0-darwin-arm.tar.gz
darwin_arm64_sha256 = a3a098a5f067b561e003c37284a9b164f28f37fd0d6371bb55e326679f48641c
linux_x86_64 = https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-490.0.0-linux-x86_64.tar.gz
linux_x86_64_sha256 = 40ce41958236f76d9cb08f377ccb9fd6502d2df4da14b36d9214bcb620e2b020
# used for autoupdate
version = 490.0.0

terraform

Our responsibility ends at installing tenv and containing TENV_ROOT at [reporoot]/.devenv/bin/tenv-root. We install terraform and terragrunt shims which use that TENV_ROOT.

Define [reporoot]/.terraform-version and [reporoot]/.terragrunt-version (if you want it) and after running sync, you should be able to just type terraform and tenv takes care of the rest.

[reporoot]/devenv/sync.py

from devenv import constants
from devenv.lib import config, tenv

def main(context: dict[str, str]) -> int:
    reporoot = context["reporoot"]
    cfg = config.get_repo(reporoot)

    tenv.install(
        cfg["tenv"]["version"],
        cfg["tenv"][SYSTEM_MACHINE],
        cfg["tenv"][f"{SYSTEM_MACHINE}_sha256"],
        reporoot,
    )

    return 0

[reporoot]/devenv/config.ini

[tenv]
darwin_x86_64 = https://github.com/tofuutils/tenv/releases/download/v1.3.0/tenv_v1.3.0_Darwin_x86_64.tar.gz
darwin_x86_64_sha256 = 994100d26f4de6de4eebc7691ca4ea7b424e2fd73e6d5d77c5bf6dfd4af94752
darwin_arm64 =  https://github.com/tofuutils/tenv/releases/download/v1.3.0/tenv_v1.3.0_Darwin_arm64.tar.gz
darwin_arm64_sha256 = c31d2b8412147316a0cadb684408bc123e567852d7948091be7e4303fc19397a
# used for autoupdate
version = v1.3.0

develop

We use tox. The easiest way to run devenv locally is just using the tox venv's executable:

~/code/sentry $  ~/code/devenv/.tox/py311/bin/devenv sync

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

sentry_devenv-1.13.0.tar.gz (42.9 kB view details)

Uploaded Source

Built Distribution

sentry_devenv-1.13.0-py3-none-any.whl (58.6 kB view details)

Uploaded Python 3

File details

Details for the file sentry_devenv-1.13.0.tar.gz.

File metadata

  • Download URL: sentry_devenv-1.13.0.tar.gz
  • Upload date:
  • Size: 42.9 kB
  • Tags: Source
  • Uploaded using Trusted Publishing? No
  • Uploaded via: twine/5.0.0 CPython/3.11.2

File hashes

Hashes for sentry_devenv-1.13.0.tar.gz
Algorithm Hash digest
SHA256 4b3818536268a10a26bd5ceb21f65f89b1549e39c437ca150de1e29878c8d737
MD5 7a232ad5f0c9dd5b4ac81b433bd8107f
BLAKE2b-256 e4cc4b6ff296f61ca80374766da8e936c7a20e51b2852995d052b7abf9b836da

See more details on using hashes here.

File details

Details for the file sentry_devenv-1.13.0-py3-none-any.whl.

File metadata

File hashes

Hashes for sentry_devenv-1.13.0-py3-none-any.whl
Algorithm Hash digest
SHA256 8deb4285a565ee634049a7ff4a7e9844e72d5ff8ac576776a4108f4769d4856b
MD5 5b12ad1d47c74284a344f6b274b21ade
BLAKE2b-256 b2b36b2748e14f43a22461a4e6be6e2cb17d3a957fe5f6765aa6d98af046b7cd

See more details on using hashes here.

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page