pulp-cookbook 0.1.0b4

Install pulpcore

Follow the installation instructions provided with pulpcore.

Users should install from either PyPI or source.

Install pulp_cookbook from source

sudo -u pulp -i
source ~/pulpvenv/bin/activate
git clone https://github.com/gmbnomis/pulp_cookbook.git
cd pulp_cookbook
pip install -e .

Install pulp-cookbook From PyPI

sudo -u pulp -i
source ~/pulpvenv/bin/activate
pip install pulp-cookbook

Make and Run Migrations

export DJANGO_SETTINGS_MODULE=pulpcore.app.settings
django-admin migrate pulp_cookbook

Run Services

django-admin runserver 24817
gunicorn pulpcore.content:server --bind 'localhost:24816' --worker-class 'aiohttp.GunicornWebWorker' -w 2
sudo systemctl restart pulpcore-resource-manager
sudo systemctl restart pulpcore-worker@1
sudo systemctl restart pulpcore-worker@2

Create a repository foo

$ http POST http://localhost:24817/pulp/api/v3/repositories/cookbook/cookbook/ name=foo

{
    "description": null,
    "latest_version_href": null,
    "name": "foo",
    "plugin_managed": false,
    "pulp_created": "2019-11-10T18:48:15.746461Z",
    "pulp_href": "/pulp/api/v3/repositories/cookbook/cookbook/8f8228d4-0d76-477e-800c-61077c0dd888/",
    "versions_href": "/pulp/api/v3/repositories/cookbook/cookbook/8f8228d4-0d76-477e-800c-61077c0dd888/versions/"
}

$ export REPO_HREF=$(http :24817/pulp/api/v3/repositories/cookbook/cookbook/ | jq -r '.results[] | select(.name == "foo") | .pulp_href')

Upload cookbooks to Pulp

As a simple example, let’s download two cookbooks from the Chef Supermarket and upload them into our repository.

Download ‘ubuntu’ and ‘apt’ cookbooks (the ‘ubuntu’ cookbooks depends on the ‘apt’ cookbook):

$ curl -Lo ubuntu-2.0.1.tgz https://supermarket.chef.io:443/api/v1/cookbooks/ubuntu/versions/2.0.1/download $ curl -Lo apt-7.0.0.tgz https://supermarket.chef.io:443/api/v1/cookbooks/apt/versions/7.0.0/download

Create a content unit for ubuntu 2.0.1:

$ http --form POST http://localhost:24817/pulp/api/v3/content/cookbook/cookbooks/ name="ubuntu" file@ubuntu-2.0.1.tgz

$ export UBUNTU_CONTENT_HREF=$(http :24817/pulp/api/v3/content/cookbook/cookbooks/?name=ubuntu | jq -r '.results[0].pulp_href')

Create a content unit for apt 7.0.0:

$ http --form POST http://localhost:24817/pulp/api/v3/content/cookbook/cookbooks/ name="apt" file@apt-7.0.0.tgz

$ export APT_CONTENT_HREF=$(http :24817/pulp/api/v3/content/cookbook/cookbooks/?name=apt | jq -r '.results[0].pulp_href')

Add content to repository foo

$ http POST :24817$REPO_HREF'modify/' add_content_units:="[\"$UBUNTU_CONTENT_HREF\",\"$APT_CONTENT_HREF\"]"

$ export LATEST_VERSION_HREF=$(http :24817$REPO_HREF | jq -r '.latest_version_href')

Create a Publication

$ http POST http://localhost:24817/pulp/api/v3/publications/cookbook/cookbook/ repository_version=$LATEST_VERSION_HREF

{
    "task": "/pulp/api/v3/tasks/cd37e3dd-fb9b-4fa3-a32b-174bcb860c79/"
}

$ export PUBLICATION_HREF=$(http :24817/pulp/api/v3/publications/cookbook/cookbook/ | jq --arg LVH "$LATEST_VERSION_HREF" -r '.results[] | select(.repository_version == $LVH) | .pulp_href')

Create a Distribution at ‘foo’ for the Publication

$ http POST http://localhost:24817/pulp/api/v3/distributions/cookbook/cookbook/ name='baz' base_path='foo' publication=$PUBLICATION_HREF

You can have a look at the published “universe” metadata now:

$ http http://localhost:24816/pulp_cookbook/content/foo/universe

{
    "apt": {
        "7.0.0": {
            "dependencies": {},
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/apt/7_0_0/apt-7.0.0.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/apt/7_0_0/apt-7.0.0.tar.gz",
            "location_type": "uri"
        }
    },
    "ubuntu": {
        "2.0.1": {
            "dependencies": {
                "apt": ">= 0.0.0"
            },
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/ubuntu/2_0_1/ubuntu-2.0.1.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/ubuntu/2_0_1/ubuntu-2.0.1.tar.gz",
            "location_type": "uri"
        }
    }
}

Use Berkshelf with the published repo

Create a Berksfile with the following content:

source 'http://localhost:24816/pulp_cookbook/content/foo/'

cookbook 'ubuntu'

$ berks install

Resolving cookbook dependencies...
Fetching cookbook index from http://localhost:24816/pulp_cookbook/content/foo/...
Installing apt (7.0.0) from http://localhost:24816/pulp_cookbook/content/foo/ ([uri] http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/apt/7_0_0/apt-7.0.0.tar.gz)
Installing ubuntu (2.0.1) from http://localhost:24816/pulp_cookbook/content/foo/ ([uri] http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/ubuntu/2_0_1/ubuntu-2.0.1.tar.gz)

Create a new remote foo_remote

In addition to uploading content, pulp_cookbook allows to synchronize a repo with an upstream repo (that has to provide a “universe” endpoint).

Let’s mirror the pulp and qpid cookbooks into our existing repo. First, we have to create a remote:

$ http POST http://localhost:24817/pulp/api/v3/remotes/cookbook/cookbook/ name='foo_remote' url='https://supermarket.chef.io/' cookbooks:='{"pulp": "", "qpid": ""}'

{
    "cookbooks": {
        "pulp": "",
        "qpid": ""
    },
    "download_concurrency": 20,
    "name": "foo_remote",
    "policy": "immediate",
    "proxy_url": null,
    "pulp_created": "2019-11-10T19:09:14.593404Z",
    "pulp_href": "/pulp/api/v3/remotes/cookbook/cookbook/6c90b3c5-2dde-4da3-8796-e4c1512b58a2/",
    "pulp_last_updated": "2019-11-10T19:09:14.593425Z",
    "ssl_ca_certificate": null,
    "ssl_client_certificate": null,
    "ssl_client_key": null,
    "ssl_validation": true,
    "url": "https://supermarket.chef.io/"
}

$ export REMOTE_HREF=$(http :24817/pulp/api/v3/remotes/cookbook/cookbook/ | jq -r '.results[] | select(.name == "foo_remote") | .pulp_href')

Sync repository foo using remote foo_remote

We don’t want to delete the apt and ubuntu coobooks imported previously. Therefore, we sync in ‘additive’ mode by setting mirror to false.

$ http POST :24817$REPO_HREF'sync/' remote=$REMOTE_HREF mirror:=false

Look at the new Repository Version created

$ http GET ':24817'$REPO_HREF'versions/2/'

{
    "base_version": null,
    "content_summary": {
        "added": {
            "cookbook.cookbook": {
                "count": 2,
                "href": "/pulp/api/v3/content/cookbook/cookbooks/?repository_version_added=/pulp/api/v3/repositories/cookbook/cookbook/8f8228d4-0d76-477e-800c-61077c0dd888/versions/2/"
            }
        },
        "present": {
            "cookbook.cookbook": {
                "count": 4,
                "href": "/pulp/api/v3/content/cookbook/cookbooks/?repository_version=/pulp/api/v3/repositories/cookbook/cookbook/8f8228d4-0d76-477e-800c-61077c0dd888/versions/2/"
            }
        },
        "removed": {}
    },
    "number": 2,
    "pulp_created": "2019-11-10T19:10:23.055048Z",
    "pulp_href": "/pulp/api/v3/repositories/cookbook/cookbook/8f8228d4-0d76-477e-800c-61077c0dd888/versions/2/"
}

At the time of writing, there was only a single version of the pulp and qpid cookbooks available, respectively. This brings the total count to 4 cookbooks.

Publish the newest version

To publish the version just created, do:

$ http POST http://localhost:24817/pulp/api/v3/publications/cookbook/cookbook/ repository=$REPO_HREF

And update the distribution:

export DISTRIBUTION_HREF=$(http :24817/pulp/api/v3/distributions/cookbook/cookbook/ | jq -r '.results[] | select(.name == "baz") | .pulp_href')
export LATEST_VERSION_HREF=$(http :24817$REPO_HREF | jq -r '.latest_version_href')
export LATEST_PUBLICATION_HREF=$(http :24817/pulp/api/v3/publications/cookbook/cookbook/ | jq --arg LVH "$LATEST_VERSION_HREF" -r '.results[] | select(.repository_version == $LVH) | .pulp_href')
http PATCH :24817$DISTRIBUTION_HREF publication=$LATEST_PUBLICATION_HREF

Now, the universe endpoint http://localhost:24816/pulp_cookbook/content/foo/universe will show the content of the new repo version.

{
    "apt": {
        "7.0.0": {
            "dependencies": {},
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/apt/7_0_0/apt-7.0.0.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/apt/7_0_0/apt-7.0.0.tar.gz",
            "location_type": "uri"
        }
    },
    "pulp": {
        "0.1.1": {
            "dependencies": {
                "qpid": ">= 0.0.0"
            },
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/pulp/0_1_1/pulp-0.1.1.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/pulp/0_1_1/pulp-0.1.1.tar.gz",
            "location_type": "uri"
        }
    },
    "qpid": {
        "0.1.3": {
            "dependencies": {
                "yum": ">= 0.0.0",
                "yum-epel": ">= 0.0.0"
            },
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/qpid/0_1_3/qpid-0.1.3.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/qpid/0_1_3/qpid-0.1.3.tar.gz",
            "location_type": "uri"
        }
    },
    "ubuntu": {
        "2.0.1": {
            "dependencies": {
                "apt": ">= 0.0.0"
            },
            "download_url": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/ubuntu/2_0_1/ubuntu-2.0.1.tar.gz",
            "location_path": "http://localhost:24816/pulp_cookbook/content/foo/cookbook_files/ubuntu/2_0_1/ubuntu-2.0.1.tar.gz",
            "location_type": "uri"
        }
    }
}

Create a repository supermarket

$ http POST http://localhost:24817/pulp/api/v3/repositories/cookbook/cookbook/ name=supermarket

{
    "description": null,
    "latest_version_href": null,
    "name": "supermarket",
    "plugin_managed": false,
    "pulp_created": "2019-11-10T19:14:00.942419Z",
    "pulp_href": "/pulp/api/v3/repositories/cookbook/cookbook/80cdec2c-dcae-42ee-9b4a-2f23ef0ebf44/",
    "versions_href": "/pulp/api/v3/repositories/cookbook/cookbook/80cdec2c-dcae-42ee-9b4a-2f23ef0ebf44/versions/"
}

$ export REPO_HREF=$(http :24817/pulp/api/v3/repositories/cookbook/cookbook/ | jq -r '.results[] | select(.name == "supermarket") | .pulp_href')

Create a new remote supermarket

$ http POST http://localhost:24817/pulp/api/v3/remotes/cookbook/cookbook/ name='supermarket' url='https://supermarket.chef.io/' policy=on_demand

{
    "cookbooks": null,
    "download_concurrency": 20,
    "name": "supermarket",
    "policy": "on_demand",
    "proxy_url": null,
    "pulp_created": "2019-11-10T19:15:05.899367Z",
    "pulp_href": "/pulp/api/v3/remotes/cookbook/cookbook/e1599387-4be3-40f1-83b2-5c42d1769a55/",
    "pulp_last_updated": "2019-11-10T19:15:05.899406Z",
    "ssl_ca_certificate": null,
    "ssl_client_certificate": null,
    "ssl_client_key": null,
    "ssl_validation": true,
    "url": "https://supermarket.chef.io/"
}

$ export REMOTE_HREF=$(http :24817/pulp/api/v3/remotes/cookbook/cookbook/ | jq -r '.results[] | select(.name == "supermarket") | .pulp_href')

Sync repository supermarket using remote supermarket

$ http POST :24817$REPO_HREF'sync/' remote=$REMOTE_HREF mirror:=true

{
    "task": "/pulp/api/v3/tasks/24990466-6602-4f4f-bb59-6d827bd48130/"
}

This will take a while. You can query the task status using the returned URL. In the example above, use http :24817/pulp/api/v3/tasks/24990466-6602-4f4f-bb59-6d827bd48130/ and inspect the “state” field.

Create a Publication

$ export LATEST_VERSION_HREF=$(http :24817$REPO_HREF | jq -r '.latest_version_href')

$ http POST http://localhost:24817/pulp/api/v3/publications/cookbook/cookbook/ repository_version=$LATEST_VERSION_HREF

{
    "task": "/pulp/api/v3/tasks/8e9d3faf-695f-4048-a11a-1a7a65bd2f8e/"
}

Again, this may take some time. When the task is finished, get the URL of the publication:

$ export PUBLICATION_HREF=$(http :24817/pulp/api/v3/publications/cookbook/cookbook/ | jq --arg LVH "$LATEST_VERSION_HREF" -r '.results[] | select(.repository_version == $LVH) | .pulp_href')

Create a Distribution at ‘supermarket’ for the Publication

$ http POST http://localhost:24817/pulp/api/v3/distributions/cookbook/cookbook/ name='supermarket' base_path='supermarket' publication=$PUBLICATION_HREF

You can have a look at the published “universe” metadata now:

$ http localhost:24816/pulp_cookbook/content/supermarket/universe

In your Berksfile you can use the following source to access the Supermarket snapshot:

source 'http://localhost:24816/pulp_cookbook/content/supermarket/'