kuryr-libnetwork 14.0.0

Prerequisites

The necessary components for an operating environment to run Kuryr are:

• Keystone (preferably configured with Keystone v3),

• Neutron (preferably mitaka or newer),

• DB management system such as MySQL or Mariadb (for Neutron and Keystone),

• Neutron agents for the vendor you choose,

• Rabbitmq if the Neutron agents for your vendor require it,

• Docker 1.9+

Building the container

The Dockerfile in the root of this repository can be used to generate a wsgi Kuryr Libnetwork server container with docker build:

docker build -t your_docker_username/libnetwork:latest .

Additionally, you can pull the upstream container:

docker pull kuryr/libnetwork:latest

Note that you can also specify the tag of a stable release for the above command instead of latest.

How to run the container

First we prepare Docker to find the driver:

sudo mkdir -p /usr/lib/docker/plugins/kuryr
sudo curl -o /usr/lib/docker/plugins/kuryr/kuryr.spec \
https://raw.githubusercontent.com/openstack/kuryr-libnetwork/master/etc/kuryr.spec
sudo service docker restart

Then we start the container:

docker run --name kuryr-libnetwork \
  --net=host \
  --cap-add=NET_ADMIN \
  -e SERVICE_USER=admin \
  -e SERVICE_PROJECT_NAME=admin \
  -e SERVICE_PASSWORD=admin \
  -e SERVICE_DOMAIN_NAME=Default \
  -e USER_DOMAIN_NAME=Default \
  -e IDENTITY_URL=http://127.0.0.1:5000/v3 \
  -v /var/log/kuryr:/var/log/kuryr \
  -v /var/run/openvswitch:/var/run/openvswitch \
  kuryr/libnetwork

Where:

• SERVICE_USER, SERVICE_PROJECT_NAME, SERVICE_PASSWORD, SERVICE_DOMAIN_NAME, USER_DOMAIN_NAME are OpenStack credentials

• IDENTITY_URL is the url to the OpenStack Keystone v3 endpoint

• A volume is created so that the logs are available on the host

• NET_ADMIN capabilities are given in order to perform network operations on the host namespace like ovs-vsctl

Other options you can set as ‘-e’ parameters in Docker run:

• CAPABILITY_SCOPE can be “local” or “global”, the latter being for when there is a cluster store plugged into the docker engine.

• LOG_LEVEL for defining, for example, “DEBUG” logging messages.

• PROCESSES for defining how many kuryr processes to use to handle the libnetwork requests.

Note that you will probably have to change the 127.0.0.1 IDENTITY_URL address for the address where your Keystone is running. In this case it is 127.0.0.1 because the example assumes running the container with –net=host on an all in one deployment where Keystone is also binding locally.

Alternatively, if you have an existing kuryr.conf, you can use it for the container:

docker run --name kuryr-libnetwork \
  --net host \
  --cap-add NET_ADMIN \
  -v /etc/kuryr:/etc/kuryr:ro \
  -v /var/log/kuryr:/var/log/kuryr:rw \
  -v /var/run/openvswitch:/var/run/openvswitch:rw \
  kuryr/libnetwork

Install prerequisites

$ sudo pip3 install -r requirements.txt

Installing Kuryr’s libnetwork driver

Running the following will grab the requirements and install kuryr:

$ sudo pip3 install .

Configuring Kuryr

Generate sample config, etc/kuryr.conf.sample, running the following:

$ ./tools/generate_config_file_samples.sh

Rename and copy config file at required path:

$ cp etc/kuryr.conf.sample /etc/kuryr/kuryr.conf

For using Keystone v3, edit the Neutron section in /etc/kuryr/kuryr.conf, replace ADMIN_PASSWORD:

[neutron]
auth_url = http://127.0.0.1:5000/v3/
username = admin
user_domain_name = Default
password = ADMIN_PASSWORD
project_name = service
project_domain_name = Default
auth_type = password

Alternatively, for using Keystone v2, edit the Neutron section in /etc/kuryr/kuryr.conf, replace ADMIN_PASSWORD:

[neutron]
auth_url = http://127.0.0.1:5000/v2.0/
username = admin
password = ADMIN_PASSWORD
project_name = service
auth_type = password

In the same file uncomment the bindir parameter with the path for the Kuryr vif binding executables. For example, if you installed it on Debian or Ubuntu:

[DEFAULT]
bindir = /usr/local/libexec/kuryr

Running Kuryr

Currently, Kuryr utilizes a bash script to start the service. Make sure that you have installed tox before the execution of the following commands:

If SSL needs to be enabled follow this step or skip to next step:

$tox -egenconfig

Add these 3 parameters in generated file[etc/kuryr.conf.sample]:
    ssl_cert_file <Absolute Path for Cert file>
    ssl_key_file <Absolute Path for private key>
    enable_ssl <True or False>

$export SSL_ENABLED=True

Add the path names in [contrib/tls/kuryr.json]:
    InsecureSkipVerify <false/true>
    CAFile: <Absolute Path for CA file>
    CertFile: <Absolute Path for Cert file>
    KeyFile: <Absolute Path for private key>

Placement of cert files:
By default Kuryr places it certs in /var/lib/kuryr/certs directory,
Please make sure that certs are on proper location as mentioned in kuryr.conf

Verification of kuryr.json:
Please make sure that your kuryr.json look similar to below sample
with appropiate paths of certs updated, and remove older .spec files
if any exists.
and https configuration url::
    {
      "Name": "kuryr",
      "Addr": "https://127.0.0.1:23750",
      "TLSConfig": {
        "InsecureSkipVerify": false,
        "CAFile": "/var/lib/kuryr/certs/ca.pem",
        "CertFile": "/var/lib/kuryr/certs/cert.pem",
        "KeyFile": "/var/lib/kuryr/certs/key.pem"
      }
    }

Optional:
For locally generating and testing, please refer to below link:
    http://tech.paulcz.net/2016/01/secure-docker-with-tls/

Run Kuryr Server with the command below. If you have uwsgi installed this command would run Kuryr under it. You can override this behaviour by setting KURYR_USE_UWSGI=False:

$ sudo ./scripts/run_kuryr.sh

After Kuryr starts, please restart your Docker service, e.g.:

$ sudo service docker restart

The bash script creates the following file if it is missing:

• /usr/lib/docker/plugins/kuryr/kuryr.json: Json spec file for libnetwork.

Note the root privilege is required for creating and deleting the veth pairs with pyroute2 to run.

How to build kuryr-libnetwork docker managed pluginv2

Docker Engine’s plugins system allows you to install, start, stop, and remove plugins using Docker Engine for docker 1.13 and older.

Download kuryr-libnetwork source code, and run contrib/docker/v2plugin/v2plugin_rootfs.sh in the top folder of kuryr-libentwork. This script will copy config.json to the top folder and build rootfs.

$ git clone https://opendev.org/openstack/kuryr-libnetwork
$ cd kuryr-libnetwork
$ ./contrib/docker/v2plugin/v2plugin_rootfs.sh
$ docker plugin create kuryr/libnetwork2 ./

How to use kuryr-libnetwork docker managed pluginv2

If user build pluginv2 locally, user need to enable pluginv2.

$ docker plugin enable kuryr/libnetwork2

If user install pluginv2 from docker hub, the pluginv2 will be enabled directly after install.

$ docker plugin install kuryr/libnetwork2

When user create kuryr network, driver name and ipam-driver name are kuryr/libnetwork2:latest

$ docker network create --driver=kuryr/libnetwork2:latest --ipam-driver=kuryr/libnetwork2:latest ...

How to try out nested-containers locally

1. Installing OpenStack running devstack with the desired local.conf file but including the next to make use of OVS-firewall and enabling Trunk Ports:

   [[post-config|/$Q_PLUGIN_CONF_FILE]]
   
   [DEFAULT]
   service_plugins=trunk
   
   [securitygroup]
   firewall_driver=openvswitch

2. Launch a VM with Neutron trunk port. <https://wiki.openstack.org/wiki/Neutron/TrunkPort>

3. Inside the VM install kuryr and kuryr-libnetwork following the normal installation steps (see above steps at Installing Kuryr’s libnetwork driver).

4. Reconfigure kuryr inside the VM to point to the neutron server and to use the vlan driver:

   • Configure /etc/kuryr/kuryr.conf:

     [binding]
     driver = kuryr.lib.binding.drivers.vlan
     link_iface = eth0 # VM vNIC
     
     [neutron]
     auth_url = http://KEYSTONE_SERVER_IP:5000/v3/
     username = admin
     user_domain_name = Default
     password = ADMIN_PASSWORD
     project_name = service
     project_domain_name = Default
     auth_type = password

   • Restart kuryr service inside the VM

Known nested-containers limitations

1. Due to the Neutron Trunk service implementation choice deployments with iptables hybrid security groups driver do not support trunk service.

2. QoS rules are not applied properly on sub-ports due to a Neutron bug, i.e. nested-container port.

Known IPv6 network limitations

Due to the Docker –ipv6 tag bug version 1.12 and 1.13 have problem to create network only with IPv6.