Cinder Library allows using storage drivers outside of Cinder.
Project description
Introduction
Cinder Library is a Python library that allows using storage drivers outside of Cinder.
Free software: Apache Software License 2.0
Documentation: https://cinderlib.readthedocs.io.
This library is currently in Alpha stage and is primarily intended as a proof of concept at this stage. While some drivers have been manually validated most drivers have not, so there’s a good chance that they could experience issues.
When using this library one should be aware that this is in no way close to the robustness or feature richness that the Cinder project provides, for detailed information on the current limitations please refer to the documentation.
Due to the limited access to Cinder backends and time constraints the list of drivers that have been manually tested are (I’ll try to test more):
LVM with LIO
Dell EMC XtremIO
Dell EMC VMAX
Kaminario K2
Ceph/RBD
NetApp SolidFire
If you try the library with another storage array I would appreciate a note on the library version, Cinder release, and results of your testing.
Features
Use a Cinder driver without running a DBMS, Message broker, or Cinder service.
Using multiple simultaneous drivers on the same program.
Basic operations support:
Create volume
Delete volume
Extend volume
Clone volume
Create snapshot
Delete snapshot
Create volume from snapshot
Connect volume
Disconnect volume
Local attach
Local detach
Validate connector
Code should support multiple concurrent connections to a volume, though this has not yet been tested.
Metadata persistence plugin:
Stateless: Caller stores JSON serialization.
Database: Metadata is stored in a database: MySQL, PostgreSQL, SQLite…
Custom plugin: Metadata is stored in another metadata storage.
Example
The following example uses CentOS 7 and the Cinder LVM driver, which should be the easiest to setup and test.
First you need to setup your system.
The easiest way to set things up is using Vagrant + libvirt using the provided docker example, as it will create a small VM (1GB and 1CPU) and provision everything so we can run a Python interpreter in a cinderlib container:
$ cd examples/docker
$ vagrant up
$ vagrant ssh -c 'sudo docker exec -it cinderlib python'
If we don’t want to use the example we have to setup an LVM VG to use:
$ sudo dd if=/dev/zero of=cinder-volumes bs=1048576 seek=22527 count=1
$ sudo lodevice=`losetup --show -f ./cinder-volumes`
$ sudo vgcreate cinder-volumes $lodevice
$ sudo vgscan --cache
Now we can install everything on baremetal:
$ sudo yum install -y centos-release-openstack-queens $ test -f /etc/yum/vars/contentdir || echo centos >/etc/yum/vars/contentdir $ sudo yum install -y openstack-cinder targetcli python-pip $ sudo pip install cinderlib
Or run it in a container. To be able to run it in a container we need to change our host’s LVM configuration and set udev_rules = 0 and udev_sync = 0 before we start the container:
$ sudo docker run --name=cinderlib --privileged --net=host \
-v /etc/iscsi:/etc/iscsi \
-v /dev:/dev \
-v /etc/lvm:/etc/lvm \
-v /var/lock/lvm:/var/lock/lvm \
-v /lib/modules:/lib/modules:ro \
-v /run:/run \
-v /var/lib/iscsi:/var/lib/iscsi \
-v /etc/localtime:/etc/localtime:ro \
-v /root/cinder:/var/lib/cinder \
-v /sys/kernel/config:/configfs \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
-it akrog/cinderlib:latest python
Or install things on baremetal/VM:
$ sudo yum install -y centos-release-openstack-queens
$ test -f /etc/yum/vars/contentdir || echo centos >/etc/yum/vars/contentdir
$ sudo yum install -y openstack-cinder targetcli python-pip
$ sudo pip install cinderlib
$ sudo dd if=/dev/zero of=cinder-volumes bs=1048576 seek=22527 count=1
$ sudo lodevice=`losetup --show -f ./cinder-volumes`
$ sudo pvcreate $lodevice
$ sudo vgcreate cinder-volumes $lodevice
$ sudo vgscan --cache
Then you need to run python with a passwordless sudo user (required to control LVM and do the attach) and execute:
import cinderlib as cl
from pprint import pprint as pp
# We setup the library to setup the driver configuration when serializing
cl.setup(output_all_backend_info=True)
# Initialize the LVM driver
lvm = cl.Backend(volume_driver='cinder.volume.drivers.lvm.LVMVolumeDriver',
volume_group='cinder-volumes',
target_protocol='iscsi',
target_helper='lioadm',
volume_backend_name='lvm_iscsi')
# Show the LVM backend stats
pp(lvm.stats())
# Create a 1GB volume
vol = lvm.create_volume(1, name='lvm-vol')
# Export, initialize, and do a local attach of the volume
attach = vol.attach()
pp('Volume %s attached to %s' % (vol.id, attach.path))
# Snapshot it
snap = vol.create_snapshot('lvm-snap')
# Show the JSON string
pp(vol.jsons)
# Save the whole environment to a file
with open('cinderlib-test.txt', 'w') as f:
f.write(cl.dumps())
# Exit python
exit()
Now we can check that the logical volume is there, exported, and attached to our system:
# lvdisplay
# targetcli ls
# iscsiadm -m session
# lsblk
And now let’s run a new python interpreter and clean things up:
import cinderlib as cl
# Get the whole environment up
with open('cinderlib-test.txt') as f:
backends = cl.load(f.read(), save=True)
# Get the volume reference we loaded from file and detach
vol = backends[0].volumes[0]
# Volume no longer knows that the attach is local, so we cannot do
# vol.detach(), but we can get the connection and use it.
conn = vol.connections[0]
# Physically detach the volume from the node
conn.detach()
# Unmap the volume and remove the export
conn.disconnect()
# Get the snapshot and delete it
snap = vol.snapshots[0]
snap.delete()
# Finally delete the volume
vol.delete()
We should confirm that the logical volume is no longer there, there’s nothing exported or attached to our system:
# lvdisplay
# targetcli ls
# iscsiadm -m session
# lsblk
History
0.3.6 (2019-02-04)
Bug fixes:
Replace stats workaround with our stats caching mechanism
0.3.5 (2019-02-03)
Bug fixes:
Support MultiOpt configuration parameters
Workaround for Cinder cached stats bug
0.3.4 (2019-01-26)
Features:
RBD volumes in container without RBD installed on host
Removals:
RBD-NBD support was prematurely added, removed in this release
0.3.3 (2019-01-24)
Features:
List drivers available in current Cinder installation.
Support RBD-NBD as well as RBD-KO
0.3.2 (2019-01-22)
Bug fixes:
Failure when the caller has arguments
0.3.1 (2019-01-16)
Bug fixes:
Translation of execute’s OSError exceptions
0.3.0 (2019-01-14)
Bug fixes:
Detach a volume when it’s unavailable.
Features:
Provide better message when device is not available.
Backend name stored in host instead of in the AZ (backward incompatible).
Support multi-pool drivers.
Support QoS
Support extra specs
0.2.2 (2018-07-24)
Features:
Use NOS-Brick to setup OS-Brick for non OpenStack usage.
Can setup persistence directly to use key-value storage.
Support loading objects without configured backend.
Support for Cinder Queens, Rocky, and Master
Serialization returns a compact string
Bug fixes:
Workaround for Python 2 getaddrinfo bug
Compatibility with requests and requests-kerberos
Fix key-value support set_key_value.
Fix get_key_value to return KeyValue.
Fix loading object without configured backend.
0.2.1 (2018-06-14)
Features:
Modify fields on connect method.
Support setting custom root_helper.
Setting default project_id and user_id.
Metadata persistence plugin mechanism
DB persistence plugin
No longer dependent on Cinder’s attach/detach code
Add device_attached method to update volume on attaching node
Support attaching/detaching RBD volumes
Support changing persistence plugin after initialization
Add saving and refreshing object’s metadata
Add dump, dumps methods
Bug fixes:
Serialization of non locally attached connections.
Accept id field set to None on resource creation.
Disabling of sudo command wasn’t working.
Fix volume cloning on XtremIO
Fix iSCSI detach issue related to privsep
Fix wrong size in volume from snapshot
Fix name & description inconsistency
Set created_at field on creation
Connection fields not being set
DeviceUnavailable exception
Multipath settings after persistence retrieval
Fix PyPi package created tests module
Fix connector without multipath info
Always call create_export and remove_export
iSCSI unlinking on disconnect
0.1.0 (2017-11-03)
First release on PyPI.
Project details
Release history Release notifications | RSS feed
Download files
Download the file for your platform. If you're not sure which to choose, learn more about installing packages.
Source Distribution
Built Distribution
Hashes for cinderlib-0.3.6-py2.py3-none-any.whl
Algorithm | Hash digest | |
---|---|---|
SHA256 | 8239c51d915f659bf28102498aca74ff323366145869e0e9168367037b5ec465 |
|
MD5 | ae3ccddaf10f66655e00a54dd86c13b2 |
|
BLAKE2b-256 | 56f74a7d4cb6b58b75ea41de4d8ee12dc0d379477230d163dc69d14bcf62c609 |