node.ext.ldap 0.9.3

LDAP Properties

To define connection properties for LDAP use node.ext.ldap.LDAPProps object:

>>> from node.ext.ldap import LDAPProps
>>> props = LDAPProps(uri='ldap://localhost:12345/',
...                   user='cn=Manager,dc=my-domain,dc=com',
...                   password='secret',
...                   cache=False)

Test server connectivity with node.ext.ldap.testLDAPConnectivity:

>>> from node.ext.ldap import testLDAPConnectivity
>>> testLDAPConnectivity(props=props)
'success'

LDAP Connection

For handling LDAP connections, node.ext.ldap.LDAPConnector is used. It expects a LDAPProps instance in the constructor. Normally there is no need to instantiate this object directly, this happens during creation of higher abstractions, see below:

>>> from node.ext.ldap import LDAPConnector
>>> connector = LDAPConnector(props=props)
>>> connector
<node.ext.ldap.base.LDAPConnector object at ...>

Calling bind creates and returns the LDAP connection:

>>> connector.bind()
<ldap.ldapobject.SimpleLDAPObject instance at ...>

Calling unbind destroys the connection:

>>> connector.unbind()

LDAP Communication

For communicating with an LDAP server, node.ext.ldap.LDAPCommunicator is used. It provides all the basic functions needed to search and modify the directory.

LDAPCommunicator expects a LDAPConnector instance at creation time:

>>> from node.ext.ldap import LDAPCommunicator
>>> communicator = LDAPCommunicator(connector)
>>> communicator
<node.ext.ldap.base.LDAPCommunicator object at ...>

Bind to server:

>>> communicator.bind()

Adding directory entry:

>>> communicator.add(
...     'cn=foo,ou=demo,dc=my-domain,dc=com',
...     {
...         'cn': 'foo',
...         'sn': 'Mustermann',
...         'userPassword': 'secret',
...         'objectClass': ['person'],
...     })

Set default search DN:

>>> communicator.baseDN = 'ou=demo,dc=my-domain,dc=com'

Search in directory:

>>> import node.ext.ldap
>>> communicator.search('(objectClass=person)', node.ext.ldap.SUBTREE)
[('cn=foo,ou=demo,dc=my-domain,dc=com',
{'objectClass': ['person'],
'userPassword': ['secret'],
'cn': ['foo'],
'sn': ['Mustermann']})]

Modify directory entry:

>>> from ldap import MOD_REPLACE
>>> communicator.modify('cn=foo,ou=demo,dc=my-domain,dc=com',
...                     [(MOD_REPLACE, 'sn', 'Musterfrau')])

>>> communicator.search('(objectClass=person)',
...                     node.ext.ldap.SUBTREE,
...                     attrlist=['cn'])
[('cn=foo,ou=demo,dc=my-domain,dc=com',
{'cn': ['foo']})]

Change the password of a directory entry which represents a user:

>>> communicator.passwd(
...     'cn=foo,ou=demo,dc=my-domain,dc=com', 'secret', '12345')

>>> communicator.search('(objectClass=person)',
...                     node.ext.ldap.SUBTREE,
...                     attrlist=['userPassword'])
[('cn=foo,ou=demo,dc=my-domain,dc=com',
{'userPassword': ['{SSHA}...']})]

Delete directory entry:

>>> communicator.delete('cn=foo,ou=demo,dc=my-domain,dc=com')

>>> communicator.search('(objectClass=person)', node.ext.ldap.SUBTREE)
[]

Close connection:

>>> communicator.unbind()

LDAP Session

A more convenient way for dealing with LDAP is provided by node.ext.ldap.LDAPSession. It basically provides the same functionality as LDAPCommunicator, but automatically creates the connectivity objects and checks the connection state before performing actions.

Instantiate LDAPSession object. Expects LDAPProps instance:

>>> from node.ext.ldap import LDAPSession
>>> session = LDAPSession(props)

LDAP session has a convenience to check given properties:

>>> session.checkServerProperties()
(True, 'OK')

Set default search DN for session:

>>> session.baseDN = 'ou=demo,dc=my-domain,dc=com'

Search in directory:

>>> session.search()
[('ou=demo,dc=my-domain,dc=com',
{'objectClass': ['top', 'organizationalUnit'],
'ou': ['demo'],
'description': ['Demo organizational unit']})]

Add directory entry:

>>> session.add(
...     'cn=foo,ou=demo,dc=my-domain,dc=com',
...     {
...         'cn': 'foo',
...         'sn': 'Mustermann',
...         'userPassword': 'secret',
...         'objectClass': ['person'],
...     })

Change the password of a directory entry which represents a user:

>>> session.passwd('cn=foo,ou=demo,dc=my-domain,dc=com', 'secret', '12345')

Authenticate a specific user:

>>> session.authenticate('cn=foo,ou=demo,dc=my-domain,dc=com', '12345')
True

Modify directory entry:

>>> session.modify('cn=foo,ou=demo,dc=my-domain,dc=com',
...                [(MOD_REPLACE, 'sn', 'Musterfrau')])

>>> session.search('(objectClass=person)',
...                node.ext.ldap.SUBTREE,
...                attrlist=['cn'])
[('cn=foo,ou=demo,dc=my-domain,dc=com', {'cn': ['foo']})]

Delete directory entry:

>>> session.delete('cn=foo,ou=demo,dc=my-domain,dc=com')
>>> session.search('(objectClass=person)', node.ext.ldap.SUBTREE)
[]

Close session:

>>> session.unbind()

LDAP Nodes

One can deal with LDAP entries as node objects. Therefor node.ext.ldap.LDAPNode is used. To get a clue of the complete node API, see node package.

Create a LDAP node. The root Node expects the base DN and a LDAPProps instance:

>>> from node.ext.ldap import LDAPNode
>>> root = LDAPNode('ou=demo,dc=my-domain,dc=com', props=props)

Every LDAP node has a DN and a RDN:

>>> root.DN
u'ou=demo,dc=my-domain,dc=com'

>>> root.rdn_attr
u'ou'

Directory entry has no children yet:

>>> root.keys()
[]

Add children to root node:

>>> person = LDAPNode()
>>> person.attrs['objectClass'] = ['person']
>>> person.attrs['sn'] = 'Mustermann'
>>> person.attrs['userPassword'] = 'secret'
>>> root['cn=person1'] = person

>>> person = LDAPNode()
>>> person.attrs['objectClass'] = ['person']
>>> person.attrs['sn'] = 'Musterfrau'
>>> person.attrs['userPassword'] = 'secret'
>>> root['cn=person2'] = person

If the RDN attribute was not set during node creation, it is computed from node key and set automatically:

>>> person.attrs['cn']
u'person2'

Some might fetch children DN’s by key from LDAP node. This only works for existing children:

>>> root.child_dn('cn=person1')
u'cn=person1,ou=demo,dc=my-domain,dc=com'

>>> root.child_dn('cn=person99')
Traceback (most recent call last):
  ...
KeyError: 'cn=person99'

Have a look at the tree:

>>> root.printtree()
<ou=demo,dc=my-domain,dc=com - True>
  <cn=person1,ou=demo,dc=my-domain,dc=com:cn=person1 - True>
  <cn=person2,ou=demo,dc=my-domain,dc=com:cn=person2 - True>

The entries have not been written to the directory yet. When modifying a LDAP node tree, everything happens im memory. Persisting is done by calling the tree, or a part of it. You can check sync state of a node with its changed flag. If changed is True it means either that the node attributes or node children has changed:

>>> root.changed
True

>>> root()
>>> root.changed
False

Modify a LDAP node:

>>> person = root['cn=person1']

Modify existing attribute:

>>> person.attrs['sn'] = 'Mustermensch'

Add new attribute:

>>> person.attrs['description'] = 'Mustermensch description'
>>> person()

Delete an attribute:

>>> del person.attrs['description']
>>> person()

Delete LDAP node:

>>> del root['cn=person2']
>>> root()
>>> root.printtree()
<ou=demo,dc=my-domain,dc=com - False>
  <cn=person1,ou=demo,dc=my-domain,dc=com:cn=person1 - False>

Searching LDAP

Add some users and groups we’ll search for:

>>> for i in range(2, 6):
...     node = LDAPNode()
...     node.attrs['objectClass'] = ['person']
...     node.attrs['sn'] = 'Surname %s' % i
...     node.attrs['userPassword'] = 'secret%s' % i
...     node.attrs['description'] = 'group1'
...     root['cn=person%s' % i] = node

>>> node = LDAPNode()
>>> node.attrs['objectClass'] = ['groupOfNames']
>>> node.attrs['member'] = [
...     root.child_dn('cn=person1'),
...     root.child_dn('cn=person2'),
... ]
... node.attrs['description'] = 'IT'
>>> root['cn=group1'] = node

>>> node = LDAPNode()
>>> node.attrs['objectClass'] = ['groupOfNames']
>>> node.attrs['member'] = [
...     root.child_dn('cn=person4'),
...     root.child_dn('cn=person5'),
... ]
>>> root['cn=group2'] = node

>>> root()
>>> root.printtree()
<ou=demo,dc=my-domain,dc=com - False>
  <cn=person1,ou=demo,dc=my-domain,dc=com:cn=person1 - False>
  <cn=person2,ou=demo,dc=my-domain,dc=com:cn=person2 - False>
  <cn=person3,ou=demo,dc=my-domain,dc=com:cn=person3 - False>
  <cn=person4,ou=demo,dc=my-domain,dc=com:cn=person4 - False>
  <cn=person5,ou=demo,dc=my-domain,dc=com:cn=person5 - False>
  <cn=group1,ou=demo,dc=my-domain,dc=com:cn=group1 - False>
  <cn=group2,ou=demo,dc=my-domain,dc=com:cn=group2 - False>

For defining search criteria LDAP filters are used, which can be combined by bool operators ‘&’ and ‘|’:

>>> from node.ext.ldap import LDAPFilter
>>> filter = LDAPFilter('(objectClass=person)')
>>> filter |= LDAPFilter('(objectClass=groupOfNames)')
>>> root.search(queryFilter=filter)
[u'cn=person1',
u'cn=person2',
u'cn=person3',
u'cn=person4',
u'cn=person5',
u'cn=group1',
u'cn=group2']

Define multiple criteria LDAP filter:

>>> from node.ext.ldap import LDAPDictFilter
>>> filter = LDAPDictFilter({'objectClass': ['person'], 'cn': 'person1'})
>>> root.search(queryFilter=filter)
[u'cn=person1']

Define a relation LDAP filter. In this case we build a relation between group ‘cn’ and person ‘description’:

>>> from node.ext.ldap import LDAPRelationFilter
>>> filter = LDAPRelationFilter(root['cn=group1'], 'cn:description')
>>> root.search(queryFilter=filter)
[u'cn=person2',
u'cn=person3',
u'cn=person4',
u'cn=person5']

Different LDAP filter types can be combined:

>>> filter &= LDAPFilter('(cn=person2)')
>>> str(filter)
'(&(description=group1)(cn=person2))'

The following keyword arguments are accepted by LDAPNode.search. If multiple keywords are used, combine search criteria with ‘&’ where appropriate:

queryFilter

Either a LDAP filter instance or a string. If given argument is string type, a LDAPFilter instance is created.

criteria

A dictionary containing search criteria. A LDAPDictFilter instance is created.

attrlist

List of attribute names to return.

relation

Either LDAPRelationFilter instance or a string defining the relation. If given argument is string type, a LDAPRelationFilter instance is created.

relation_node

In combination with relation argument, when given as string, use relation_node instead of self for filter creation.

exact_match

Flag whether 1-length result is expected. Raises an error if empty result or more than one entry found.

or_search

In combination with criteria, this parameter is passed to the creation of LDAPDictFilter controlling whether to combine criteria with ‘&’ or ‘|’.

You can define search defaults on the node which are always considered when calling search on this node. If set, they are always ‘&’ combined with any (optional) passed filters.

Define the default search scope:

>>> from node.ext.ldap import SUBTREE
>>> root.search_scope = SUBTREE

Define default search filter, could be of type LDAPFilter, LDAPDictFilter, LDAPRelationFilter or string:

>>> root.search_filter = LDAPFilter('objectClass=groupOfNames')
>>> root.search()
[u'cn=group1', u'cn=group2']

>>> root.search_filter = None

Define default search criteria as dict:

>>> root.search_criteria = {'objectClass': 'person'}
>>> root.search()
[u'cn=person1',
u'cn=person2',
u'cn=person3',
u'cn=person4',
u'cn=person5']

Define default search relation:

>>> root.search_relation = \
...     LDAPRelationFilter(root['cn=group1'], 'cn:description')
>>> root.search()
[u'cn=person2',
u'cn=person3',
u'cn=person4',
u'cn=person5']

Again, like with the keyword arguments, multiple defined defaults are ‘&’ combined:

# empty result, there are no groups with group 'cn' as 'description'
>>> root.search_criteria = {'objectClass': 'group'}
>>> root.search()
[]

User and Group management

LDAP is often used to manage Authentication, thus node.ext.ldap provides an API for User and Group management. The API follows the contract of node.ext.ugm:

>>> from node.ext.ldap import ONELEVEL
>>> from node.ext.ldap.ugm import (
...     UsersConfig,
...     GroupsConfig,
...     RolesConfig,
...     Ugm,
... )

Instantiate users, groups and roles configuration. They are based on PrincipalsConfig class and expect this settings:

baseDN

Principals container base DN.

attrmap

Principals Attribute map as odict.odict. This object must contain the mapping between reserved keys and the real LDAP attribute, as well as mappings to all accessible attributes for principal nodes if instantiated in strict mode, see below.

scope

Search scope for principals.

queryFilter

Search Query filter for principals

objectClasses

Object classes used for creation of new principals. For some objectClasses default value callbacks are registered, which are used to generate default values for mandatory attributes if not already set on principal vessel node.

defaults

Dict like object containing default values for principal creation. A value could either be static or a callable accepting the principals node and the new principal id as arguments. This defaults take precedence to defaults detected via set object classes.

strict

Define whether all available principal attributes must be declared in attmap, or only reserved ones. Defaults to True.

memberOfSupport

Flag whether to use ‘memberOf’ attribute (AD) or memberOf overlay (openldap) for Group membership resolution where appropriate.

Reserved attrmap keys for Users, Groups and roles:

id

The attribute containing the user id (mandatory).

rdn

The attribute representing the RDN of the node (mandatory) XXX: get rid of, should be detected automatically

Reserved attrmap keys for Users:

login

Alternative login name attribute (optional)

Create config objects:

>>> ucfg = UsersConfig(
...     baseDN='ou=demo,dc=my-domain,dc=com',
...     attrmap={
...         'id': 'cn',
...         'rdn': 'cn',
...         'login': 'sn',
...     },
...     scope=ONELEVEL,
...     queryFilter='(objectClass=person)',
...     objectClasses=['person'],
...     defaults={},
...     strict=False,
... )

>>> gcfg = GroupsConfig(
...     baseDN='ou=demo,dc=my-domain,dc=com',
...     attrmap={
...         'id': 'cn',
...         'rdn': 'cn',
...     },
...     scope=ONELEVEL,
...     queryFilter='(objectClass=groupOfNames)',
...     objectClasses=['groupOfNames'],
...     defaults={},
...     strict=False,
...     memberOfSupport=False,
... )

Roles are represented in LDAP like groups. Note, if groups and roles are mixed up in the same container, make sure that query filter fits. For our demo, different group object classes are used. Anyway, in real world it might be worth considering a seperate container for roles:

>>> rcfg = GroupsConfig(
...     baseDN='ou=demo,dc=my-domain,dc=com',
...     attrmap={
...         'id': 'cn',
...         'rdn': 'cn',
...     },
...     scope=ONELEVEL,
...     queryFilter='(objectClass=groupOfUniqueNames)',
...     objectClasses=['groupOfUniqueNames'],
...     defaults={},
...     strict=False,
... )

Instantiate Ugm object:

>>> ugm = Ugm(props=props, ucfg=ucfg, gcfg=gcfg, rcfg=rcfg)
>>> ugm
<Ugm object 'None' at ...>

The Ugm object has 2 children, the users container and the groups container. The are accessible via node API, but also on users respective groups attribute:

>>> ugm.keys()
['users', 'groups']

>>> ugm.users
<Users object 'users' at ...>

>>> ugm.groups
<Groups object 'groups' at ...>

Fetch user:

>>> user = ugm.users['person1']
>>> user
<User object 'person1' at ...>

User attributes. Reserved keys are available on user attributes:

>>> user.attrs['id']
u'person1'

>>> user.attrs['login']
u'Mustermensch'

‘login’ maps to ‘sn’:

>>> user.attrs['sn']
u'Mustermensch'

>>> user.attrs['login'] = u'Mustermensch1'
>>> user.attrs['sn']
u'Mustermensch1'

>>> user.attrs['description'] = 'Some description'
>>> user()

Check user credentials:

>>> user.authenticate('secret')
True

Change user password:

>>> user.passwd('secret', 'newsecret')
>>> user.authenticate('newsecret')
True

Groups user is member of:

>>> user.groups
[<Group object 'group1' at ...>]

Add new User:

>>> user = ugm.users.create('person99', sn='Person 99')
>>> user()

>>> ugm.users.keys()
[u'person1',
u'person2',
u'person3',
u'person4',
u'person5',
u'person99']

Delete User:

>>> del ugm.users['person99']
>>> ugm.users()
>>> ugm.users.keys()
[u'person1',
u'person2',
u'person3',
u'person4',
u'person5']

Fetch Group:

>>> group = ugm.groups['group1']

Group members:

>>> group.member_ids
[u'person1', u'person2']

>>> group.users
[<User object 'person1' at ...>, <User object 'person2' at ...>]

Add group member:

>>> group.add('person3')
>>> group.member_ids
[u'person1', u'person2', u'person3']

Delete group member:

>>> del group['person3']
>>> group.member_ids
[u'person1', u'person2']

Group attribute manipulation works the same way as on user objects.

Manage roles for users and groups. Roles can be queried, added and removed via ugm or principal object. Fetch a user:

>>> user = ugm.users['person1']

Add role for user via ugm:

>>> ugm.add_role('viewer', user)

Add role for user directly:

>>> user.add_role('editor')

Query roles for user via ugm:

>>> ugm.roles(user)
[u'viewer', u'editor']

Query roles directly:

>>> user.roles
[u'viewer', u'editor']

Call UGM to persist roles:

>>> ugm()

Delete role via ugm:

>>> ugm.remove_role('viewer', user)
>>> user.roles
[u'editor']

Delete role directly:

>>> user.remove_role('editor')
>>> user.roles
[]

Call UGM to persist roles:

>>> ugm()

Same with group. Fetch a group:

>>> group = ugm.groups['group1']

Add roles:

>>> ugm.add_role('viewer', group)
>>> group.add_role('editor')

>>> ugm.roles(group)
[u'viewer', u'editor']

>>> group.roles
[u'viewer', u'editor']

>>> ugm()

Remove roles:

>>> ugm.remove_role('viewer', group)
>>> group.remove_role('editor')
>>> group.roles
[]

>>> ugm()

Character Encoding

LDAP (v3 at least, RFC 2251) uses utf8 string encoding only. LDAPNode does the encoding for you. Consider it a bug, if you receive anything else than unicode from LDAPNode, except attributes configured as binary. The LDAPSession, LDAPConnector and LDAPCommunicator are encoding-neutral, they do no decoding or encoding.

Unicode strings you pass to nodes or sessions are automatically encoded as uft8 for LDAP, except if configured binary. If you feed them ordinary strings they are decoded as utf8 and reencoded as utf8 to make sure they are utf8 or compatible, e.g. ascii.

If you have an LDAP server that does not use utf8, monkey-patch node.ext.ldap._node.CHARACTER_ENCODING.

Caching Support

node.ext.ldap can cache LDAP searches using bda.cache. You need to provide a cache factory utility in you application in order to make caching work. If you don’t, node.ext.ldap falls back to use bda.cache.NullCache, which does not cache anything and is just an API placeholder.

To provide a cache based on Memcached install memcached server and configure it. Then you need to provide the factory utility:

>>> # Dummy registry.
>>> from zope.component import registry
>>> components = registry.Components('comps')

>>> from node.ext.ldap.cache import MemcachedProviderFactory
>>> cache_factory = MemcachedProviderFactory()
>>> components.registerUtility(cache_factory)

In case of multiple memcached backends on various IPs and ports initialization of the factory looks like this:

>>> # Dummy registry.
>>> components = registry.Components('comps')

>>> cache_factory = MemcachedProviderFactory(servers=['10.0.0.10:22122',
...                                                   '10.0.0.11:22322'])
>>> components.registerUtility(cache_factory)

Dependencies

• python-ldap

• smbpasswd

• argparse

• plumber

• node

• node.ext.ugm

• bda.cache

Notes on python-ldap

There are different compile issues on different platforms. If you experience problems with python-ldap, make sure it is available in the python environment you run buildout in, so it won’t be fetched and built by buildout itself.

Test Coverage

Summary of the test coverage report:

lines   cov%   module
  6   100%   node.ext.ldap.__init__
455    99%   node.ext.ldap._node
154    94%   node.ext.ldap.base
 13   100%   node.ext.ldap.cache
 13   100%   node.ext.ldap.events
129   100%   node.ext.ldap.filter
 60   100%   node.ext.ldap.interfaces
 51   100%   node.ext.ldap.properties
 34    97%   node.ext.ldap.schema
  6   100%   node.ext.ldap.scope
 62   100%   node.ext.ldap.session
437    98%   node.ext.ldap.testing.__init__
 28   100%   node.ext.ldap.tests
  1   100%   node.ext.ldap.ugm.__init__
691    96%   node.ext.ldap.ugm._api
 21   100%   node.ext.ldap.ugm.defaults
 35   100%   node.ext.ldap.ugm.posix
 29    96%   node.ext.ldap.ugm.samba
 21   100%   node.ext.ldap.ugm.shadow

0.9.3

• Fix schema to not bind to test BaseDN only and make binding deferred. [jensens, 30-05-2012]

0.9.2

• Remove escape_queries property from node.ext.ldap.properties.LDAPProps. [rnix, 2012-05-18]

• Use zope.interface.implementer instead of zope.interface.implements. [rnix, 2012-05-18]

• Structural object class inetOrgPerson instead of account on posix users and groups related test LDIF’s [rnix, 2012-04-23]

• session no longer magically decodes everything and prevents binary data from being fetched from ldap. LDAP-Node has semantic knowledge to determine binary data LDAP-Node converts all non binary data and all keys to unicode. [jensens, 2012-04-04]

• or_values and or_keys for finer control of filter criteria [iElectric, chaoflow, 2012-03-24]

• support paged searching [iElectric, chaoflow, 2012-03-24]

0.9.1

• added is_multivalued to properties and modified node to use this list instead of the static list. prepare for binary attributes. [jensens, 2012-03-19]

• added schema_info to node. [jensens, 2012-03-19]

• shadowInactive defaults to 0. [rnix, 2012-03-06]

• Introduce expiresAttr and expiresUnit in principals config. Considered in Users.authenticate. [rnix, 2012-02-11]

• Do not throw KeyError if secondary key set but attribute not found on entry. In case, skip entry. [rnix, 2012-02-10]

• Force unicode ids and keys in UGM API. [rnix, 2012-01-23]

• Add unicode support for filters. [rnix, 2012-01-23]

• Add LDAPUsers.id_for_login. [rnix, 2012-01-18]

• Implement memberOf Support for openldap memberof overlay and AD memberOf behavior. [rnix, 2011-11-07]

• Add LDAPProps.escape_queries for ActiveDirectory. [rnix, 2011-11-06]

• Add group object class to member attribute mapping for ActiveDirectory. [rnix, 2011-11-06]

• Make testlayer and testldap more flexible for usage outside this package. [jensens, 2010-09-30]

0.9

• refactor form bda.ldap. [rnix, chaoflow]