mutate 0.0.1.dev13

Adding Depends-On to commit message

A lot of times, especially when adding a new module.. Changes made to scribe, will not ensure the CI to work propery until the respective changes to stockpile are merged. In this case, to ensure CI doesn’t work with master branch of stockpile but rather the patchset you submitted to stockpile, You can use the depends-on functionality.

To add Depends-On functionality, please copy the Change-Id of the patchset you submitted to stockpile, and add it to the commit message at the end like this.

Note: Please add it after Change-Id in commit message.

The commit message should look like:

Your commit message

Change-Id: I9bc121f076b8625da88705c9d96bd00117f94c22

Depends-On: {Change-Id of the review submitted to stockpile}

Say for example, you’re working on adding a module to process satellite data, the CI won’t be able to test it because stockpile doesn’t have a satellite collection yet. However, because you’ve a commit thats still yet to be merged like https://review.gerrithub.io/#/c/redhat-performance/stockpile/+/425015/

You can still ensure and verify stockpile-scribe workflow by adding Depends-On to your commit message in scribe, so commit message will look like:

Adding satellite Module to work with stockpile.

Change-Id: some_random_change_id_generated_after_git_review

Depends-On: I66329511b38a558ce61efb7edb4c3be18625b252

Note that the change ID in Depends-On is the same one in https://review.gerrithub.io/#/c/redhat-performance/stockpile/+/425015/

For another example look at: https://review.gerrithub.io/#/c/redhat-performance/scribe/+/425969/

Adding new scribes

Steps to extend scribe to work with a new input-type ‘example1’ would involve:

1. Creating ‘example1.py’ in the ‘mutate/scribes/’ directory. The sample code would look like:

from . import ScribeBaseClass


class Example1(ScribeBaseClass):

    def example1_build_initial_dict(self):
        output_dict = {}
        Example1_data = load_file(self._path)
        # .... some sort of data manipulation
        # .... to build the output_dict
        return output_dict

    def __init__(self, path=None, source_type=None):
        ScribeBaseClass.__init__(self, source_type=source_type, path=path)
        self._dict = self.example1_build_initial_dict()

    def emit_scribe_dict(self):
        return self._dict

Note the following:

1. from . import ScribeBaseClass needs to be present as we are inheriting from the ScribeBaseClass

2. class Example1(ScribeBaseClass) is where inheritance occurs, ensure that ‘(ScribeBaseClass)’ is present when you write the class definition

3. The first letter in classname must be uppercase that’s how factory method is defined.

4. The __init__ function first calls the parent’s __init__ function and passes the default arguments which are path and source_type, however more can be added. and they won’t be needed to passed on to parent class’s __init__ function.

5. emit_scribe_dict is an abstractmethod and thus it needs to be defined in any other class that is written. However the method itself can be changed but it should return the dictionary object as described above.

2. Add the module to choices list in scribe.py at L14, currently it looks like choices=[‘stockpile’], because at the time of creating this documentation only stockpile data could be transcribed using scribe.

Adding new scribe_modules

Steps to extend scribe_modules to work with a new module ‘scribe_module_1’ would involve:

1. Adding a new class ‘scribe_module_1.py’ to directory ‘mutate/scribe_modules’. It’d probably look something like this: ```python

from . import ScribeModuleBaseClass

class Scribe_module_1(ScribeModuleBaseClass):

def __init__(self, input_dict=None, module_name=None, host_name=None,
             input_type=None, scribe_uuid=None):
    ScribeModuleBaseClass.__init__(self, module_name=module_name,
                                   input_dict=input_dict,
                                   host_name=host_name,
                                   input_type=input_type,
                                   scribe_uuid=scribe_uuid)
    if input_dict:
        new_dict = {}
        # ... this is where transformation occurs
        # ... can call other member functions of class
        # ... can set the entities of the class object like
        self.entity_1 = input_dict

# This isn't needed here, as it's how the __iter__ function is defined
# in the parent class and it's not an abstractmethod, so only if you'd
# like to change how __iter__ method should work for your class, you
# should add the following next lines.
# Not recommended, unless you know what you're doing
def __iter__(self):
      # ... your definition of how to make it iterable

```

Note the following important things:

1. from . import ScribeModuleBaseClass needs to be present as we are inheriting from the ScribeModuleBaseClass

2. class Example1(ScribeModuleBaseClass) is where inheritance occurs, ensure that ‘(ScribeModuleBaseClass)’ is present when you write the class.

3. The first letter in classname must be uppercase that’s how factory method is defined.

4. The __init__ function first calls the parent’s __init__ function and passes the default arguments which are module_name, input_dict, host_name, input_type and scribe_uuid. Please note that no more arguments can be passed.

5. setting the new entities should be done inside the __init__ function only, but the user has flexibility of calling another method from either same class or from lib/util.py to do transformation.

2. Add schema for the new class ‘example1.yml’ to the directory ‘mutate/schema’. Scribe currently uses cerberus to validate the iterable produced by the scribe_modules subclass. Please look at http://docs.python-cerberus.org/en/stable/validation-rules.html for more information on how to write the schema for your class’s output.

Note: The name of the yml file should match that of the scribe_modules class that you create it for. Thus, for ‘example1’ class the file should be named ‘example1.yml’