cubicweb-narval 4.0.0

Summary

Narval is a CubicWeb based framework to run automated tests. It consists in 2 parts:

• the narval cube which implements the schema and some web UIs to create, configure and run test campaigns, and

• the narval bot which waits for jobs to execute. It polls the CubicWeb application for new tasks to run (called Plans in narval’s jargon), and executes them when some are waiting for exectution.

The narval bot communicates with the web application by doing HTTP(S) requests.

Recipe and Plan

The schema of the cube defines 2 entities:

Recipe:

represents a Python script to be executed to run the tests,

Plan:

represents the execution of a Recipe; it has a workflow (with the following states: ‘ready’, ‘running’, ‘done’, ‘error’, ‘killed’); when executed, the execution log file (stdout and stderr) of the recipe is attached to the Plan (via the execution_log relation).

Execution process of a Recipe

In order to run a Recipe, one must create a Plan (an execution plan).

In the web UI, this can be done via the Start Plan button on a Recipe main view. This creates a new Plan entity (which references the Recipe) in the ready state.

Then, as soon as the narval bot asks the application for waiting jobs (i.e. Plan in the ready workflow state), it eventually gets the Plan eid.

The narval daemon then spawns a new process to manage the execution of the plan in a separate process; the executed command is something like:

narval run-plan narval https://webapp.com/3071/ --uid narval --threads 1 --max-reprieve 1min --log-threshold DEBUG

where 3071 is the eid of the Plan to be executed. The options attribute of the Plan (which is a string of the form: “key1=value1nkey2=value2n[…]”) is converted into command line arguments (–key1 value1 –key2 value2 […]) to be passed to the run-plan command.

This run-plan narval command retrieves the parameters of the Plan (the Python code to be executed and some execution options), then:

• it fires the start workflow transition (by means of a HTTP request),

• it executes the Recipe Python script (by means of an execfile call); the script is executed with a globals (and locals) dictionary defining one variable, plan, which references a Python object having the folowing attributes:

  • cnxh: an HTTP connection handler allowing to make requests to the web application from the executed script,

  • plandata: a dictionary holding the executed Plan parameters,

  • options: a dictionary with all defined options of the Plan,

  • name: the name of the Recipe to be executed,

  • script: a string with the Python script to run.

  Warning: the Python script is executed in the context of the run-plan Python process.

• it fires the transition:

  • end if the execution went fine (did not crash),

  • kill if the execution exceeded some resource limits (memory, execution time),

  • fail if some uncatched Exception has been raised.

When the narval run-plan shell command returns, the bot checks for the return code, retrieves the stdout and stderr of the process and uploads them as the execution_log of the Plan, so the user can have access to the full execution log (print statements, logging messages, etc.).

Plugins

User can write plugins that will be importable from recipes when executed by narval run-plan. The normal way is to write a cubicweb cube in which you add a _narval directory where you can add python modules and packages. When run from source directory, the _narval directory of every available cube will be added in the PYTHONPATH of the narval command.

Please refer to the apycot cube as an example.