robotsuite 1.5.0

Setting robot variables from environment variables

Robot Framework supports overriding test variables from command-line, which is not-available when running tests as robotsuite-wrapped with other test runners. That’s why robotsuite supports settings variables as environment variables so that every ROBOT_-prefixed environment variable will be mapped into corresponding test variable without the ROBOT_-prefix.

Declaring tests non-critical by given set of tags

Robot Framework supports declaring tests with given tags as non-critical to prevent their failing to fail the complete build on CI. This is supported as keyword argument for RobotTestSuite as follows:

def test_suite():
    suite = unittest.TestSuite()
    suite.addTests([
        layered(RobotTestSuite('mysuite.txt',
                               noncritical=['non-critical-tag']),
                layer=ACCEPTANCE_TESTING),
    ])
    return suite

Setting zope.testrunner-level

zope.testrunner supports annotating test suites with levels to avoid slow test being run unless wanted:

def test_suite():
    suite = unittest.TestSuite()
    suite.addTests([
        layered(RobotTestSuite('mysuite.txt'),
                layer=ACCEPTANCE_TESTING),
    ])
    suite.level = 10
    return suite

Appending test results to existing test report

When running Robot Framework through robotsuite, its test reports are created into the current working directory with filenames robot_output.xml, robot_log.html and robot_report.html. The default behavior is to override the existing robot_output.xml (and also the other report files generated from that).

To merge test results from separate test runs into the same test report, set environment variable ROBOTSUITE_APPEND_OUTPUT_XML=1 to prevent robotsuite from overriding the existing test results, but to always append to the existing robot_output.xml.

1.5.0 (2014-04-13)

• Restore original behavior to override the exiting robot_output.xml. Set environment variable ROBOTSUITE_APPEND_OUTPUT_XML=1 to append test results to the existing robot_output.xml. [Asko Soukka]

1.4.3 (2014-01-27)

• Fix issue where test suites with sub-suites were not supported [Asko Soukka]

1.4.2 (2013-11-22)

• Fix issue where suite setups were run for suites without tests wit robotframework >= 2.8.0 [Asko Soukka]

1.4.1 (2013-10-13)

• Fix regression in 1.4.0, which dropped relative path names from robot test cases [Asko Soukka]

• Fix RobotTestCase to mimic DocTestCase enough to allow plone.testing to set ‘layer’ for it [Asko Soukka]

• Fix to set given test ‘setUp’ and ‘tearDown’ methods properly as instancemethods (will break existing methods not accepting ‘self’ parameters) [Asko Soukka]

1.4.0 (2013-06-19)

• Supports only roboframework >= 2.8. Use robotsuite 1.3.x with robotframework 1.7.x [Asko Soukka]

1.3.3 (2013-06-05)

• Fix to support custom screenshot filenames with paths [fix #2] [Asko Soukka]

1.3.2 (2013-06-03)

• Remove screenshot capturing from functional test [Asko Soukka]

1.3.1 (2013-06-03)

• Pin robotframework < 2.8rc1, because robotsuite is not yet compatible with robotframework 2.8.x [Asko Soukka]

1.3.0 (2013-04-09)

• Allow passing arguments critical/noncritical to only raise an AssertionError if the test is critical [reinhardt]

1.2.2 (2013-04-08)

• Fix to copy all screenshots (not only selenium-screenshot*.png) when mergin test reports

1.2.1 (2013-03-08)

• Revert setting the default zope.testrunner level and support setting it via environment variable ROBOTSUITE_LEVEL (e.g. ROBOTSUITE_LEVEL=2).

1.2.0 (2013-03-08)

• Override test suite __module__ test case id() to support collective.xmltestreport

• Add support for zope.testrunner test suite levels (with default value of 2 to allow easily to run all but robot tests with zope.testrunner

1.1.0 (2013-02-20)

• Change of behavior to not remove existing robot_output.xml, but always just merge new test data into it. robot_output.xml can be reseted by deleting it manually between tests.

1.0.4 (2013-02-19)

• Fix issue in merging of test reports from tests form two or more separate test suites

1.0.3 (2013-02-18)

• Fix the example test to read ZSERVER_PORT -environment variable properly

1.0.2 (2013-02-09)

• Fix to replace spaces in test report folder names

1.0.1 (2013-02-09)

• Add newline before stdout dump on failing tests

1.0.0 (2013-02-08)

• Stop replacing spaces with underscores in testnames

0.9.0 (2012-09-21)

• Added injecting logged errors into captured stdout to be dumped with robot output when a test fails.

0.8.0 (2012-09-13)

• Added support for setting pybot cli-variables from ROBOT_-prefixed environment variables.

0.7.0 (2012-09-11)

• Implemented custom merge for Robot Framework reports to create better concatenated test report and fully support Robot Framework plugin for Jenkins.

• Fixed to copy (selenium-)screenshots into the root test directory to make it easier to publish them in Jenkins.

0.6.0 (2012-08-21)

• Fixed screenshot paths in concatenated report.

• Fixed test names to include suite filename instead of classname.

• Fixed to allow dash in normalized test names.

0.5.0 (2012-07-23)

• Included robotframework test tags as part of tests’ string representation to allow test be filtered with tags (e.g. with zope.testrunner).

• Added creation of concatenated log file.

0.4.0 (2012-06-04)

• Fixed to rename test method of each test after the actual robot framework test run by the test case.

0.3.1 (2012-06-03)

• Updated README.

0.3.0 (2012-06-03)

• Enabled robotframework reports for tests. Each test suite and each test will have their own outputdir, recursively.

0.2.0 (2012-06-02)

• Support for directories as (recursive) suites.

0.1.0 (2012-05-30)

• Proof of concept.