robotframework-datadriver 1.10.0

Excel Support

For file support of xls or xlsx file you need to install the extra XLS or the dependencies. It contains the dependencies of pandas, numpy and xlrd. Just add [XLS] to your installation. New since version 3.6.

pip install --upgrade robotframework-datadriver[XLS]

Python 2

or if you have Python 2 and 3 installed in parallel you may use

pip3 install --upgrade robotframework-datadriver

DataDriver is compatible with Python 2.7 only in Version 0.2.7.

pip install --upgrade robotframework-datadriver==0.2.7

Because Python 2.7 is deprecated, there are no new feature to python 2.7 compatible version.

RoboCon 2020 Talk

Brief overview what DataDriver is and how it works at the RoboCon 2020 in Helsiki.

Alternative approach

DataDriver gives an alternative to the build in data driven approach like:

*** Settings ***
Resource    login_resources.robot

Suite Setup    Open my Browser
Suite Teardown    Close Browsers
Test Setup      Open Login Page
Test Template    Invalid login


*** Test Cases ***       User        Passwort
Right user empty pass    demo        ${EMPTY}
Right user wrong pass    demo        FooBar

Empty user right pass    ${EMPTY}    mode
Empty user empty pass    ${EMPTY}    ${EMPTY}
Empty user wrong pass    ${EMPTY}    FooBar

Wrong user right pass    FooBar      mode
Wrong user empty pass    FooBar      ${EMPTY}
Wrong user wrong pass    FooBar      FooBar

*** Keywords ***
Invalid login
    [Arguments]    ${username}    ${password}
    Input username    ${username}
    Input pwd    ${password}
    click login button
    Error page should be visible

This inbuilt approach is fine for a hand full of data and a hand full of test cases. If you have generated or calculated data and specially if you have a variable amount of test case / combinations these robot files become quite a pain. With DataDriver you may write the same test case syntax but only once and deliver the data from en external data file.

One of the rare reasons when Microsoft® Excel or LibreOffice Calc may be used in testing… ;-)

See example test suite

See example csv table

Requirements

In the Moment there are some requirements how a test suite must be structured so that the DataDriver can get all the information it needs.

• only the first test case will be used as a template. All other test cases will be deleted.

• Test cases have to be defined with a Test Template in Settings secion. Reason for this is, that the DataDriver needs to know the names of the test case arguments. Test cases do not have named arguments. Keywords do.

• The keyword which is used as Test Template must be defined within the test suite (in the same *.robot file). If the keyword which is used as Test Template is defined in a Resource the DataDriver has no access to its arguments names.

Example Test Suite

***Settings***
Library           DataDriver
Resource          login_resources.robot
Suite Setup       Open my Browser
Suite Teardown    Close Browsers
Test Setup        Open Login Page
Test Template     Invalid Login

*** Test Case ***
Login with user ${username} and password ${password}    Default    UserData

***** *Keywords* *****
Invalid login
    [Arguments]    ${username}    ${password}
    Input username    ${username}
    Input pwd    ${password}
    click login button
    Error page should be visible

In this example, the DataDriver is activated by using it as a Library. It is used with default settings. As Test Template the keyword Invalid Login is used. This keyword has two arguments. Argument names are ${username} and ${password}. These names have to be in the CSV file as column header. The test case has two variable names included in its name, which does not have any functionality in Robot Framework®. However, the Data Driver will use the test case name as a template name and replaces the variables with the specific value of the single generated test case. This template test will only be used as a template. The specified data Default and UserData would only be used if no CSV file has been found.

min. required columns

• *** Test Cases *** column has to be the first one.

• Argument columns: For each argument of the Test Template keyword one column must be existing in the data file as data source. The name of this column must match the variable name and syntax.

optional columns

• [Tags] column may be used to add specific tags to a test case. Tags may be comma separated.

• [Documentation] column may be used to add specific test case documentation.

Example Data file

In this data file, eight test cases are defined. Each line specifies one test case. The first two test cases have specific names. The other six test cases will generate names based on template test cases name with the replacement of variables in this name. The order of columns is irrelevant except the first column, *** Test Cases ***

Supported Data Types

In general DataDriver supports any Object that is handed over from the DataReader. However the text based readers for csv, excel and so do support different types as well. DataDriver supports Robot Framework® Scalar variables as well as Dictionaries and Lists. It also support python literal evaluations.

@{DataDriver_DATA_LIST}

A list as suite variable containing a robot dictionary for each test case that is selected for execution.

[
  {
    "test_case_name": "Right user empty pass",
    "arguments": {
      "${username}": "demo",
      "${password}": "${EMPTY}"
    },
    "tags": [
      "1"
    ],
    "documentation": "This is a test case documentation of the first one."
  },
  {
    "test_case_name": "Right user wrong pass",
    "arguments": {
      "${username}": "demo",
      "${password}": "FooBar"
    },
    "tags": [
      "2",
      "3",
      "foo"
    ],
    "documentation": "This test case has the Tags 2,3 and foo"
  },
  {
    "test_case_name": "Login with user '${EMPTY}' and password 'mode'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "mode"
    },
    "tags": [
      "1",
      "2",
      "3",
      "4"
    ],
    "documentation": "This test case has a generated name based on template name."
  },
  {
    "test_case_name": "Login with user '${EMPTY}' and password '${EMPTY}'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "${EMPTY}"
    },
    "tags": [
      ""
    ],
    "documentation": ""
  },
  {
    "test_case_name": "Login with user '${EMPTY}' and password 'FooBar'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "FooBar"
    },
    "tags": [
      ""
    ],
    "documentation": ""
  },
  {
    "test_case_name": "Login with user 'FooBar' and password 'mode'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "mode"
    },
    "tags": [
      "foo",
      "1"
    ],
    "documentation": ""
  },
  {
    "test_case_name": "Login with user 'FooBar' and password '${EMPTY}'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "${EMPTY}"
    },
    "tags": [
      "foo"
    ],
    "documentation": ""
  },
  {
    "test_case_name": "Login with user 'FooBar' and password 'FooBar'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "FooBar"
    },
    "tags": [
      "foo",
      "2"
    ],
    "documentation": ""
  }
]

This can be accessed as usual in Robot Framework®.

${DataDriver_DATA_LIST}[2][arguments][\\${password}] would result in mode .

&{DataDriver_DATA_DICT}

A dictionary as suite variable that contains the same data as the list, with the test names as keys.

{
  "Right user empty pass": {
    "test_case_name": "Right user empty pass",
    "arguments": {
      "${username}": "demo",
      "${password}": "${EMPTY}"
    },
    "tags": [
      "1"
    ],
    "documentation": "This is a test case documentation of the first one."
  },
  "Right user wrong pass": {
    "test_case_name": "Right user wrong pass",
    "arguments": {
      "${username}": "demo",
      "${password}": "FooBar"
    },
    "tags": [
      "2",
      "3",
      "foo"
    ],
    "documentation": "This test case has the Tags 2,3 and foo"
  },
  "Login with user '${EMPTY}' and password 'mode'": {
    "test_case_name": "Login with user '${EMPTY}' and password 'mode'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "mode"
    },
    "tags": [
      "1",
      "2",
      "3",
      "4"
    ],
    "documentation": "This test case has a generated name based on template name."
  },
  "Login with user '${EMPTY}' and password '${EMPTY}'": {
    "test_case_name": "Login with user '${EMPTY}' and password '${EMPTY}'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "${EMPTY}"
    },
    "tags": [
      ""
    ],
    "documentation": ""
  },
  "Login with user '${EMPTY}' and password 'FooBar'": {
    "test_case_name": "Login with user '${EMPTY}' and password 'FooBar'",
    "arguments": {
      "${username}": "${EMPTY}",
      "${password}": "FooBar"
    },
    "tags": [
      ""
    ],
    "documentation": ""
  },
  "Login with user 'FooBar' and password 'mode'": {
    "test_case_name": "Login with user 'FooBar' and password 'mode'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "mode"
    },
    "tags": [
      "foo",
      "1"
    ],
    "documentation": ""
  },
  "Login with user 'FooBar' and password '${EMPTY}'": {
    "test_case_name": "Login with user 'FooBar' and password '${EMPTY}'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "${EMPTY}"
    },
    "tags": [
      "foo"
    ],
    "documentation": ""
  },
  "Login with user 'FooBar' and password 'FooBar'": {
    "test_case_name": "Login with user 'FooBar' and password 'FooBar'",
    "arguments": {
      "${username}": "FooBar",
      "${password}": "FooBar"
    },
    "tags": [
      "foo",
      "2"
    ],
    "documentation": ""
  }
}

&{DataDriver_TEST_DATA}

A dictionary as test variable that contains the test data of the current test case. This dictionary does also contain arguments that are not used in the Test Template keyword. This can be used in Test Setup and within a test case.

{
  "test_case_name": "Right user wrong pass",
  "arguments": {
    "${username}": "demo",
    "${password}": "FooBar"
  },
  "tags": [
    "2",
    "3",
    "foo"
  ],
  "documentation": "This test case has the Tags 2,3 and foo"
}

CSV / TSV (Character-separated values)

By default DataDriver reads csv files. With the Encoding and CSV Dialect settings you may configure which structure your data source has.

XLS / XLSX Files

To use Excel file types, you have to install DataDriver with the Extra XLS.

If you want to use Excel based data sources, you may just set the file to the extention or you may point to the correct file. If the extention is “.xls” or “.xlsx” DataDriver will interpret it as Excel file. You may select the sheet which will be read by the option sheet_name. By default it is set to 0 which will be the first table sheet. You may use sheet index (0 is first sheet) or sheet name(case sensitive). XLS interpreter will ignore all other options like encoding, delimiters etc.

*** Settings ***
Library    DataDriver    .xlsx

or:

*** Settings ***
Library    DataDriver    file=my_data_source.xlsx    sheet_name=2nd Sheet

*** Settings ***
Library    DataDriver    file=my_data_source.xlsx    preserve_xls_types=True

PICT (Pairwise Independent Combinatorial Testing)

Pict is able to generate data files based on a model file. https://github.com/Microsoft/pict

Documentation: https://github.com/Microsoft/pict/blob/master/doc/pict.md

*** Settings ***
Library    DataDriver    my_model_file.pict

*** Settings ***
Library    DataDriver    pict_arg.pict    pict_options=/o:3 /r

Glob File Pattern

This module implements a reader class that creates a test case for each file or folder that matches the given glob pattern.

With an optional argument “arg_name” you can modify the argument that will be set. See folder example.

Example with json files:

*** Settings ***
Library           DataDriver    file=${CURDIR}/DataFiles/*_File.json    reader_class=glob_reader
Library           OperatingSystem
Test Template     Test all Files


*** Test Cases ***
Glob_Reader_Test    Wrong_File.NoJson


*** Keywords ***
Test all Files
    [Arguments]    ${file_name}
    ${file_content}=    Get File    ${file_name}
    ${content}=    Evaluate    json.loads($file_content)["test_case"]
    Should Be Equal    ${TEST_NAME}    ${content}

Example with folders:

*** Settings ***
Library           DataDriver    file=${CURDIR}/FoldersToFind/*/    reader_class=glob_reader    arg_name=\\${folder_name}
Library           OperatingSystem
Test Template     Test all Files


*** Test Cases ***
Glob_Reader_Test    Wrong_File.NoJson


*** Keywords ***
Test all Files
    [Arguments]    ${folder_name}
    ${content}=    Get File    ${folder_name}/verify.txt
    Should Be Equal    ${TEST_NAME}    ${content}

file=

*** Settings ***
Library         DataDriver    file=../data/my_data_source.csv

• None(default): Data Driver will search in the test suites folder if a *.csv file with the same name than the test suite *.robot file exists

• only file extention: if you just set a file extentions like “.xls” or “.xlsx” DataDriver will search

• absolute path: If an absolute path to a file is set, DataDriver tries to find and open the given data file.

• relative path: If the option does not point to a data file as an absolute path, Data Driver tries to find a data file relative to the folder where the test suite is located.

encoding=

encoding= must be set if it shall not be cp1252.

Examples:

cp1252, ascii, iso-8859-1, latin-1, utf_8, utf_16, utf_16_be, utf_16_le

cp1252 is:

• Code Page 1252

• Windows-1252

• Windows Western European

Most characters are same between ISO-8859-1 (Latin-1) except for the code points 128-159 (0x80-0x9F). These Characters are available in cp1252 which are not present in Latin-1.

€ ‚ ƒ „ … † ‡ ˆ ‰ Š ‹ Œ Ž ‘ ’ “ ” • – — ˜ ™ š › œ ž Ÿ

See Python Standard Encoding for more encodings

dialect=

You may change the CSV Dialect here. The dialect option can be one of the following: - Excel-EU - excel - excel-tab - unix - UserDefined

supported Dialects are:

"Excel-EU"
    delimiter=';',
    quotechar='"',
    escapechar='\\',
    doublequote=True,
    skipinitialspace=False,
    lineterminator="\\r\\n",
    quoting=csv.QUOTE_ALL

"excel"
    delimiter = ','
    quotechar = '"'
    doublequote = True
    skipinitialspace = False
    lineterminator = '\\r\\n'
    quoting = QUOTE_MINIMAL

"excel-tab"
    delimiter = '\\t'
    quotechar = '"'
    doublequote = True
    skipinitialspace = False
    lineterminator = '\\r\\n'
    quoting = QUOTE_MINIMAL

"unix"
    delimiter = ','
    quotechar = '"'
    doublequote = True
    skipinitialspace = False
    lineterminator = '\\n'
    quoting = QUOTE_ALL

Usage in Robot Framework®

*** Settings ***
Library    DataDriver    my_data_file.csv    dialect=excel

*** Settings ***
Library    DataDriver    my_data_file.csv    dialect=excel_tab

*** Settings ***
Library    DataDriver    my_data_file.csv    dialect=unix_dialect

*** Settings ***
Library    DataDriver    my_data_file.csv
...    dialect=UserDefined
...    delimiter=.
...    lineterminator=\\n

Defaults:

file=None,
encoding='cp1252',
dialect='Excel-EU',
delimiter=';',
quotechar='"',
escapechar='\\\\',
doublequote=True,
skipinitialspace=False,
lineterminator='\\r\\n',
sheet_name=0

Using Custom DataReader

DataReader classes are loaded dynamically into DataDriver while runtime. DataDriver identifies the DataReader to load by the file extantion of the data file or by the option reader_class.

*** Settings ***
Library    DataDriver    file=mydata.csv

*** Settings ***
    Library    DataDriver   file=mydata.csv    reader_class=generic_csv_reader    dialect=userdefined   delimiter=\\t    encoding=UTF-8

Create Custom Reader

Recommendation:

Have a look to the Source Code of existing DataReader like csv_reader.py or generic_csv_reader.py .

To write your own reader, create a class inherited from AbstractReaderClass.

Your class will get all available configs from DataDriver as an object of ReaderConfig on __init__.

DataDriver will call the method get_data_from_source This method should then load your data from your custom source and stores them into list of object of TestCaseData. This List of TestCaseData will be returned to DataDriver.

AbstractReaderClass has also some optional helper methods that may be useful.

You can either place the custom reader with the others in DataDriver folder or anywhere on the disk. In the first case or if your custom reader is in python path just use it like the others by name:

*** Settings ***
Library          DataDriver    reader_class=my_reader

In case it is somewhere on the disk, it is possible to use an absolute or relative path to a custom Reader. Imports of custom readers follow the same rules like importing Robot Framework® libraries. Path can be relative to ${EXECDIR} or to DataDriver/__init__.py:

*** Settings ***
Library          DataDriver    reader_class=C:/data/my_reader.py    # set custom reader
...                            file_search_strategy=None            # set DataDriver to not check file
...                            min=0                                # kwargs arguments for custom reader
...                            max=62

This my_reader.py should implement a class inherited from AbstractReaderClass that is named my_reader.

from DataDriver.AbstractReaderClass import AbstractReaderClass  # inherit class from AbstractReaderClass
from DataDriver.ReaderConfig import TestCaseData  # return list of TestCaseData to DataDriver


class my_reader(AbstractReaderClass):

    def get_data_from_source(self):  # This method will be called from DataDriver to get the TestCaseData list.
        test_data = []
        for i in range(int(self.kwargs['min']), int(self.kwargs['max'])):  # Dummy code to just generate some data
            args = {'${var_1}': str(i), '${var_2}': str(i)}  # args is a dictionary. Variable name is the key, value is value.
            test_data.append(TestCaseData(f'test {i}', args, ['tag']))  # add a TestCaseData object to the list of tests.
        return test_data  # return the list of TestCaseData to DataDriver

See other readers as example.

Selection of test cases by name

robot --variable "DYNAMICTEST:my suite name.test case to be executed" my_suite_name.robot

robot --variable DYNAMICTESTS:firstsuitename.testcase1|firstsuitename.testcase3|anothersuitename.othertestcase foldername

Re-run failed test cases:

Because it is not possible to use the command line argument --rerunfailed from robot directly, DataDriver brings a Pre-Run-Modifier that handles this issue.

Normally reexecution of failed testcases has three steps.

• original execution

• re-execution the failed ones based on original execution output

• merging original execution output with re-execution output

The DataDriver.rerunfailed Pre-Run-Modifier removes all passed test cases based on a former output.xml.

Example:

robot --output original.xml tests                                                    # first execute all tests
robot --prerunmodifier DataDriver.rerunfailed:original.xml --output rerun.xml tests  # then re-execute failing
rebot --merge original.xml rerun.xml                                                 # finally merge results

Be aware, that in this case it is not allowed to use “:” as character in the original output file path. If you want to set a full path on windows like e:\\myrobottest\\output.xml you have to use “;” as argument seperator.

Example:

robot --prerunmodifier DataDriver.rerunfailed;e:\\myrobottest\\output.xml --output e:\\myrobottest\\rerun.xml tests

Filtering with tags.

New in 0.3.1

It is possible to use tags to filter the data source. To use this, tags must be assigned to the test cases in data source.

*** Settings ***
Library    DataDriver    include=1OR2    exclude=foo