This is a short description of the Python library to access the ecFeed online service. For the latest and full documentation of the ecfeed module, please refer to the docstring of the ecfeed module or check the sources directly at github.
The module was developed and tested against Python 3.6. In theory, it should work with earlier versions of Python as well.
Prerequisites:
directory (Linux users) or in the ~/ecfeed/ directory (Windows users).
EcFeed module is hosted at the PyPI repository and can be downloaded with the pip command:
pip install ecfeedThe examples assume that the user has a valid keystore downloaded to the '.ecfeed' folder ('ecfeed' in Windows) in his home directory and the accessed model contains called methods. The methods are available in a welcome model created at registration at the 'ecfeed.com' webpage. If the model is not there, it can be imported from here.
from ecfeed import TestProvider, DataSource, TemplateType
ecfeed = TestProvider(model="XXXX-XXXX-XXXX-XXXX-XXXX") # The model ID.
for line in ecfeed.export_nwise(method='QuickStart.test', n=3, template=TemplateType.Gherkin, coverage=10): # The name of the method.
print(line)Scenario: executing test
Given the value of arg1 is <arg1>
And the value of arg2 is <arg2>
And the value of arg3 is <arg3>
When test is executed
Examples:
| <arg1> | <arg2> | <arg3> |
| 4 | 3 | 3 |
| 1 | 3 | 4 |
| 3 | 4 | 2 |
| 3 | 3 | 1 |
| 3 | 1 | 1 |
| 2 | 1 | 1 |
| 1 | 2 | 3 |
Try and experiment with following:
for line in ecfeed.export_pairwise(method='QuickStart.test', template=TemplateType.XML, coverage=10):
print(line)for line in ecfeed.generate_cartesian(method='QuickStart.test', coverage=40, choices={'arg1':['choice1'], 'arg3':['choice2', 'choice4']}):
print(line)for line in ecfeed.export_random(method='QuickStart.test', length=5, duplicates=True, adaptive=True, template=TemplateType.CSV):
print(line) for line in ecfeed.export_static_suite(method='QuickStart.test', test_suites=['suite1'], template=TemplateType.XML):
print(line)for line in ecfeed.generate_cartesian(method='QuickStart.test', coverage=40, choices={'arg1':['choice1'], 'arg3':['choice2', 'choice4']}):
print(line)Don't hesitate to modify the welcome model. It can be recreated easily and there is no better way to learn than hands-on exercises.
However, have in mind that the ID of every model (including the welcome model) is unique. If you want to copy and paste the example, be sure to update it accordingly.
Pytest is one of most popular testing frameworks for Python, and luckily, it supports parameterized tests. Even more luckily, the format returned by ecFeed's TestProvider generators is directly usable for pydoc tests. And by some crazy coincidence, the util functions in TestProvider class can be used in pydoc's decorator to provide argument names:
class TestedClass:
@pytest.mark.parametrize(ecfeed.method_arg_names(method_name='QuickStart.test'), ecfeed.generate_random(method='QuickStart.test', length=5))
def test_method_1(self, arg1, arg2, arg3):
print('method(' + str(arg1) + ', ' + str(arg2) + ', ' + str(arg3) + ')')To send feedback, you need to have a BASIC account type or be a member of a TEAM.
An example looks as follows:
ecfeed = TestProvider(model='XXXX-XXXX-XXXX-XXXX-XXXX')
method = 'QuickStart.test'
@pytest.mark.parametrize(ecfeed.test_header(method, feedback=True), ecfeed.random(method=method, length=20, feedback=True))
def test_method_1b(arg1, arg2, arg3, test_handle):
assert arg1 < 2, test_handle.add_feedback(False)
test_handle.add_feedback(True)To the 'test_header' method an additional argument, i.e. 'feedback=True', must be added. The same applies to any of the generation methods. Afterwards, in the parameter list, the 'test_handle' class must be included. It consists of one public method, namely 'add_feedback'. The required argument denotes the result of the test, everything else is optional.
test_handle.add_feedback(True, comment='Passed', duration=1000 , custom={'key1': 'value1'})status - The result of the test. duration - The optional execution time in milliseconds. comment - The optional description of the execution. custom - The optional object of custom key-value pairs.
Additionally, to the test generation method one optional argument can be added, namely 'label'. It provides a short description of the generated test suite.
Note, that each test must return a feedback, regardless whether it has passed or failed.
The ecFeed Python module provides connectivity with the ecFeed online test generation service using the class TestProvider. The class needs a keystore generated at the 'ecfeed.com' page to authenticate and identify the user at the generator service.
The 'TestProvider' constructor takes one required and three optional arguments.
None.An example call to construct a TestProvider object can look like this:
import ecfeed
ecfeed = TestProvider(model='XXXX-XXXX-XXXX-XXXX-XXXX')TestProvider provides 11 generator functions to access ecfeed generator service. The function generate contains the actual code doing the call, but it is rather cumbersome in use, so the 10 other functions wrap it and should be used in the code. Nonetheless we will document this function as well. If a function name starts with the prefix generate_, the generator yields tuples of arguments casted to their types in the model. Otherwise (the prefix is export_) the functions yield lines of text, exported by the ecfeed service according to the chosen template. The only required parameter for all the generators is the method parameter which must be a full name of the method used for the generation (full means including full class name).
A convenient way to call the nwise generator.
generate_ is invoked, data is always casted to argument type (regardless of the value of the template). Templates are defined by TemplateType enum (supported values are RAW, CSV, JSON, Gherkin and XML). Check the docstring for ecFeed TemplateType (pydoc.doc(ecfeed.TemplateType)) to check all supported export templates. Default is 'CSV'.choices={'arg1' : [choice1, choice2], 'arg2' : [choice3]}.constraints=['constraint1', 'constraint2'].If a template is not provided, the function yields tuples of values casted to types defined by the signature of the function used for the generation. If a template is provided, the function yields lines of the exported data according to the template.
If the generator service responses with error, the function raises an EcFeedError exception.
Calls nwise with n=2. For people that like being explicit. Uses the same arguments as nwise, excluding 'n'.
Generates all possible combinations of parameters (considering constraints). Uses the same parameters as nwise, except 'n'.
Generates random combinations of method choices.
Downloads generated test cases (does not start the generator).
Some other functions are provided to facilitate using TestProvider directly as data source in test frameworks like pytest.
Verifies if the connection settings (including the keystore) are correct. If something is wrong, an exception is thrown.
Queries generator service for information about the method. Returns a dictionary with following entries:
Returns list of argument names of the method.
Returns list of argument types of the method.
method_info - See method_arg_names. method_name - See method_arg_names.
The generator can also be accessed from the command line, for example:
ecfeed --model XXXX-XXXX-XXXX-XXXX-XXXX --method QuickStart.test --cartesianThese arguments must be always provided when invoking an ecfeed command.
Arguments related to connection and authorization to ecFeed server. In most cases the default option will be fine.
These arguments are valid only with the NWise generator.
These arguments are valid only with the random generator.
These arguments are valid only with the '--static' option.
These arguments are valid with all or only some data sources.