Integration with Python

Introduction

This is a short description of python library to access ecfeed online service. For the latest and full documentation of the ecfeed module, please refer to the docstring of the ecfeed module or check the sources directly at github.

Requirements

The module was developed and tested against python 3.6. In theory it should work with earlier versions of python as well.

Installation

EcFeed module is hosted at pypi.org and can be simply downloaded with pip3 command:

pip3 install ecfeed

Examples

The examples assume that the user has a valid keystore downloaded to '.ecfeed' folder ('ecfeed' in Windows) in his home directory and the accessed model contains called methods. The method should be available in a welcome model created at registration at ecfeed.com. If the model is not there it can be imported from here.

>>> from ecfeed import TestProvider, DataSource, TemplateType
>>> ecfeed = TestProvider(model='0168-4412-8644-9433-6380') #change model id to your model
>>> for line in ecfeed.export_nwise(method='QuickStart.test', n=3, template=TemplateType.Gherkin, coverage=10):
...		print(line)
...
Scenario: executing test
	Given the value of arg1 is <arg1>
	And the value of arg2 is <arg2>
	And the value of arg3 is <arg3>
	When test is executed

Examples:
| <arg1> | <arg2> | <arg3> | 
|      4 |      3 |      3 | 
|      1 |      3 |      4 | 
|      3 |      4 |      2 | 
|      3 |      3 |      1 | 
|      3 |      1 |      1 | 
|      2 |      1 |      1 | 
|      1 |      2 |      3 |

Try and experiment with following:

for line in ecfeed.export_pairwise(method='QuickStart.test', template=TemplateType.XML, coverage=10):
	print(line)

for line in ecfeed.generate_cartesian(method='QuickStart.test', coverage=40, choices={'arg1':['choice1'], 'arg3':['choice2', 'choice4']}):
	print(line)

for line in ecfeed.export_random(method='QuickStart.test', length=5, duplicates=True, adaptive=True, template=TemplateType.CSV):
	print(line)

for line in ecfeed.export_static_suite(method='QuickStart.test', test_suites=['suite1'], template=TemplateType.XML):
	print(line)

for line in ecfeed.generate_cartesian(method='QuickStart.test', coverage=40, choices={'arg1':['choice1'], 'arg3':['choice2', 'choice4']}):
	print(line)

pytest

Pytest is one of most popular testing frameworks for python, and luckily it supports parameterized tests. Even more luckily, the format returned by ecfeed's TestProvider generators (called with no templates) is directly usable for pydoc tests. And by some crazy coincidence, the util functions in TestProvider class can be used in pydoc's decorator to provide argument names:

class TestedClass:
	@pytest.mark.parametrize(ecfeed.method_arg_names(method_name='QuickStart.test'), ecfeed.generate_random(method='QuickStart.test', length=5))
	def test_method_1(self, arg1, arg2, arg3):
		print('method(' + str(arg1) + ', ' + str(arg2) + ', ' + str(arg3) + ')')

TestProvider class API

The ecfeed python module provides connectivity with the ecFeed online test generation service using the class TestProvider. The class needs a keystore generated at ecfeed.com page to authenticate and identify the user at the gen service.

Constructor

TestProvider constructor takes 4 optional arguments:

genserver - The url to the ecfeed generator service. By default it is gen.ecfeed.com and this should be fine with most cases.
keystore_path - The path to a keystore downloaded from ecfeed.com (the Settings->Security page). The keystore contains user's certificate that is used to identify and authenticate the user at the generator service. Also, it contains generator's public key to validate the generator. By default the constructor looks for the keystore in ~/.ecfeed/security.p12, except for Windows, where the default path is $HOME/ecfeed/security.p12.
password - Keystore password. The default value is 'changeit' and this is the password used to encrypt the keystore downloaded from ecfeed.com, so if it wasn't changed, the default value should be fine.
model - The model id. The model id is a 20 digit number (grouped by 4) that can be found in the My projects page at ecfeed.com under each model. It is also in an url of the model editor page opened on a model. By default it is None.

The gen service url, keystore location and password are constant and can't be changed in object's lifetime. The model id is accessible and mutable at any time. Also, the model id can be provided explicitly to a generation function each time.

An example call to construct a TestProvider object can look like this:

import ecfeed

ecfeed = TestProvider(model='0168-4412-8644-9433-6380')

Generator calls

TestProvider provides 5 generator functions to access ecfeed generator service. The function generate contains the actual code doing the call, but it is rather cumbersome in use, so the additional functions wrap it and should be used in the code. The generators yield tuples of arguments casted to their types in the model. The only required parameter is the method parameter that must represent the full name of the method used for the generation (full means including full class name). If the generator service responses with error, the function raises EcFeedError exception.

generate_nwise(method, **kwargs)

A convenient way to call nwise generator.

method - Full name (including full class path) of the method that will be used for generation. Method parameters are not required. If parameters are not provided, the generator will generate data from the first method it finds with that name.
n - The 'N' in NWise. Default is 2.
coverage - The percent of N-tuples that the generator will try to cover. Default is 100%.
choices - Dictionary. The keys are names of method parameters. The values define list of choices that will be used for these parameters in the generation. If an argument is skipped in the dictionary, all defined choices will be used. For example: choices={'arg1' : [choice1, choice2], 'arg2' : [choice3]}.
constraints - List of constraints used for the generation. If not provided, all constraints will be used. For example: constraints=['constraint1', 'constraint2'].
model - The id of the model used for generation. If not provided, the model set for the TestProvider object will be used.

generate_pairwise(method, **kwargs)

Calls nwise with n=2. For people that like being explicit. Uses the same arguments that nwise, excluding 'n'.

method - See 'generate_nwise'.
coverage - See 'generate_nwise'.
choices - See 'generate_nwise'.
constraints - See 'generate_nwise'.
model - See 'generate_nwise'.

generate_cartesian(method, **kwargs)

Generates all possible combinations of parameters (considering constraints).

method - See 'generate_nwise'.
choices - See 'generate_nwise'.
constraints - See 'generate_nwise'.
model - See 'generate_nwise'.

generate_random(method, **kwargs)

Generates random combinations of method choices.

'method' - See 'generate_nwise'.
'template' - See 'generate_nwise'.
'choices' See 'generate_nwise'.
'constraints' - See 'generate_nwise'.
'model' - See 'generate_nwise'.
'length' - The number of tests to be generated (1 by default).
'duplicates' - If two identical tests are allowed to be generated. If set to false, the generator will stop after all allowed combinations are generated.
'adaptive' - If set to true, the generator will try to provide tests that are farthest (in Hamming distance) from the ones already generated.

generate_static_suite(method, **kwargs)

Downloads generated test cases (does not use the generator).

'method' - See 'generate_nwise'.
'model' - See 'generate_nwise'.
'testSuites' - An array of test case names to be downloaded. Additionally, one string value can be used instead, i.e. "ALL".

Export calls

The generate function accepts one more parameter, namely 'template'. If it is present (and not set to None), it yield lines of text, exported by the ecfeed service according to the chosen template.

Templates are defined by the TemplateType enum, supported values are: CSV, JSON, Gherkin and XML. Check the docstring for details (TemplateType pydoc.doc(ecfeed.TemplateType)) to check all supported export templates. The default is 'CSV'.

For convenience, similarly to the functions described int the previous paragraph, supplementary wrappers exist. The only difference is that they they are extended by the default value of the 'template' parameter.

export_nwise(self, **kwargs)
export_pairwise(self, **kwargs)
export_cartesian(self, **kwargs)
export_random(self, **kwargs)
export_static_suite(self, **kwargs)

Other functions

Some other functions are provided to facilitate using TestProvider directly as data source in test frameworks like pytest.

method_info(method, model=None)

Queries generator service for information about the method. Returns a dictionary with following entries:

'package_name' - The package of the method, e.g. 'com.example'.
'class_name' - Full name of the class, where the method is defined, e.g 'com.example.TestClass'.
'method_name' - Full name of the method. Repeated from the argument.
'args' - A list of tuples containing type and name of arguments, e.g. '[[int, arg1], [String, arg2]]'.

method_arg_names(method_info=None, method_name=None)

Returns list of argument names of the method.

'method_info' - If provided, the method parses this dictionary for names of the method arguments.
'method_name' - If method_info not provided, this function first calls method_info(method_name), and then recursively calls itself with the result.

method_arg_types(self, method_info=None, method_name=None)

Returns list of argument types of the method.

'method_info' - see 'method_arg_names'.

CLI

The generator can also be accessed from the command line, for example:

ecfeed --model 0168-4412-8644-9433-6380 --method QuickStart.test --cartesian

Required arguments

These arguments must be always provided when invoking an ecfeed command.

'--model MODEL' - ID of the accessed model.
'--method METHOD' - Full name of the method used for generation tests. If the model contains only one method with this name, the argument types may be skipped. For example "--method com.test.TestClass.testMethod", or "--method com.test.TestClass.TestMethod(int, String)".
'--pairwise' - Use pairwise generator. Equal to '--nwise -n 2'.
'--nwise' - Use NWise generator.
'--cartesian' - Use cartesian generator.
'--random' - Use random generator.
'--static' - Fetch pre generated tests from the server.

Connection arguments:

Arguments related to connection and authorization to ecFeed server. In most cases the default options will be fine.

'--keystore KEYSTORE' - Path of the keystore file. Default is "~/.ecfeed/security.p12".
'--password PASSWORD' - Password to keystore. Default is "changeit".
'--genserver GENSERVER' - Address of the ecfeed service. Default is "gen.ecfeed.com".

NWise generator arguments

These arguments are valid only with NWise generator

'-n' N - n in nwise.

Random generator arguments

These arguments are valid only with random generator.

'--length LENGTH' - Number of test cases to generate.
'--duplicates' - If used, the same test can appear more than once in the generated suite.
'--adaptive' - If used, the generator will try to generate tests that are furthest possible from already generated once (in Hamming distance).

Static data arguments

These arguments are valid only with --static option.

'--suites SUITES' - list of test suites that will be fetched from the ecFeed service. If skipped, all test suites will be fetched.

Other optional arguments

These arguments are valid with all or only some data sources.

'--template {CSV,XML,Gherkin,JSON}' - Format for generated data. If not used, the data will be generated in CSV format.
'--choices CHOICES' - Map of choices used for generation, for example {'arg1':['c1', 'c2'], 'arg2':['c3', 'abs:c4']}. Skipped arguments will use all defined choices. This argument is ignored for static generator.
'--constraints CONSTRAINTS' - List of constraints used for generation, for example ['constraint1', 'constraint2']. If skipped, all constraints will be used. This argument is ignored for static generator.
'--coverage COVERAGE' - Requested coverage in percent. The generator will stop after the requested percent of n-tuples will be covered. Valid for pairwise, nwise and cartesian generators.
'--output OUTPUT, -o OUTPUT' - Output file. If omitted, the standard output will be used.
'--url' - Show the endpoint URL instead of generating test cases.

Documentation