SearchBestie can be executed by executing cz.vutbr.fit.sbestie.Manager
class.
Usage: Manager [-options] supported options: -config <filename> - XML config file to be used instead of default 'sbestie.xml'. -import <filename> - XML file containing exported results which should be imported before execution. If also -onlyImport is specified results are imported and exploration is not performed.
-config
is not defined, Manager tries to find sbestie.xml file in the CLASSPATH and use it. -import
is defined, Manager creates state space according to configuration file and then fill it with content of specified file. Then, normal exploration is executed if not -onlyImport is used.
Configuration file is an XML file that defines behavior of SearchBestie infrastructure. The configuration file must follow format defined in sbestie.dtd
file (located in the lib
subdirectory by default). Examples of such XML files can be found also in the lib
subdirectory.
The configuration file has the following structure:
<searchbestie> <searchengine name="some name" class="class to be used"> <parameters> <parameter key="name of parameter" value="value of parameter"> </parameters> </searchengine> <executor> <iterationchecker> ... </iterationchecker> <timeout time="number of seconds"> <pretask> ... </pretask> <posttask> ... </posttask> </executor> <statespace addpolicy="chosen policy"> <dimension name="some identifier" ordinal="true or false"> ... </dimension> <dimension name="some identifier" ordinal="true or false"> ... </dimension> ... <testdimension name="some identifier" ordinal="true or false"> <test name="some identifier"> </test> ... </testdimension> <fitness name="some identifier" class="name of class"> </fitness> </statespace> </searchbestie>
The configuration file has the following three main parts:
Controls which state of the state space is explored in the following step. One has to provide class which contains search algorithm and optionally can specify parameters of this file. Currently, only a few search engines are implemented:
Controls exploration process (main loop). executor
part is optional and can contain parameters of executor. Currently, four parameters are available:
iterationchecker
controls number of times the searchengine
is asked for a state to explore. There are multiple iteration checkers available - look bellow. timeout
controls maximal time for which the SearchBestie exploration loop is allowed to run - then interruption mechanism is used to stop exploration. pretask
and posttask
are executed only once during the exploration process. pretask
is executed before exploration loop is entered and can be used for instance to prepare testing, posttask
is executed after exploration.Defines state space including testdimension that contains tests to be used for exploration of a given state. State space contains n dimensions of various type and one testdimension.
Statespace has one parameter that determines how state space behaves when a new result is being added to a state that has been already explored. Several methods are available:
Each dimension can be set to be ordinal or not. Dimensions can be of the following types:
There are several building blocks available for easier development of tests (or experimenting with them). You can combine these building blocks in order to get complex tests:
Fitness is represented by a class that defines the computation. Currently, two fitness functions are available:
Gives decision whether to continue in exploration. It can be used either in executor (to control number of execution of the exploration loop) or in repeated test (to control number of times the state should be explored). Only a few are available right now:
Task is similarly to test a unit of computation used during exploration process. Tasks can be executed before/after exploration process or before/after execution of the test. Tasks can use integrated event subsytem to send an extracted information to the generator of results (e.g., setting some variable, informing about a result etc.). Tasks can be used to prepare testing environment or to clean after test evaluation. Similarly to javatest, there are a few general implementations of task available:
When executed, SearchBestie informs you about initialized state space and its setting. Then, the exploration loop is executed end standard output could be used by exploration algorithms. Finally, SearchBestie prints out statistics concerning explored state space and export the state space into sbestie_output output directory. A simple example of output follows:
SBestie INFO: Search-Based Testing Environment - SearchBestie SBestie INFO: version: 0.4 Beta, any sugestions and comments: iletko@fit.vutbr.cz SBestie INFO: Reading input file with parameters... SBestie INFO: Found 1 search engine. SBestie INFO: Found 4 dimensions. SBestie INFO: Found 1 test dimension. SBestie INFO: Initalizing state space storage module... SBestie INFO: Created integer dimension 'dimension1' (1,100) ordinal: true SBestie INFO: Created integer dimension 'dimension2' (1,100) ordinal: true SBestie INFO: Created integer dimension 'dimension3' (1,100) ordinal: true SBestie INFO: Created integer dimension 'dimension4' (1,100) ordinal: true SBestie INFO: Cteated test dimension 'testdimension' (1 tests) ordinal: false SBestie INFO: Results AddPolicy is set to ADDTOSET. SBestie INFO: ResultItemCreator 'resItem' created. SBestie INFO: Fitness function 'MyFitness' created. SBestie INFO: Initializing search module... SBestie INFO: HillClimbingSearch: Using seed 123456789. SBestie INFO: HillClimbingSearch: First-choice hill climbing: false. SBestie INFO: HillClimbingSearch: Neighborhood size: 1. SBestie INFO: HillClimbingSearch: Complex neighborhood: false. SBestie INFO: HillClimbingSearch: Performing hill climbing dimension by dimension: false. SBestie INFO: Created search engine 'hill climbing'. SBestie INFO: Initializing exploration module... SBestie INFO: Exploration timeout set to 60 seconds. SBestie INFO: Entering state space exploration loop... ...output produced during exploration... SBestie INFO: State space exploration loop exited... SBestie INFO: State space statistics: SBestie INFO: States(Total/Explored): 100000000/1076 SBestie INFO: Results(Total/Pass/Interrupted/Fail): 1076/1076/0/0 SBestie INFO: Best (State: Value): [99, 99, 99, 99, 0]: {401.0;PASS;L358;I401}: 1 SBestie INFO: Worst (State: Value): [65, 0, 83, 43, 0]: {196.0;PASS;L365;I196}: 1 SBestie INFO: Exporting results... SBestie INFO: Creating file: '/somewhere/sbestie_output/sbestie_1292598162.xml' SBestie INFO: Finalizing... SBestie INFO: Exiting...
Currently, SearchBestie does not provide logging into file or any debug level options.
Exported XML file contains definition of state space and collected results. The state space additionally contains information what kind of results were used. Currently, exported state space does not contain exact information concerning test dimension (each test is replaced by an Empty test with the same identifier as original test). The exported results can be then imported by -import
command line parameter.