The Regression Test Tool of the Analyzer stores the test data in folder regressiontest within the Builder project. This folder can contain up to five main folders:

FolderContents
.$output  Contains the output data of the latest test run.
log Contains log files of test runs, if a logfile option has been defined (see Logging Test Runs).
reference Contains the reference data of all tests. The Regression Test Tool will compare the reference data from this folder with the actual output data of the latest run stored in .$output.
source Contains the actual test cases.
testsuite Contains the regression test structure, namely: test suites and sub test suites, libraries, options, tests.

In the test definitions, all folders are stored relatively to the Builder project folder. Therefore, the test definitions contain no absolute paths, but path variables:

  • ${project.folder}
  • ${output.folder}, ${library.output.folder}, ${reference.folder}, ${source.folder} (which are all dependent on ${project.folder})

Folder "testsuite"

The sub-folder structure of folder testsuite reflects the structure of the test suites in the Regression Test Tool.

The definitions in the figure on the left lead to the following folder structure of folder testsuite:

  • Build
  • dev
  • qa
    • linux64qa.e2e.ch
    • ws-qa1.e2e.ch
    • e2esun01.e2e.ch
    • win2012qa.e2e.ch
  • user
  • libraries

All folders within the folder structure contain a file testsuite.xml. This file contains properties and options of the test suite, and the actual tests.

<testcases name="PurchaseOrderExample">

Opening tags.
   <property name="source.folder"
             value="${project.folder}\regressiontest/source/libraries/AllPlatforms/PersistentState/PurchaseOr"/>
   <property name="reference.folder"
             value="${project.folder}\regressiontest/reference/libraries/AllPlatforms/PersistentState/PurchaseOr"/>
   <property name="library.output.folder"
             value="${output.folder}\AllPlatforms\PersistentState\PurchaseOr"/>

List of properties defined on the test suite.
   <options type="ServiceCommand">
      <option name="service.command.service" value="PurchaseOrderExample"/>
      <option name="service.command.url.port" value="21212"/>
   </options>
   <options compareType="equal" name="" type="XMLCompare">
      <ignore element="Creation"/>
      <ignore element="Id"/>
      <ignore element="Update"/>
   </options>

List of Options defined on the test suite.
   <testcase name="Start PurchaseOrderExample" type="startservice">
      <options>
         <option directory="false" name="service.test.logfile"
                 value="${library.output.folder}\PurchaseOrderExample\start.log"/>
         <option name="service.test.service" value="PurchaseOrderExample"/>
         <option name="key.option.test.name" value="Start PurchaseOrderExample"/>
      </options>
      <compare>
         <xmlCompare compareType="equal" name="Compare PurchaseOrderExample start.log">
            <reference file="${reference.folder}\PurchaseOrderExample\start.log" zipped="false"/>
            <test file="${library.output.folder}\PurchaseOrderExample\start.log" zipped="false"/>
         </xmlCompare>
      </compare>
   </testcase>

Actual test cases defined within this test suite.
   <include file="queryObjec\testsuite.xml"/>

Include definitions of sub test suites.
</testcases>

Closing tags.

You may edit the testsuite.xml files with a text editor, but

  • you must not forget to reload the changes in the Regression Test Tool
  • you can break the whole test suite, if you corrupt one of the files

Folder "source"

Folder source contains the actual imported test cases, means -  amongst others -  the .request and .testcase files. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Each time you import a new test case to a test suite, the test case files will be added to folder source to a location corresponding to the location within the tree of test suites.

Folder "reference"

Folder reference contains the reference data of the test cases - usually this will be the request.log and response.log files. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Each time you import a new test case to a test suite, the data of the last test case run will be added to folder reference to serve as initial reference data. Each time you click Accept Differences, the reference files in this folder will be updated.

Folder "log"

Folder log contains the log files of test runs, if a logfile option has been defined on a test suite (see Logging Test Runs).

Folder ".$output"

Folder .$output is a temporary folder. It contains the output data of the latest test run. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Having trouble with some tests, it may sometimes be helpful to clean-up the .$output folder and run the tests again.

On this Page:

  • No labels