Regression Test File and Folder Structure

The Regression Test Tool of the Analyzer stores the test data in folder regressiontest within the Builder project. This folder can contain up to five main folders:

In the test definitions, all folders are stored relatively to the Builder project folder. Therefore, the test definitions contain no absolute paths, but path variables:

• ${project.folder}

• ${output.folder}, ${library.output.folder}, ${reference.folder}, ${source.folder} (which are all dependent on ${project.folder})

Folder "testsuite"

The sub-folder structure of folder testsuite reflects the structure of the test suites in the Regression Test Tool:

The definitions in the figure on the left lead to the following folder structure of folder testsuite:

• Build

• dev

• qa

  • linux64qa.e2e.ch

  • ws-qa1.e2e.ch

  • e2esun01.e2e.ch

  • win2012qa.e2e.ch

• user

• libraries

All folders within the folder structure contain a file testsuite.xml. This file contains properties and options of the test suite, and the actual tests.

<testcases name="PurchaseOrderExample">

   <property name="source.folder"
             value="${project.folder}\regressiontest/source/libraries/AllPlatforms/PersistentState/PurchaseOr"/>
   <property name="reference.folder"
             value="${project.folder}\regressiontest/reference/libraries/AllPlatforms/PersistentState/PurchaseOr"/>
   <property name="library.output.folder"
             value="${output.folder}\AllPlatforms\PersistentState\PurchaseOr"/>

   <options type="ServiceCommand">
      <option name="service.command.service" value="PurchaseOrderExample"/>
      <option name="service.command.url.port" value="21212"/>
   </options>
   <options compareType="equal" name="" type="XMLCompare">
      <ignore element="Creation"/>
      <ignore element="Id"/>
      <ignore element="Update"/>
   </options>

   <testcase name="Start PurchaseOrderExample" type="startservice">
      <options>
         <option directory="false" name="service.test.logfile"
                 value="${library.output.folder}\PurchaseOrderExample\start.log"/>
         <option name="service.test.service" value="PurchaseOrderExample"/>
         <option name="key.option.test.name" value="Start PurchaseOrderExample"/>
      </options>
      <compare>
         <xmlCompare compareType="equal" name="Compare PurchaseOrderExample start.log">
            <reference file="${reference.folder}\PurchaseOrderExample\start.log" zipped="false"/>
            <test file="${library.output.folder}\PurchaseOrderExample\start.log" zipped="false"/>
         </xmlCompare>
      </compare>
   </testcase>

   <include file="queryObjec\testsuite.xml"/>

</testcases>

You may edit the testsuite.xml files with a text editor, but

• you must not forget to reload the changes in the Regression Test Tool

• you can break the whole test suite, if you corrupt one of the files

Folder "source"

Folder source contains the actual imported test cases, means -  amongst others -  the .request and .testcase files. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Each time you import a new test case to a test suite, the test case files will be added to folder source to a location corresponding to the location within the tree of test suites.

Folder "reference"

Folder reference contains the reference data of the test cases - usually this will be the request.log and response.log files. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Each time you import a new test case to a test suite, the data of the last test case run will be added to folder reference to serve as initial reference data. Each time you click Accept Differences, the reference files in this folder will be updated.

Folder "log"

Folder log contains the log files of test runs, if a logfile option has been defined on a test suite (see Logging Test Runs).

Folder ".$output"

Folder .$output is a temporary folder. It contains the output data of the latest test run. The sub folder structure of this folder corresponds to the test suite structure as defined in the Regression Test Tool.

Having trouble with some tests, it may sometimes be helpful to clean-up the .$output folder and run the tests again.