From: Matthias Maier Date: Sun, 30 Aug 2015 02:43:21 +0000 (-0500) Subject: Documentation: Update testsuite.html to mention multiple comparison files X-Git-Tag: v8.4.0-rc2~519^2~5 X-Git-Url: https://gitweb.dealii.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=1af961e1e194f4fa5491bf12984a14054075d89b;p=dealii.git Documentation: Update testsuite.html to mention multiple comparison files --- diff --git a/doc/developers/testsuite.html b/doc/developers/testsuite.html index ab59777e3f..2ef7985f70 100644 --- a/doc/developers/testsuite.html +++ b/doc/developers/testsuite.html @@ -53,6 +53,7 @@
  • Restricting tests to feature configurations
  • Running tests with MPI
  • Tests with binary output
  • +
  • Tests with multiple comparison files
  • Changing condition for success
  • Adding new tests
  • Adding new categories
  • @@ -532,6 +533,7 @@ category/test.mpirun=N.output files for different numbers of MPI processes.

    +

    Tests with binary output

    @@ -546,6 +548,43 @@ category/test.binary.output files.

    + + +

    Tests with multiple comparison files

    +

    + Sometimes it is necessary to provide multiple comparison files for a + single test. Additional comparison files have the same path as the + main comparison file (in this case test.output) followed + by a dot and a variant description: +

    +category/test.output
    +category/test.output.2
    +category/test.output.3
    +category/test.output.4
    +
    + The testsuite will try to match the output against all variants in + alphabetical order starting with the main output file. +

    + +

    + Warning: This mechanism is only meant as a last resort for + tests where all no alternative approach is viable. Especially, + consider first to +

      +
    1. make the test more robust such that differences can be + expressed in round-off errors detectable by numdiff. +
    2. restrict comparison files to specific feature versions + versions of an external feature. +
    +

    + +

    + Note: The main comparison file (i.e., the one ending in + output is mandatory. Otherwise, no test will be + configured. +

    + +

    Changing condition for success

    @@ -561,7 +600,6 @@ category/test.expect=run.output

    -

    Adding new tests