From: Timo Heister Date: Fri, 13 Oct 2017 19:38:00 +0000 (-0400) Subject: update testsuite documentation X-Git-Tag: v9.0.0-rc1~922^2 X-Git-Url: https://gitweb.dealii.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=refs%2Fpull%2F5239%2Fhead;p=dealii.git update testsuite documentation --- diff --git a/cmake/macros/macro_deal_ii_pickup_tests.cmake b/cmake/macros/macro_deal_ii_pickup_tests.cmake index 581cc97fa1..1ce3748519 100644 --- a/cmake/macros/macro_deal_ii_pickup_tests.cmake +++ b/cmake/macros/macro_deal_ii_pickup_tests.cmake @@ -34,8 +34,11 @@ # - specifying a test target to be executed for a parameter run. # # TEST_TIME_LIMIT -# - specifying the maximal wall clock time in seconds a test is -# allowed to run +# - Specifies the maximal wall clock time in seconds a test is +# allowed to run. Defaults to 600. +# TEST_PICKUP_REGEX +# - A regular expression to select only a subset of tests during setup. +# An empty string is interpreted as a catchall (this is the default). # # Either numdiff (if available), or diff are used for the comparison of # test results. Their location can be specified with NUMDIFF_DIR and diff --git a/doc/developers/testsuite.html b/doc/developers/testsuite.html index 6b18642dfc..7d8c625570 100644 --- a/doc/developers/testsuite.html +++ b/doc/developers/testsuite.html @@ -15,12 +15,13 @@

The deal.II Testsuite

- deal.II has a testsuite that has, at the time this article was last - updated (April 2015), some 3,600 small programs (growing by roughly - one per day) that we run every time we make a change to make sure - that no existing functionality is broken. The expected output for - every test is stored in an *.output, and when you run a - test you are notified if a test produces different output. + deal.II has a testsuite with thousands of small programs that we run every time + we make a change to make sure + that no existing functionality is broken. It is located in the tests/ + directory of the development repository. The expected output for + every test is stored in an *.output next to the .cc file, + and when running a test, you are notified if the output differs from the expected + output.

@@ -28,9 +29,9 @@ we add at least one new test to the testsuite, and we also do so if we fix a bug, in both cases to make sure that future changes do not break this functionality (again). Machines running the testsuite - submit the results to - a webpage showing the status of our regression tests. + submit the results to our cdash + CDash instance, + a webpage showing the status of our regression tests.

@@ -765,9 +766,8 @@ ctest -D<variable>=<value> [...] 
 CTEST_SOURCE_DIRECTORY
   - The source directory of deal.II
-  - If unspecified, "../deal.II" and "../../" relative to the location
-    of this script is used. If this is not a source directory, an error
-    thrown.
+  - If unspecified, "../" relative to the location of this script is
+    used. If this is not a source directory, an error is thrown.
 
 CTEST_BINARY_DIRECTORY
   - The designated build directory (already configured, empty, or non
@@ -779,12 +779,12 @@ CTEST_BINARY_DIRECTORY
 CTEST_CMAKE_GENERATOR
   - The CMake Generator to use (e.g. "Unix Makefiles", or "Ninja", see
     $ man cmake)
-  - If unspecified the current generator of a configured build directory
-    will be used, otherwise "Unix Makefiles".
+  - If unspecified the generator of a configured build directory will
+    be used, otherwise "Unix Makefiles".
 
 TRACK
-  - The track the test should be submitted to. Defaults to "Experimental".
-    Possible values are:
+  - The track the test should be submitted to. Defaults to
+    "Experimental". Possible values are:
 
     "Experimental"     - all tests that are not specifically "build" or
                          "regression" tests should go into this track
@@ -793,13 +793,12 @@ TRACK
                          directory (without actually running the
                          testsuite)
 
-    "Nightly"          - Reserved for nightly regression tests for
-                         build bots on various architectures
+    "Regression Tests" - Reserved for the "official" regression tester
 
-    "Regression Tests" - Reserved for the regression tester
+    "Continuous"       - Reserved for the "official" regression tester
 
 CONFIG_FILE
-  - A configuration file (see docs/users/config.sample)
+  - A configuration file (see doc/users/config.sample)
     that will be used during the configuration stage (invokes
     $ cmake -C ${CONFIG_FILE}). This only has an effect if
     CTEST_BINARY_DIRECTORY is empty.
@@ -809,15 +808,20 @@ DESCRIPTION
 
 COVERAGE
   - If set to ON deal.II will be configured with
-    DEAL_II_SETUP_COVERAGE=ON, CMAKE_BUILD_TYPE=Debug and the
-    CTEST_COVERAGE() stage will be run. Test results must go into the
-    "Experimental" section.
+  DEAL_II_SETUP_COVERAGE=ON, CMAKE_BUILD_TYPE=Debug and the
+  CTEST_COVERAGE() stage will be run. Test results must go into the
+  "Experimental" section.
 
 MAKEOPTS
   - Additional options that will be passed directly to make (or ninja).
- Furthermore, the variables described above can also be - set and will be handed automatically down to cmake. + + Furthermore, the variables TEST_TIME_LIMIT and TEST_PICKUP_REGEX (as + described above), DIFF_DIR, and NUMDIFF_DIR + can also be set and will be handed automatically down + to cmake. For more details on the different tracks, see + the + Testing Infrastructure Wiki page.

diff --git a/tests/run_testsuite.cmake b/tests/run_testsuite.cmake index 0b420faf7d..dbf2fe1143 100644 --- a/tests/run_testsuite.cmake +++ b/tests/run_testsuite.cmake @@ -48,8 +48,8 @@ # CTEST_CMAKE_GENERATOR # - The CMake Generator to use (e.g. "Unix Makefiles", or "Ninja", see # $ man cmake) -# - If unspecified the generator of a configured build directory will -# be used, otherwise "Unix Makefiles". +# - If unspecified the current generator of a configured build directory +# will be used, otherwise "Unix Makefiles". # # TRACK # - The track the test should be submitted to. Defaults to @@ -67,7 +67,7 @@ # "Continuous" - Reserved for the "official" regression tester # # CONFIG_FILE -# - A configuration file (see ../docs/development/Config.sample) +# - A configuration file (see ../doc/users/config.sample) # that will be used during the configuration stage (invokes # # cmake -C ${CONFIG_FILE}). This only has an effect if # CTEST_BINARY_DIRECTORY is empty. @@ -76,8 +76,8 @@ # - A string that is appended to CTEST_BUILD_NAME # # COVERAGE -# - If set to TRUE deal.II will be configured with -# DEAL_II_SETUP_COVERAGE=TRUE, CMAKE_BUILD_TYPE=Debug and the +# - If set to ON deal.II will be configured with +# DEAL_II_SETUP_COVERAGE=ON, CMAKE_BUILD_TYPE=Debug and the # CTEST_COVERAGE() stage will be run. Test results must go into the # "Experimental" section. #