From: bangerth Date: Thu, 30 Aug 2012 21:18:13 +0000 (+0000) Subject: Remove link to a hopelessly outdated report. X-Git-Url: https://gitweb.dealii.org/cgi-bin/gitweb.cgi?a=commitdiff_plain;h=2c58b64b8e54d8dca61ae2279639620bff8ea6f0;p=dealii-svn.git Remove link to a hopelessly outdated report. git-svn-id: https://svn.dealii.org/trunk@26177 0785d39b-7218-0410-832d-ea1e28bc413d --- diff --git a/deal.II/doc/documentation.html b/deal.II/doc/documentation.html index e2ab216172..7b69415d86 100644 --- a/deal.II/doc/documentation.html +++ b/deal.II/doc/documentation.html @@ -42,7 +42,7 @@

Note that along with the rest of the documentation, the local HTML pages - of the tutorials need to be generated first. Please follow the + of the tutorials need to be generated first. Please follow the instructions in the ReadMe file on how to do this. @@ -132,25 +132,6 @@ Bangerth).

- A report on how - multithreading is implemented and - supported in deal.II (by Wolfgang - Bangerth). This report is also available as preprint - 2000-11 from the - - IWR preprint server. However, this report described a - previous version of the threading scheme. After - deal.II 3.4, this scheme was replaced by - another one that is more flexible and easier to use. While the - general observations of the report are still valid, the syntax - presented there is no longer. There is a short - document describing the new syntax and some considerations we - had in implementing it. -

A brief report on mapping functions of higher polynomial diff --git a/deal.II/doc/reports/multithreading/img1.gif b/deal.II/doc/reports/multithreading/img1.gif deleted file mode 100644 index 4525a58002..0000000000 Binary files a/deal.II/doc/reports/multithreading/img1.gif and /dev/null differ diff --git a/deal.II/doc/reports/multithreading/img2.gif b/deal.II/doc/reports/multithreading/img2.gif deleted file mode 100644 index 28505c1000..0000000000 Binary files a/deal.II/doc/reports/multithreading/img2.gif and /dev/null differ diff --git a/deal.II/doc/reports/multithreading/img3.gif b/deal.II/doc/reports/multithreading/img3.gif deleted file mode 100644 index 58acfe253b..0000000000 Binary files a/deal.II/doc/reports/multithreading/img3.gif and /dev/null differ diff --git a/deal.II/doc/reports/multithreading/img4.gif b/deal.II/doc/reports/multithreading/img4.gif deleted file mode 100644 index 1068152d8f..0000000000 Binary files a/deal.II/doc/reports/multithreading/img4.gif and /dev/null differ diff --git a/deal.II/doc/reports/multithreading/index.html b/deal.II/doc/reports/multithreading/index.html deleted file mode 100644 index 283e3126ba..0000000000 --- a/deal.II/doc/reports/multithreading/index.html +++ /dev/null @@ -1,125 +0,0 @@ - - - - -No Title - - - - - - - - - - - - - - -
- Next: Background -
-
- - -

-Multi-threading support in deal.II - -

-
-
- -

-Wolfgang Bangerth -
-University of Heidelberg -
-
-March 2000 -

- -

Abstract:

-In this report, we describe the implementational techniques of - multi-threading support in deal.II, which we use for the - parallelization of independent operations. Writing threaded programs in - C++ is obstructed by two problems: operating system dependent - interfaces and that these interfaces are created for C programs - rather than for C++. We present our solutions to these problems and - describe first experiences using multi-threading in deal.II. -

- - -

- -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/multithreading.css b/deal.II/doc/reports/multithreading/multithreading.css deleted file mode 100644 index 5a961f9951..0000000000 --- a/deal.II/doc/reports/multithreading/multithreading.css +++ /dev/null @@ -1,56 +0,0 @@ - -body { background-image: none; - background-color: white; - } - -frameset { background-color: white; - border-color: white; - border: 0; - frameborder: 0; - } - -td { vertical-align: top; } - -dt { font-weight: bold; } - -.figure { font-weight: bold; - font-size: larger; - } - -.pagetoc {} - -.chapter_title {} - -span.parhead { font-weight: bold; - } - -span.example { font-weight: bold; - font-style: italic; - } -pre.example { padding-left: 2 em; - text-align: left; - text-indent: 0; - } - -table.navbar { } - - - -/* Century Schoolbook font is very similar to Computer Modern Math: cmmi */ -.MATH { font-family: "Century Schoolbook", serif; } -.MATH I { font-family: "Century Schoolbook", serif; font-weight: bold } -.BOLDMATH { font-family: "Century Schoolbook", serif; font-weight: bold } - -/* implement both fixed-size and relative sizes */ -SMALL.XTINY { font-size : xx-small } -SMALL.TINY { font-size : x-small } -SMALL.SCRIPTSIZE { font-size : smaller } -SMALL.FOOTNOTESIZE { font-size : small } -SMALL.SMALL { } -BIG.LARGE { } -BIG.XLARGE { font-size : large } -BIG.XXLARGE { font-size : x-large } -BIG.HUGE { font-size : larger } -BIG.XHUGE { font-size : xx-large } - -/* document-specific styles come next */ diff --git a/deal.II/doc/reports/multithreading/node1.html b/deal.II/doc/reports/multithreading/node1.html deleted file mode 100644 index 9d0c6456f9..0000000000 --- a/deal.II/doc/reports/multithreading/node1.html +++ /dev/null @@ -1,145 +0,0 @@ - - - - -Background - - - - - - - - - - - - - - -

- -

-
- Next: Threads - Up: No Title - Previous: No Title -
-
- - -

-Background -

- -

-Realistic finite element simulations tend to use enormous amounts of computing -time and memory. Scientists and programmers have therefore long tried to use -the combined power of several processors or computers to tackle these -problems. - -

-The usual approach is to use physically separated computers (e.g. clusters) or -computing units (e.g. processor nodes in a parallel computer), each of which -is equipped with its own memory, and split the problem at hand into separate -parts which are then solved on these computing units. Unfortunately, this -approach tends to pose significant problems, both for the mathematical -formulation as well as for the application programmer, which make the -development of such programs overly difficult and expensive. - -

-For these reasons, parallelized implementations and their -mathematical background are still subject to intense research. In recent -years, however, multi-processor machines have been developed, which pose a -reasonable alternative to small parallel computers with the advantage of -simple programming and the possibility to use the same mathematical -formulation that can also be used for single-processor machines. These -computers typically have between two and eight processors that can access the -global memory at equal cost. - -

-Due to this uniform memory access (UMA) architecture, communication can be -performed in the global memory and is no more costly than access to any other -memory location. Thus, there is also no more need to change the mathematical -formulation to reduce communication, and programs using this architecture look -very much like programs written for single processor machines. - -

-The purpose of this report is to explain the techniques used in -deal.II (see [1,2]) -by which we try to program these computers. We will first -give a brief introduction in what threads are and what the problems are which -we have to solve when we want to use multi-threading. The third section takes an -in-depth look at the way in which the functionality of the operating system is -represented in a C++ program in order to allow simple and robust -programming; in particular, we describe the design decisions which led us to -implement these parts of the library in the way they are implemented. In the -fourth section, we show several examples of parallelization and explain how -they work. Readers who are more interested in actually using the framework -laid out in this report, rather than the internals, may skip Section 3 and go -directly to the applications in Section 4 (page -). - - -

-Note (2003-01-11): In this report we frequently make reference to the -ACE -(Adaptive Communications Environment) library. This library has been used -in previous versions of the library to start and control threads, and to -provide other multithreading features in a cross-platform way. We dropped -support for ACE after version 3.4 of the deal.II library -since we found that using POSIX functions instead is much simpler to -support. One of the main problems with ACE was the complicated installation, -while POSIX functions are provided by most modern system's C libraries. - -

Since the features we used from ACE are limited to the features that POSIX -also provides for multithreading, the impact of this replacement on the -underlying foundations used in this report are minor, and can be ignored. - -

- - -

- -

-
- Next: Threads - Up: No Title - Previous: No Title - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node10.html b/deal.II/doc/reports/multithreading/node10.html deleted file mode 100644 index 49373d991e..0000000000 --- a/deal.II/doc/reports/multithreading/node10.html +++ /dev/null @@ -1,311 +0,0 @@ - - - - -Assembling the matrix - - - - - - - - - - - - - - -

- -

-
- Next: Parallel Jacobi preconditioning - Up: Applications - Previous: Writing output detached to -
-
- - -

-Assembling the matrix -

- -

-Setting up the system matrix is usually done by looping over all cells and -computing the contributions of each cell separately. While the computations of -the local contributions is strictly independent, we need to transfer these -contributions to the global matrix afterward. This transfer has to be -synchronized, in order to avoid that one thread overwrites values that another -thread has just written. - -

-In most cases, building the system matrix in parallel will look like the -following template: -

-    void MainClass::build_matrix () {
-      // define how many threads will be used (here: 4)
-      const unsigned int n_threads = 4;
-      const unsigned int n_cells_per_thread
-             = triangulation.n_active_cells () / n_threads;
-
-      // define the Mutex that will be used to synchronise
-      // accesses to the matrix
-      ACE_Thread_Mutex mutex;
-
-      // define thread manager
-      ACE_Thread_Manager thread_manager;
-
-      vector<DoFHandler<dim>::active_cell_iterator>
-             first_cells (n_threads),
-             end_cells (n_threads);
-
-      DoFHandler<dim>::active_cell_iterator 
-             present_cell = dof_handler.begin_active ();
-      for (unsigned int thread=0; thread<n_threads; ++thread)
-        {
-          // for each thread: first determine the range of cells on
-          // which it shall operate:
-          first_cells[thread] = present_cell;
-
-          end_cells[thread] = first_cells[thread];
-          if (thread != n_threads-1)
-            for (unsigned int i=0; i<n_cells_per_thread; ++i)
-              ++end_cells[thread];
-          else
-             end_cells[thread] = dof_handler.end();
-
-          // now start a new thread that builds the contributions of
-          // the cells in the given range
-          Threads::spawn (thread_manager,
-                   Threads::encapsulate(&MainClass::build_matrix_threaded)
-                         .collect_args (this,
-                                        first_cells[thread], 
-                                        end_cells[thread],
-                                        mutex));
-
-          // set start iterator for next thread
-          present_cell = end_cells[thread];
-        };
-
-      // wait for the threads to finish
-      thread_manager.wait ();
-    };
-
-
-    void MainClass::build_matrix_threaded 
-         (const DoFHandler<dim>::active_cell_iterator &first_cell,
-          const DoFHandler<dim>::active_cell_iterator &end_cell,
-          ACE_Thread_Mutex                            &mutex) 
-    {
-      FullMatrix<double>   cell_matrix;
-      vector<unsigned int> local_dof_indices;
-
-      DoFHandler<dim>::active_cell_iterator cell;
-      for (cell=first_cell; cell!=end_cell; ++cell)
-        {
-          // compute the elements of the cell matrix
-          ...
-
-          // get the indices of the DoFs of this cell
-          cell->get_dof_indices (local_dof_indices);
-
-          // now transfer local matrix into the global one.
-          // synchronise this with the other threads
-          mutex.acquire ();
-          for (unsigned int i=0; i<dofs_per_cell; ++i)
-            for (unsigned int j=0; j<dofs_per_cell; ++j)
-              global_matrix.add (local_dof_indices[i],
-                                 local_dof_indices[j],
-                                 cell_matrix(i,j));
-          mutex.release ();
-        };
-    };
-

-Note that since the build_matrix_threaded function takes its -arguments as references, we have to make sure that the variables to which -these references point live at least as long as the spawned threads. It is -thus not possible to use the same variables for start and end iterator for all -threads, as the following example would do: -

-      ....
-      DoFHandler<dim>::active_cell_iterator 
-             first_cell = dof_handler.begin_active ();
-      for (unsigned int thread=0; thread<n_threads; ++thread)
-        {
-          // for each thread: first determine the range of threads on
-          // which it shall operate:
-          DoFHandler<dim>::active_cell_iterator end_cell = first_cell;
-          if (thread != n_threads-1)
-            for (unsigned int i=0; i<n_cells_per_thread; ++i)
-              ++end_cell;
-          else
-             end_cell = dof_handler.end();
-
-          // now start a new thread that builds the contributions of
-          // the cells in the given range
-          Threads::spawn (thread_manager,
-                 Threads::encapsulate(&MainClass::build_matrix_threaded)
-                       .collect_args (this, first_cell, end_cell, mutex));
-
-          // set start iterator for next thread
-          first_cell = end_cell;
-        };
-      ....
-

-Since splitting a range of iterators (for example the range -begin_active() to end()) is a very common task when setting -up threads, there is a function -

-    template <typename ForwardIterator>
-    vector<pair<ForwardIterator,ForwardIterator> >
-    split_range (const ForwardIterator &begin, const ForwardIterator &end,
-                 const unsigned int n_intervals);
-

in the Threads namespace that splits the range -[begin,end) into n_intervals subintervals of -approximately the same size. - -

-Using this function, the thread creation function can now be written as -follows: -

-    void MainClass::build_matrix () {
-      const unsigned int n_threads = 4;
-      ACE_Thread_Mutex   mutex;
-      ACE_Thread_Manager thread_manager;
-
-      // define starting and end point for each thread
-      typedef DoFHandler<dim>::active_cell_iterator active_cell_iterator;
-      vector<pair<active_cell_iterator,active_cell_iterator> >
-           thread_ranges 
-           = split_range<active_cell_iterator> (dof_handler.begin_active (),
-                                                dof_handler.end (),
-                                                n_threads);
-
-      for (unsigned int thread=0; thread<n_threads; ++thread)
-        spawn (thread_manager,
-               encapsulate(&MainClass::build_matrix_threaded)
-                    .collect_args (this,
-                                   thread_ranges[thread].first,
-                                   thread_ranges[thread].second,
-                                   mutex));
-
-      thread_manager.wait ();
-    };
-

We have here omitted the Threads:: prefix to make things more -readable. Note that we had to explicitly specify the iterator type -active_cell_iterator to the split_range function, since -the two iterators given have different type (dof_handler.end() has -type DoFHandler<dim> :: raw_cell_iterator, which can be converted to -DoFHandler<dim>::active_cell_iterator) and C++ requires that either -the type is explicitly given or the type be unique. - -

-A word of caution is in place here: since usually in finite element -computations, the system matrix is ill-conditioned, small changes in a data -vector or the matrix can lead to significant changes in the output. -Unfortunately, since the order in which contributions to elements of the -matrix or vector are computed can not be predicted when using multiple -threads, round-off can come into play here. For example, taken from a -real-world program, the following contributions for an element of a right hand -side vector are computed from four cells: - --3.255208333333328815, - - --3.255208333333333694, - --3.255208333333333694, and - --3.255208333333331526; -however, due to round-off the sum of these numbers depends on the order in -which they are summed up, such that the resulting element of the vector -differed depending on the number of threads used, the number of other programs -on the computer, and other random sources. In subsequent runs of exactly the -same programs, the sum was either - --13.02083333333332827 or - - --13.02083333333332610. Although the difference is still only in the range -of round-off error, it caused a change in the fourth digit of a derived, very -ill-conditioned quantity after the matrix was inverted several times (this -accuracy in this quantity was not really needed, but it showed up in the -output and also led to different grid refinement due to comparison with other -values of almost the same size). Tracking down the source of such problems is -extremely difficult and frustrating, since they occur non-deterministically in -subsequent runs of the same program, and it can take several days until the -actual cause is found. - -

-One possible work-around is to reduce the accuracy of the summands such that -the value of the sum becomes irrespective of the order of the summands. One, -rather crude method is to use a conversion to data type float and -back; the update loop from above would then look as follows: -

-          for (unsigned int i=0; i<dofs_per_cell; ++i)
-            for (unsigned int j=0; j<dofs_per_cell; ++j)
-              global_matrix.add (local_dof_indices[i],
-                                 local_dof_indices[j],
-                                 static_cast<float>(cell_matrix(i,j)));
-

Note that the cast back to double is performed here implicitly. The -question whether a reduction in accuracy in the order shown here is tolerable, -is problem dependent. There are methods that lose less accuracy than shown -above. - -

-The other, less computationally costly possibility would be to decrease the -accuracy of the resulting sum, in the hope that all accumulated round-off -error is deleted. However, this is unsafe since the order dependence remains -and may even be amplified if the values of the sum lie around a boundary where -values are rounded up or down when reducing the accuracy. Furthermore, -problems arise if the summands have different signs and the result of -summation consists of round-off error only. - -

- - -

- -

-
- Next: Parallel Jacobi preconditioning - Up: Applications - Previous: Writing output detached to - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node11.html b/deal.II/doc/reports/multithreading/node11.html deleted file mode 100644 index c169df46e3..0000000000 --- a/deal.II/doc/reports/multithreading/node11.html +++ /dev/null @@ -1,186 +0,0 @@ - - - - -Parallel Jacobi preconditioning - - - - - - - - - - - - - -

- -

-
- Next: Conclusions - Up: Applications - Previous: Assembling the matrix -
-
- - -

-Parallel Jacobi preconditioning -

- -

-When preconditioning a matrix, for example in a Conjugate Gradients solver, -one may choose the Jacobi scheme for preconditioning. The preconditioned -vector $\tilde v$ -is computed from the vector v using the following -relationship: -

- - - - $\begin{displaymath}\tilde v_i = \frac 1{a_{ii}} v_i, -\end{displaymath}$ -

-
-

-where a_ii are the diagonal elements of the matrix which we are presently -inverting. As is obvious, the result of preconditioning one element of v is -entirely independent of all other elements, so this operation is trivially -parallelizable. In practice, this is done by splitting the interval [0,n)into equal parts - - $[n_i,n_{i+1}), i=0,\dots,p-1$ , -where n is the size of the -matrix, and p is the number of processors. Obviously, - -n₀=0, n_p=n, and - - -n_i<n_i+1. - -

-Just like for splitting a range of iterators using the function -split_range used above, there is a function -

-    vector<pair<unsigned int, unsigned int> >
-    split_interval (const unsigned int &begin, const unsigned int &end,
-                    const unsigned int n_intervals);
-

that splits the interval [begin,end) into n_intervals equal -parts. This function will be used to assign each processor its share of -elements v_i. - -

-Furthermore, we will use some functionality provided by the -MultithreadInfo class in deal.II. Upon start-up of the -library, the static variable multithread_info.n_cpus is set to the -number of processors in the computer the program is presently running on. -multithread_info is a global variable of type -MultithreadInfo available in all parts of the library. Furthermore, -there is a variable multithread_info.n_default_threads, which by -default is set to n_cpus, but which can be changed by the user; it -denotes the default number of threads which the library shall use whenever -multi-threading is implemented for some operation. We will use this variable to -decide how many threads shall be used to precondition the vector. - -

-The implementation of the preconditioning function then looks like this: -

-                        // define an abbreviatory data type for an interval
-    typedef pair<unsigned int, unsigned int> Interval;
-
-    void Preconditioner::precondition_jacobi (const Matrix &m,
-                                              const Vector &v,
-                                              Vector       &v_tilde) {
-                            // define an abbreviation to the number
-                            // of threads which we will use
-      const unsigned int n_threads = multithread_info.n_default_threads;
-                            // first split the interval into equal pieces
-      vector<Interval> intervals = Threads::split_interval (0, m.rows(),
-                                                            n_threads);
- 
-                            // then define a thread manager
-      ACE_Thread_Manager thread_manager;
-                            // and finally start all the threads:
-      for (unsigned int i=0; i<n_threads; ++i)
-        Threads::spawn (thread_manager,
-                   Threads::encapsulate (&Preconditioner::threaded_jacobi)
-                          .collect_args (this, m, v, v_tilde, intervals[i]));
-  
-                            // wait for all the threads to finish
-      thread_manager.wait ();
-    };
-
-
-    void Preconditioner::threaded_jacobi (const Matrix   &m,
-                                          const Vector   &v,
-                                          Vector         &v_tilde,
-                                          const Interval &interval) {
-                           // apply the preconditioner in the given interval
-      for (unsigned int i=interval.first; i<interval.second; ++i)
-        v_tilde(i) = v(i) / m(i,i);
-    };
-

-It is noted, however, that more practical preconditioners are usually not -easily parallelized. However, matrix-vector and vector-vector operations can -often be reduced to independent parts and can then be implemented using -multiple threads. - -

- - -

- -

-
- Next: Conclusions - Up: Applications - Previous: Assembling the matrix - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node12.html b/deal.II/doc/reports/multithreading/node12.html deleted file mode 100644 index 5e6fd4cef3..0000000000 --- a/deal.II/doc/reports/multithreading/node12.html +++ /dev/null @@ -1,115 +0,0 @@ - - - - -Conclusions - - - - - - - - - - - - - - -

- -

-
- Next: Bibliography - Up: No Title - Previous: Parallel Jacobi preconditioning -
-
- - -

-Conclusions -

- -

-We have shown how multi-threading is supported in deal.II and how it -can be used in several examples occurring in common finite element programs. It -was demonstrated that implementing a usable C++ interface poses several -difficulties, both from the aspect of user friendliness as well as program -correctness. In order to overcome these difficulties, first the more simple -framework implemented in deal.II version 3.0 was discussed, followed -by a rather complex scheme which will be the base of implementations in future -versions. - -

-The second framework features a more complicated hierarchy of classes as well -as intricate use of templates and synchronization mechanisms, which however -led to a design in which threads can be created in a user friendly, system -independent, C++ like way suitable for common programs. The use of this -framework is inherently safe and does not require special knowledge of the -internals by the user, and is simple to use. By using it, the overhead -required for programming multi-threaded applications is reduced to a minimum -and the programmer can concentrate on the task of getting the semantics of -multi-threaded programs right, in particular managing concurrent access to -data and distributing work to different threads. - -

-The framework has been used in several application programs and has shown that -with only marginally increased programming effort, finite element programs can -be made significantly faster on multi-processor machines. -
- -

- -

-Acknowledgments. -

The author would like to thank Thomas Richter -for his work in parallelizing several parts of the deal.II library, -and Ralf Hartmann for help in the preparation of this report. - -

- -

- - -

- -

-
- Next: Bibliography - Up: No Title - Previous: Parallel Jacobi preconditioning - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node13.html b/deal.II/doc/reports/multithreading/node13.html deleted file mode 100644 index b28522f579..0000000000 --- a/deal.II/doc/reports/multithreading/node13.html +++ /dev/null @@ -1,110 +0,0 @@ - - - - -Bibliography - - - - - - - - - - - - - - -

- -

-
- Next: About this document ... - Up: No Title - Previous: Conclusions -

- -

Bibliography -

1 -: -Wolfgang Bangerth and Guido Kanschat. -
Concepts for object-oriented finite element software - the - deal.II library. -
Preprint 99-43, SFB 359, Universität Heidelberg, October 1999. - -
2 -: -Wolfgang Bangerth and Guido Kanschat. -
deal.II Differential Equations Analysis Library, - Technical Reference. -
IWR Heidelberg, October 1999. -
http://gaia.iwr.uni-heidelberg.de/~deal/. - -
3 -: -H. Custer. -
Inside Windows NT. -
Microsoft Press, Redmont, Washington, 1993. - -
4 -: -Douglas C. Schmidt et al. -
WWW homepage of the Adaptive Communications Environment - ACE, http://www.cs.wustl.edu/~schmidt/ACE.html. - -
5 -: -J. Eykholt, S. Kleinman, S. Barton, R. Faulkner, A. Shivalingiah, M. Smith, - D. Stein, J. Voll, M. Weeks, and D. Williams. -
Beyon multiprocessing... Multithreading the SunOS kernel. -
In Proceedings of the Summer USENIX C++ Technical Conference, - San Antonio, Texas, June 1992. - -
6 -: -IEEE. -
Threads extensions for portable operating systems. -
Technical report, IEEE, 1995. - -
7 -: -Douglas C. Schmidt. -
ACE: an object-oriented framework for developing distributed - applications. -
In Proceedings of the Sixth USENIX C++ Technical Conference, - Cambridge, Massachusetts. USENIX Association, April 1994. - -
8 -: -Douglas C. Schmidt and Nanbor Wang. -
An OO encapsulation of lightweight OS concurrency mechanisms - in the ACE toolkit. -
Technical Report WUCS-95-31, Washington University, St. Louis, 1995. -

- -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node14.html b/deal.II/doc/reports/multithreading/node14.html deleted file mode 100644 index c93ca784b6..0000000000 --- a/deal.II/doc/reports/multithreading/node14.html +++ /dev/null @@ -1,61 +0,0 @@ - - - - -About this document ... - - - - - - - - - - - -

- -

-
- Up: No Title - Previous: Bibliography -
-
- - -

- About this document ... -

-This document was generated using the -LaTeX2HTML translator Version 98.1p1 release (March 2nd, 1998) -

-The command line arguments were:
- latex2html -split 5 -antialias multithreading.tex. -

-The translation was initiated by Wolfgang Bangerth on 2000-04-20 -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node2.html b/deal.II/doc/reports/multithreading/node2.html deleted file mode 100644 index 088d0d77e7..0000000000 --- a/deal.II/doc/reports/multithreading/node2.html +++ /dev/null @@ -1,100 +0,0 @@ - - - - -Threads - - - - - - - - - - - - - - -

- -

-
- Next: Creating and managing threads - Up: No Title - Previous: Background -
-
- - -

-Threads -

- -

-The basic entity for programming multi-processor machines are -threads. They represent parts of the program which are executed in -parallel. Threads can be considered as separate programs that work on the same -main memory. On single-processor machines, they are simulated by letting each -thread run for some time (usually a few milliseconds) before switching to the -next thread. On multi-processor machines, threads can truly be executed in -parallel. In order to let programs use more than one thread (which would be -the regular sequential program), several aspects need to be covered: -

How do we assign operations to different threads? Of course, operations - which depend on each other must not be executed in reverse order. This can - be achieved by only letting independent operations run on different threads, - or by using synchronization methods. this is mostly a question of program - design and thus problem dependent, which is why both aspects will only be - briefly touched below. -
How does the operating system and the whole programming environment - support this? -

As mentioned, only the second aspect can be canonicalized, so we will treat it -first. Some examples of actual parallelized applications are discussed in -Section 4. - -

- - -

- -

-
- Next: Creating and managing threads - Up: No Title - Previous: Background - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node3.html b/deal.II/doc/reports/multithreading/node3.html deleted file mode 100644 index bbee3591d6..0000000000 --- a/deal.II/doc/reports/multithreading/node3.html +++ /dev/null @@ -1,84 +0,0 @@ - - - - -Creating and managing threads - - - - - - - - - - - - - - -

- -

-
- Next: Operating system dependence and - Up: No Title - Previous: Threads -
-
- - -

-Creating and managing threads -

- -

- - -

- -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node4.html b/deal.II/doc/reports/multithreading/node4.html deleted file mode 100644 index bbae54ec83..0000000000 --- a/deal.II/doc/reports/multithreading/node4.html +++ /dev/null @@ -1,114 +0,0 @@ - - - - -Operating system dependence and ACE - - - - - - - - - - - - - - -

- -

-
- Next: C interface to threads - Up: Creating and managing threads - Previous: Creating and managing threads -
-
- - -

-Operating system dependence and ACE -

- -

-While all relevant operating systems now support multi-threaded programs, they -all have different notions on what threads actually are on an operating system -level, how they shall be managed and created. Even on Unix systems, which are -usually well-standardized, there are at least three different and mutually -incompatible interfaces to threads: POSIX threads [6], -Solaris threads [5], and Linux -threads. Some operating systems support more than one interface, but there is -no interface that is supported by all operating systems. Furthermore, other -systems like Microsoft Windows have interfaces that are incompatible to all -Unix systems [3]. - -

-Writing multi-threaded programs based on the operating system interfaces is -therefore something inherently incompatible unless much effort is spent to -port it to a new system. To avoid this, we chose to use the ACE (Adaptive -Communication Environment, see [7,8,4]) library which encapsulates -the operating system dependence and offers a uniform interface to the user. - -

-We chose ACE over other libraries, since it runs on almost all relevant -platforms, including most Unix systems and Microsoft Windows, -and since it is to our knowledge the only library which is actively developed by a -large group. Furthermore, -it also is significantly larger than only thread management, offering -interprocess communication and communication between different computers, as -well as many other services. Contrary to most other libraries, it therefore -offers both the ability to support a growing deal.II as well as the -prospect to support independence also with respect to future platforms. - -

- - -

- -

-
- Next: C interface to threads - Up: Creating and managing threads - Previous: Creating and managing threads - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node5.html b/deal.II/doc/reports/multithreading/node5.html deleted file mode 100644 index 0cc08edd19..0000000000 --- a/deal.II/doc/reports/multithreading/node5.html +++ /dev/null @@ -1,106 +0,0 @@ - - - - -C interface to threads versus C++ - - - - - - - - - - - - - - -

- -

-
- Next: First approach - Up: Creating and managing threads - Previous: Operating system dependence and -
-
- - -

-`C` interface to threads versus `C++` -

- -

-While ACE encapsulates almost all of the synchronization and interprocess -interface into C++ classes, it for some reason does not do so for -thread creation. Rather it only offers the basic C interface: -when creating a new thread, a function is called which has the following -signature:

Code sample 1 -

-    void *  f (void * arg);
-

-Thus, only functions which take a single parameter of type void* and -return a void* may be called. Further, these functions must be global -or static member functions, as opposed to true member functions of -classes. This is not in line with the C++ philosophy and in fact does -not fit well into deal.II as well: there is not a single function in -the library that has this signature. - -

-The task of multi-threading support in deal.II is therefore to -encapsulate member functions, arbitrary types and numbers of parameters, and -return types of functions into mechanisms built atop of ACE. This has been -done twice for deal.II, and we will explain both approaches. At -present, i.e. with version 3.0, only the first approach is distributed with -deal.II, since the second is still experimental and due to the high -complexity. The latter approach, however, has clear advantages over the first -one, and it is planned to switch to it in the next major version of -deal.II. - -

- - -

- -

-
- Next: First approach - Up: Creating and managing threads - Previous: Operating system dependence and - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node6.html b/deal.II/doc/reports/multithreading/node6.html deleted file mode 100644 index 3e3b4af662..0000000000 --- a/deal.II/doc/reports/multithreading/node6.html +++ /dev/null @@ -1,281 +0,0 @@ - - - - -First approach - - - - - - - - - - - - - - -

- -

-
- Next: Second approach - Up: Creating and managing threads - Previous: C interface to threads -
-
- - -

-First approach -

- -

-The first idea is the following: assume that we have a class -TestClass

Code sample 2 -

-    class TestClass {
-      public:
-        void test_function (int i, double d);
-    };
-

-and we would like to call -test_object.test_function(1,3.1415926) on a newly created -thread, where test_object is -an object of type TestClass. We then need an object that encapsulates -the address of the member function, a pointer to the object for which we want -to call the function, and both parameters. This class would be suitable:

Code sample 3 -

-    struct MemFunData {
-        typedef void (TestClass::*MemFunPtr) (int, double);
-        MemFunPtr  mem_fun_ptr;
-        TestClass *object;
-        int        arg1;
-        double     arg2;
-    };
-

- -

-We further need a function that satisfies the signature required by the -operating systems (or ACE, respectively), see Code Sample -1, and that can call the member function -if we pass it an object of type MemFunData:

Code sample 4 -

-    void * start_thread (void *arg_ptr) {
-                        // first reinterpret the void* as a
-                        // pointer to the object which
-                        // encapsulates the arguments
-                        // and addresses:
-      MemFunData *mem_fun_data
-            = reinterpret_cast<MemFunData *>(arg_ptr);
-                        // then call the member function:
-      (mem_fun_data->object)
-            ->*(mem_fun_data->mem_fun_ptr) (mem_fun_data->arg1,
-                                            mem_fun_data->arg2);
-                        // since the function does not return
-                        // a value, we do so ourselves:
-      return 0;
-    };
-

-Such functions are called trampoline functions since they only serve -as jump-off point for other functions. - -

-We can then perform the desired call using the following sequence of commands: -

-    MemFunData mem_fun_data;
-    mem_fun_data.mem_fun_ptr = &TestClass::test_function;
-    mem_fun_data.object      = &test_object;
-    mem_fun_data.arg1        = 1;
-    mem_fun_data.arg2        = 3.1415926;
-    
-    ACE_Thread_Manager::spawn (&start_thread,
-                               (void*)&mem_fun_data);
-

ACE_Thread_Manager::spawn is the function from ACE that actually -calls the operating system and tells it to create a new thread and call the -function which it is given as first parameter (here: start_thread) -with the parameter which is given as second parameter. start_thread, -when called, will then get the address of the function which we wanted to call -from its parameter, and call it with the values we wanted as arguments. - -

-In practice, this would mean that we needed a structure like -MemFunData and a function like start_thread for each class -TestClass and all functions test_function with different -signatures. This is clearly not feasible in practice and places an -inappropriate burden on the programmer who wants to use multiple threads in -his program. Fortunately, C++ offers an elegant way for this problem, -in the form of templates: we first define a data type which encapsulates -address and arguments for all binary functions:

Code sample 5 -

-    template <typename Class, typename Arg1, typename Arg2>
-    struct MemFunData {
-        typedef void (Class::*MemFunPtr) (Arg1, Arg2);
-        MemFunPtr  mem_fun_ptr;
-        Class     *object;
-        Arg1       arg1;
-        Arg2       arg2;
-    };
-

-Next, we need a function that can process these arguments:

Code sample 6 -

-    template <typename Class, typename Arg1, typename Arg2>
-    void * start_thread (void *arg_ptr) {
-      MemFunData<Class,Arg1,Arg2> *mem_fun_data
-            = reinterpret_cast<MemFunData<Class,Arg1,Arg2>*>(arg_ptr);
-      (mem_fun_data->object)
-            ->*(mem_fun_data->mem_fun_ptr) (mem_fun_data->arg1,
-                                            mem_fun_data->arg2);
-      return 0;
-    };
-

-Then we can start the thread as follows: -

-    MemFunData<TestClass,int,double>  mem_fun_data;
-    mem_fun_data.mem_fun_ptr = &TestClass::test_function;
-    mem_fun_data.object      = &test_object;
-    mem_fun_data.arg1        = 1;
-    mem_fun_data.arg2        = 3.1415926;
-    
-    ACE_Thread_Manager::spawn (&start_thread<TestClass,int,double>,
-                               (void*)&mem_fun_data);
-

Here we first create an object which is suitable to encapsulate the parameters -of a binary function that is a member function of the TestClass class -and takes an integer and a double. Then we start the thread using the correct -trampoline function. It is the user's responsibility to choose the correct -trampoline function (i.e. to specify the correct template parameters) since -the compiler only sees a void* and cannot do any type checking. - -

-We can further simplify the process and remove the user responsibility by -defining the following class and function:

Code sample 7 -

-    class ThreadManager : public ACE_Thread_Manager {
-      public:
-        template <typename Class, typename Arg1, typename Arg2>
-        static void 
-        spawn (MemFunData<Class,Arg1,Arg2> &MemFunData) {
-          ACE_Thread_Manager::spawn (&start_thread<Class,Arg1,Arg2>,
-                                     (void*)&MemFunData);
-        };
-    };
-

-This way, we can call -

-    ThreadManager::spawn (mem_fun_data);
-

and the compiler will figure out which the right trampoline function is, since -it knows the data type of mem_fun_data and therefore the values of -the template parameters in the ThreadManager:: spawn function. - -

-The way described above is basically the way which is used in deal.II -version 3.0. Some care has to be paid to details, however. In particular, -C++ functions often pass references as arguments, which however are -not assignable after initialization. Therefore, the MemFunData class -needs to have a constructor, and arguments must be set through it. Assume, for -example, TestClass had a second member function -

-        void f (int &i, double &d);
-

Then, we would have to use MemFunData<TestClass,int&,doubleSPMamp;>, -which in a form without templates would look like this: -

-    struct MemFunData {
-        typedef void (TestClass::*MemFunPtr) (int &, double &);
-        MemFunPtr  mem_fun_ptr;
-        TestClass *object;
-        int       &arg1;
-        double    &arg2;
-    };
-

The compiler would require us to initialize the references to the two -parameters at construction time of the MemFunData object, since -it is not possible in C++ to change to which object a reference -points to after initialization. Adding a constructor to the -MemFunData class would then enable us to write -

-    int    i = 1;
-    double d = 3.1415926;
-    MemFunData<TestClass,int&,double&>  
-           mem_fun_data (&test_object, i, d, &TestClass::f);
-

Non-reference arguments could then still be changed after construction. For -historical reasons, the pointer to the member function is passed as last -parameter here. - -

-The last point is that this interface is only usable for functions with two -parameters. Basically, the whole process has to be reiterated for any number -of parameters which we want to support. In deal.II, we therefore have -classes MemFunData0 through MemFunData10, corresponding to -member function that do not take parameters through functions that take ten -parameters. Equivalently, we need the respective number of trampoline -functions. - -

-Additional thoughts need to be taken on virtual member functions and constant -functions. While the first are handled by the compiler (member function -pointers can also be to virtual functions, without explicitly stating so), the -latter can be achieved by writing -MemFunData<const TestClass,int,double>, which would be the correct -object if we had declared test_function constant. - -

-Finally we note that it is often the case that one member function starts a -new thread by calling another member function of the same object. Thus, the -declaration most often used is the following: -

-    MemFunData<TestClass,int&,double&>  
-           mem_fun_data (this, 1, 3.1415926, &TestClass::f);
-

Here, instead of an arbitrary test_object, the present object is -used, which is represented by the this pointer. - -

- - -

- -

-
- Next: Second approach - Up: Creating and managing threads - Previous: C interface to threads - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node7.html b/deal.II/doc/reports/multithreading/node7.html deleted file mode 100644 index 0d05a9604a..0000000000 --- a/deal.II/doc/reports/multithreading/node7.html +++ /dev/null @@ -1,708 +0,0 @@ - - - - -Second approach - - - - - - - - - - - - - -

- -

-
- Next: Applications - Up: Creating and managing threads - Previous: First approach -
-
- - -Subsections -

- -

- -
-Second approach -

- -

-While the approach outlined above works satisfactorily, it has one serious -drawback: the programmer has to provide the data types of the arguments of the -member function himself. While this seems to be a simple task, in practice it -is often not, as will be explained in the sequel. - -

-To expose the problem, we take an example from one of our application programs -where we would like to call the function -

-    template <int dim>
-    void DoFHandler<dim>::distribute_dofs (const FiniteElement<dim> &,
-                                           const unsigned int);
-

on a new thread. Correspondingly, we would need to use -

-    MemFunData2<DoFHandler<dim>, const FiniteElement<dim> &, unsigned int>
-        mem_fun_data (dof_handler, fe, 0,
-                      &DoFHandler<dim>::distribute_dofs);
-

to encapsulate the parameters. However, if one forgets the const -specifier on the second template parameter, one receives the following error -message (using gcc 2.95.2): -

-  test.cc: In method `void InterstepData<2>::wake_up(unsigned int, Interst
-  epData<2>::PresentAction)':
-  test.cc:683:   instantiated from here
-  test.cc:186: no matching function for call to `ThreadManager::Mem_Fun_Da
-  ta2<DoFHandler<2>,FiniteElement<2> &,unsigned int>::MemFunData2 (DoFHa
-  ndler<2> *, const FiniteElement<2> &, int, void (DoFHandler<2>::*)(const
-   FiniteElement<2> &, unsigned int))'
-  /home/atlas1/wolf/program/newdeal/deal.II/base/include/base/thread_manag
-  er.h:470: candidates are: ThreadManager::MemFunData2<DoFHandler<2>,Fin
-  iteElement<2> &,unsigned int>::MemFunData2(DoFHandler<2> *, FiniteElem
-  ent<2> &, unsigned int, void * (DoFHandler<2>::*)(FiniteElement<2> &, un
-  signed int))
-  /home/atlas1/wolf/program/newdeal/deal.II/base/include/base/thread_manag
-  er.h:480:                 ThreadManager::MemFunData2<DoFHandler<2>,Fin
-  iteElement<2> &,unsigned int>::MemFunData2(DoFHandler<2> *, FiniteElem
-  ent<2> &, unsigned int, void (DoFHandler<2>::*)(FiniteElement<2> &, unsi
-  gned int))
-  /home/atlas1/wolf/program/newdeal/deal.II/base/include/base/thread_manag
-  er.h:486:                 ThreadManager::MemFunData2<DoFHandler<2>,Fin
-  iteElement<2> &,unsigned int>::MemFunData2(const ThreadManager::Mem_Fu
-  n_Data2<DoFHandler<2>,FiniteElement<2> &,unsigned int> &)
-

-While the compiler is certainly right to complain, the message is not very -helpful. Furthermore, since interfaces to functions sometimes change, for -example by adding additional default parameters that do not show up in usual -code, programs that used to compile do no more so with messages as shown -above. - -

-Due to the lengthy and complex error messages, even very experienced -programmers usually need between five and ten minutes until they get an -expression like this correct. In most cases, they don't get it right in the -first attempt, so the time used for the right declaration dominates the whole -setup of starting a new thread. To circumvent this bottleneck at least in most -cases, we chose to implement a second strategy at encapsulating the parameters -of member functions. This is done in several steps: first let the compiler -find out about the right template parameters, then encapsulate the parameters, -use the objects, and finally solve some technical problems with virtual -constructors and locking of destruction. We will treat these steps -sequentially in the following. - -

- -

-Finding the correct template parameters. -

-C++ offers the possibility of templatized functions that deduce their -template arguments themselves. In fact, we have used them in the -ThreadManager::spawn function in Code Sample 7 -already. Here, this can be used as follows: assume we have a -function encapsulation class -

-    template <typename Class, typename Arg1, typename Arg2>
-    class MemFunData { ... };
-

as above, and a function -

-    template <typename Class, typename Arg1, typename Arg2>
-    MemFunData<Class,Arg1,Arg2>
-    encapsulate (void (Class::*mem_fun_ptr)(Arg1, Arg2)) {
-      return MemFunData<Class,Arg1,Arg2> (mem_fun_ptr);
-    };
-

Then, if we call this function with the test class of Code Sample -2 like this: -

-    encapsulate (&TestClass::test_function);
-

it can unambiguously determine the template parameters to be -Class=TestClass, Arg1=int, Arg2=double. - -

- -

-Encapsulating the parameters. -

-We should not try to include the argument values for the new thread right -away, for example by declaring encapsulate like this: -

-    template <typename Class, typename Arg1, typename Arg2>
-    MemFunData<Class,Arg1,Arg2>
-    encapsulate (void (Class::*mem_fun_ptr)(Arg1, Arg2),
-                 Arg1  arg1,
-                 Arg2  arg2,
-                 Class object) {
-      return MemFunData<Class,Arg1,Arg2> (mem_fun_ptr, object, arg1, arg2);
-    };
-

The reason is that for template functions, no parameter promotion is -performed. Thus, if we called this function as in -

-    encapsulate (&TestClass::test_function,
-                 1, 3,
-                 test_object);
-

then the compiler would refuse this since from the function pointer it must -deduce that Arg2 = double, but from the parameter ``3'' it must assume -that Arg2 = int. The resulting error message would be similarly lengthy -as the one shown above. - -

-One could instead write MemFunData like this: -

-    template <typename Class, typename Arg1, typename Arg2>
-    class MemFunData { 
-      public:
-        typedef void (Class::*MemFunPtr)(Arg1, Arg2);
-
-        MemFunData (MemFunPtr mem_fun_ptr_) {
-          mem_fun_ptr = mem_fun_ptr_;
-        };
-
-        void collect_args (Class *object_,
-                           Arg1   arg1_,
-                           Arg2   arg2_) {
-          object = object_;
-          arg1   = arg1_;
-          arg2   = arg2_;
-        };
-
-        MemFunPtr  mem_fun_ptr;
-        Class     *object;
-        Arg1       arg1;
-        Arg2       arg2;
-    };
-

One would then create an object of this type including the parameters to be -passed as follows: -

-    encapsulate(&TestClass::test_function).collect_args(test_object, 1, 3);
-

Here, the first function call creates an object with the right template -parameters and storing the member function pointer, and the second one, -calling a member function, fills in the function arguments. - -

-Unfortunately, this way does not work: if one or more of the parameter types -is a reference, then the respective reference variable needs to be initialized -by the constructor, not by collect_args. It needs to be known which -object the reference references at construction time, since later on only the -referenced object can be assigned, not the reference itself anymore. - -

-Since we feel that we are close to a solution, we introduce one more -indirection, which indeed will be the last one:

Code sample 8 -

-    template <typename Class, typename Arg1, typename Arg2>
-    class MemFunData { 
-      public:
-        typedef void (Class::*MemFunPtr)(Arg1, Arg2);
-
-        MemFunData (MemFunPtr mem_fun_ptr_,
-                      Class *object_,
-                      Arg1   arg1_,
-                      Arg2   arg2_) :
-             mem_fun_ptr (mem_fun_ptr_),
-             object      (object_),
-             arg1        (arg1_),
-             arg2        (arg2_)            {};
-
-        MemFunPtr  mem_fun_ptr;
-        Class     *object;
-        Arg1       arg1;
-        Arg2       arg2;
-    };
-
-
-    template <typename Class, typename Arg1, typename Arg2>
-    struct ArgCollector { 
-        typedef void (Class::*MemFunPtr)(Arg1, Arg2);
-
-        ArgCollector (MemFunPtr mem_fun_ptr_) {
-          mem_fun_ptr = mem_fun_ptr_;
-        };
-
-        
-        MemFunData<Class,Arg1,Arg2>
-        collect_args (Class *object_,
-                      Arg1   arg1_,
-                      Arg2   arg2_) {
-          return MemFunData<Class,Arg1,Arg2> (mem_fun_ptr, object,
-                                              arg1, arg2);
-        };
-
-        MemFunPtr  mem_fun_ptr;
-    };
-
-
-    template <typename Class, typename Arg1, typename Arg2>
-    ArgCollector<Class,Arg1,Arg2>
-    encapsulate (void (Class::*mem_fun_ptr)(Arg1, Arg2)) {
-      return ArgCollector<Class,Arg1,Arg2> (mem_fun_ptr);
-    };
-

- -

-Now we can indeed write for the test class of Code Sample 2: -

-    encapsulate(&TestClass::test_function).collect_args(test_object, 1, 3);
-

The first call creates an object of type ArgCollector<...> with the -right parameters and storing the member function pointer, while the second -call, a call to a member function of that -intermediate class, generates the final object we are interested in, including -the member function pointer and all necessary parameters. Since -collect_args already has its template parameters fixed from -encapsulate, it can convert between data types. - -

- -

-Using these objects. -

-Now we have an object of the correct type -automatically generated, without the need to type in any template parameters -by hand. What can we do with that? First, we can't assign it to a variable of -that type, e.g. for use in several spawn commands: -

-  MemFunData mem_fun_data = encapsulate(...).collect_args(...);
-

Why? Since we would then have to write the data type of that variable by hand: -the correct data type is not MemFunData as written above, but -MemFunData<TestClass,int,double>. Specifying all these template -arguments was exactly what we wanted to avoid. However, we can do some such -thing if the variable to which we assign the result is of a type which is a -base class of MemFunData<...>. Unfortunately, the data values that -MemFunData<...> encapsulates depend on the template parameters, so -the respective variables in which we store the values can only be placed in -the derived class and could not be copied when we assign the variable to a -base class object, since that does not have these variables. - -

-What can we do here? Assume we have the following class structure:

Code sample 9 -

-    class FunDataBase {};
-
-    template <...> class MemFunData : public FunDataBase 
-    {  /* as above */ };
-
-    class FunEncapsulation {
-      public:
-        FunEncapsulation (FunDataBase *f)
-                   : fun_data_base (f) {};
-        FunDataBase *fun_data_base;
-    };
-
-
-    template <typename Class, typename Arg1, typename Arg2>
-    FunEncapsulation
-    ArgCollector<Class,Arg1,Arg2>::collect_args (Class *object_,
-                                                 Arg1   arg1_,
-                                                 Arg2   arg2_) {
-      return new MemFunData<Class,Arg1,Arg2> (mem_fun_ptr, object,
-                                              arg1, arg2);
-    };
-

-Note that in the return statement of the collect_args function, -first a cast from MemFunData* to FunDataBase*, and then a -constructor call to FunEncapsulation :: FunEncapsulation (FunDataBase*) -was performed. - -

-In the example above, the call to encapsulate(...).collect_args(...) -generates an object of type FunEncapsulation, which in turn stores a -pointer to an object of type FunDataBase, here to -MemFunData<...> with the correct template parameters. We can assign -the result to a variable the type of which does not contain any template -parameters any more, as desired: -

-    FunEncapsulation 
-        fun_encapsulation = encapsulate (&TestClass::test_function)
-                                          .collect_args(test_object, 1, 3);
-

-But how can we start a thread with this object if we have lost the full -information about the data types? This can be done as follows: add a variable -to FunDataBase which contains the address of a function that knows -what to do. This function is usually implemented in the derived classes, and -its address is passed to the constructor:

Code sample 10 -

-    class FunDataBase {
-      public:
-        typedef void * (*ThreadEntryPoint) (void *);
-
-        FunDataBase (ThreadEntryPoint t) :
-                 thread_entry_point (t) {};
-
-        ThreadEntryPoint thread_entry_point;
-    };
-
-    template <...>
-    class MemFunData : public FunDataBase {
-      public:
-                 // among other things, the constructor now does this:
-        MemFunData () :
-                 FunDataBase (&start_thread) {};
-
-        static void * start_thread (void *args) {
-          // do the same as in Code Sample 4 above
-        }
-    };
-
-
-    void spawn (ACE_Thread_Manager &thread_manager,
-                FunEncapsulation   &fun_encapsulation) {
-      thread_manager.spawn (*fun_encapsulation.fun_data_base
-                                      ->thread_entry_point,
-                            &fun_data_base);
-    };
-

-fun_encapsulation.fun_data_base->thread_entry_point is given -by the derived class as that function that knows how to handle objects of the -type which we are presently using. Thus, we can now write the whole sequence -of function calls (assuming we have an object thread_manager of type -ACE_Thread_Manager): -

-    FunEncapsulation 
-        fun_encapsulation = encapsulate (&TestClass::test_function)
-                                          .collect_args(test_object, 1, 3);
-    spawn (thread_manager, fun_encapsulation);
-

This solves our problem in that no template parameters need to be specified by -hand any more. The only source for lengthy compiler error messages is if the -parameters to collect_args are in the wrong order or can not be -casted to the parameters of the member function which we want to call. These -problems, however, are much more unlikely in our experience, and are also much -quicker sorted out. - -

- -

-Virtual constructors. -

While the basic techniques have been fully -developed now, there are some aspects which we still have to take care of. The -basic problem here is that the FunEncapsulation objects store a -pointer to an object that was created using the new operator. To -prevent a memory leak, we need to destroy this object at some time, preferably -in the destructor of FunEncapsulation: -

-    FunEncapsulation::~FunEncapsulation () {
-      delete fun_data_base;
-    };
-

However, what happens if we have copied the object before? In particular, this -is always the case using the functions above: collect_args generates -a temporary object of type FunEncapsulation, but there could be other -sources of copies as well. If we do not take special precautions, only the -pointer to the object is copied around, and we end up with stale pointers -pointing to invalid locations in memory once the first object has been -destroyed. What we obviously need to do when copying objects of type -FunEncapsulation is to not copy the pointer but to copy the object -which it points to. Unfortunately, the following copy constructor is not -possible: -

-    FunEncapsulation::FunEncapsulation (const FunEncapsulation &m) {
-      fun_data_base = new FunDataBase (*m.fun_data_base);
-    };
-

The reason, of course, is that we do not want to copy that part of the object -belonging to the abstract base class. But we can emulate something like this -in the following way (this programming idiom is called ``virtual -constructors''):

Code sample 11 -

-    class FunDataBase {
-      public:
-        // as above
-
-        virtual FunDataBase * clone () const = 0;
-    };
-
-    template <...>
-    class MemFunData : public FunDataBase {
-      public:
-        // as above
-
-                          // copy constructor:
-        MemFunData (const MemFunData<...> &mem_fun_data) {...};
-
-                          // clone the present object, i.e.
-                          // create an exact copy:
-        virtual FunDataBase * clone () const {
-          return new MemFunData<...>(*this);
-        };
-    };
-
-
-    FunEncapsulation::FunEncapsulation (const FunEncapsulation &m) {
-      fun_data_base = m.fun_data_base->clone ();
-    };
-

-Thus, whenever the FunEncapsulation object is copied, it creates a -copy of the object it harbors (the MemFunData<...> object), and -therefore always owns its copy. When the destructor is called, it is free to -delete its copy without affecting other objects (from which it may have been -copied, or to which it was copied). Similar to the copy constructor, we have -to modify the copy operator, as well. - -

- -

-Spawning independent threads. -

- -

-Often, one wants to spawn a thread which will have its own existence until it -finishes, but is in no way linked to the creating thread any more. An example -would be the following, assuming a function -TestClass::compress_file(const string file_name) exists and that -there is an object thread_manager not local to this function: -

-  
-    ...
-    string file_name;
-    ...    // write some output to a file
-
-    // now create a thread which runs `gzip' on that output file to reduce
-    // disk space requirements. don't care about that thread any more
-    // after creation, i.e. don't wait for its return
-    FunEncapsulation 
-        fun_encapsulation = encapsulate (&TestClass::compress_file)
-                                  .collect_args(test_object, file_name);
-    spawn (thread_manager, fun_encapsulation);
-
-    // quit the present function
-    return;
-

The problem here is that the object fun_encapsulation goes out -of scope when we quit the present function, and therefore also deletes its -pointer to the data which we need to start the new thread. If in this case the -operating system was a bit lazy in creating the new thread, the function -start_thread would at best find a pointer pointing to an object -which is already deleted. Further, but this is obvious, if the function is -taking references or pointers to other objects, it is to be made sure that -these objects persist at least as long as the spawned thread runs. - -

-What one would need to do here at least, is wait until the thread is started -for sure, before deletion of the FunEncapsulation is allowed. To -this end, we need to use a ``Mutex'', to allow for exclusive operations. A -Mutex (short for mutually exclusive) is an object managed by -the operating system and which can only be ``owned'' by one thread at a -time. You can try to ``acquire'' a Mutex, and you can later ``release'' it. If -you try to acquire it, but the Mutex is owned by another thread, then your -thread is blocked until the present owner releases it. Mutices (plural of -``Mutex'') are therefore most often used to guarantee that only one thread is -presently accessing some object: a thread that wants to access that object -acquires a Mutex related to that object and only releases it once the access -if finished; if in the meantime another thread wants to access that object as -well, it has to acquire the Mutex, but since the Mutex is presently owned -already, the second thread is blocked until the first one has finished its -access. - -

-Alternatively, one can use Mutices to synchronize things. We will use it for -the following purpose: the Mutex is acquired by the starting thread; when -later the destructor of the FunEncapsulation class (running on the -same thread) is called, it tries to acquire the lock again; it will thus only -continue its operations once the Mutex has been released by someone, which we -do on the spawned thread once we don't need the data of the -FunEncapsulation object any more and destruction is safe. - -

-All this can then be done in the following way:

Code sample 12 -

-    class FunEncapsulation {
-      public:
-        ...       // as before
-        ~FunEncapsulation ();
-    };
-
-
-    class FunDataBase {
-      public:
-        ...       // as before
-        Mutex       lock;
-    };
-
-    template <typename Class, typename Arg1, typename Arg2>
-    void * start_thread (void *arg_ptr) {
-      MemFunData<Class,Arg1,Arg2> *mem_fun_data
-            = reinterpret_cast<MemFunData *>(arg_ptr);
-
-      // copy the data arguments:
-      MemFunData<Class,Arg1,Arg2>::MemFunPtr
-              mem_fun_ptr = mem_fun_data->mem_fun_ptr;
-      Class * object      = mem_fun_data->object;
-      Arg1    arg1        = mem_fun_data->arg1;
-      Arg2    arg2        = mem_fun_data->arg2;
-
-      // data is now copied, so the original object may be deleted:
-      mem_fun_data->lock.release ();
-
-      // now call the thread function:
-      object->*mem_fun_ptr (arg1, arg2);
-
-      return 0;
-    };
-
-
-    FunEncapsulation::~FunEncapsulation () {
-      // wait until the data is copied by the new thread and
-      // `release' is called by `start_thread':
-      fun_data_base->lock.acquire ();
-      // now delete the object which is no more needed
-      delete fun_data_base;
-    };
-
-
-    void spawn (ACE_Thread_Manager  &thread_manager,
-                FunEncapsulation &fun_encapsulation) {
-      // lock the fun_encapsulation object
-      fun_encapsulation.fun_data_base->lock.acquire ();
-      thread_manager.spawn (*fun_encapsulation.fun_data_base
-                                      ->thread_entry_point,
-                            &fun_data_base);
-    };
-

-When we call spawn, we set a lock -on the destruction of the FunEncapsulation object just before we -start the new thread. This lock is only released when inside the new thread -(i.e. inside the start_thread function) all arguments have been -copied to a safe place. Now we have local copies and don't need the ones from the -fun_encapsulation object any more, which we indicate by -releasing the lock. Inside the destructor of that object, we wait until we can -obtain the lock, which is only after it has been released by the newly started -thread; after having waited till this moment, the destruction can go on -safely, and we can exit the function from which the thread was started, if we -like so. - -

-The scheme just described also works if we start multiple threads using only -one object of type FunEncapsulation: -

-    FunEncapsulation 
-        fun_encapsulation = encapsulate (&TestClass::test_function)
-                                  .collect_args(test_object, arg_value);
-    spawn (thread_manager, fun_encapsulation);
-    spawn (thread_manager, fun_encapsulation);
-
-    // quit the present function
-    return;
-

Here, when starting the second thread the spawn function has to wait until the -newly started first thread has released its lock on the object; however, this -delay is small and should not pose a noticeable problem. Thus, no special -treatment of this case is necessary, and we can in a simple way emulate the -spawn_n function provided by most operating systems, which spawns -several new threads at once: -

-    void spawn_n (ACE_Thread_Manager &thread_manager,
-                  FunEncapsulation   &fun_encapsulation,
-                  const unsigned int  n_threads) {
-      for (unsigned int i=0; i<n_threads; ++i)
-        spawn (thread_manager, fun_encapsulation);
-    };
-

A direct support of the spawn_n function of the operating system -would be difficult, though, since each of the new threads would call -lock.release(), even though the lock was only acquired once. - -

-Since we have now made sure that objects are not deleted too early, even the -following sequence is possible, which does not involve any named variables at -all, only a temporary one, which immediately released after the call to -spawn:

Code sample 13 -

-    spawn (thread_manager, 
-           encapsulate (&TestClass::test_function)
-              .collect_args(test_object, arg_value));
-

-We most often use this very short idiom in the applications in Section -4 and in our own programs. - -

- -

-Number of parameters. Non-member functions. -

-Above, we have explained how we can define classes for a binary member -function. This approach is easily extended to member functions taking any -number of parameters. We simply have to write classes MemFunData0, -MemFunData1, and so on, which encapsulate member functions that take -zero, one, etc parameters. Likewise, we have to have classes -ArgCollectorN for each number of parameters, and functions -encapsulate that return an object of type -ArgCollectorN. Since functions can be overloaded on their argument -types, we need not call the encapsulate functions differently. - -

-All of which has been said above can also easily be adopted to global -functions or static member functions. Instead of the classes -MemFunDataN we can then use classes FunDataN that are also -derived from FunDataBase. The respective ArgCollector -classes then collect only the arguments, not the object on which we will -operate. The class, FunEncapsulation is not affected by this, nor is -FunDataBase. - -

- - -

- -

-
- Next: Applications - Up: Creating and managing threads - Previous: First approach - -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node8.html b/deal.II/doc/reports/multithreading/node8.html deleted file mode 100644 index 95cdde1e4f..0000000000 --- a/deal.II/doc/reports/multithreading/node8.html +++ /dev/null @@ -1,76 +0,0 @@ - - - - -Applications - - - - - - - - - - - - - - -

- -

-
- Next: Writing output detached to - Up: No Title - Previous: Second approach -
-
- - -

- -
-Applications -

- -

-In the next few subsections, we will show usual applications of multi-threading -in the deal.II library. The programs already use the new scheme -discussed in Section 3.4. - -

- - -

- -

-Wolfgang Bangerth -
2000-04-20 -

- - diff --git a/deal.II/doc/reports/multithreading/node9.html b/deal.II/doc/reports/multithreading/node9.html deleted file mode 100644 index 16c029dc59..0000000000 --- a/deal.II/doc/reports/multithreading/node9.html +++ /dev/null @@ -1,169 +0,0 @@ - - - - -Writing output detached to disk - - - - - - - - - - - - - - -

- -

-
- Next: Assembling the matrix - Up: Applications - Previous: Applications -
-
- - -

-Writing output detached to disk -

- -

-The output classes, i.e. basically the classes DataOut and -DataOutStack and their base classes, follow a strictly hierarchical -model of data flow. The two terminal classes know about such things as -triangulations, degrees of freedom, or finite elements, but they translate -this structured information into a rather simple intermediate format. This -conversion is done in the build_patches functions of these -classes. The actual output routines only convert this intermediate format into -one of the supported graphics formats, which is then a relatively simple task. - -

-This separation of processing of structured data and actual output of the -intermediate format was chosen since the actual output routines became rather -complex with growing scope of the whole library. For example, we had to update -all output functions when vector-valued finite elements were supported, and we -had to do so again when discontinuous elements were developed. This became an -unmanageable burden with the growing number of output formats, and we decided -that an intermediate format would be more appropriate, which is created by -only one function, but can be written to output formats by a number of -different functions. - -

-In the present context, this has the following implications: once the -intermediate data is created by the build_patches function, we need -no more preserve the data from which it was made (i.e. the grid which it was -computed on, or the vector holding the actual solution values) and we can go -on with computing on the next finer grid, or the next time step, while the -intermediate data is converted to a graphics format file detached from the -main process. The only thing which we must make sure is that the program only -terminates after all detached output threads are finished. This can be done in -the following way: -

-    // somewhere define a thread manager that keeps track of all
-    // detached (`global') threads
-    ACE_Thread_Manager global_thread_manager;
-
-    // This is the class which does the computations:
-    class MainClass {
-        ...
-
-        // now two functions, the first is called from the main program
-        // for output, the second will manage detached output
-        void write_solution ();
-        void write_detached (DataOut<dim> *data_out);
-    };
-
-
-    void MainClass::write_solution () {
-      DataOut<dim> *data_out = new DataOut<dim>();
-
-      // attach DoFHandler, add data vectors, ...
-     
-      data_out->build_patches ();
-
-      // now everything is in place, and we can write the data detached
-      // Note that we transfer ownership of `data_out' to the other thread
-      Threads::spawn (global_thread_manager,
-                      Threads::encapsulate(&MainClass<dim>::write_detached)
-                          .collect_args(this, data_out));
-    };
-
-    
-    void MainClass::write_detached (DataOut<dim> *data_out) {
-      ofstream output_file ("abc");
-      data_out->write_gnuplot (output_file); 
-
-      // now delete the object which we got from the starting thread
-      delete data_out;
-    };
-
-
-    int main () {
-      ...  // do all the work
-   
-      // now wait for all detached threads to finish
-      global_thread_manager.wait ();
-    };
-

-Note that the functions spawn and encapsulate are prefixed -by Threads:: since in the actual implementation in deal.II -they are declared within a namespace of that name. - -

-It should be noted that if you want to write output detached from the main -thread, and from the main thread at the same time, you need a version of the -C++ standard library delivered with your compiler that supports parallel -output. For the GCC compiler, this can be obtained by configuring it with the -flag -enable-threads at build time, or by using GCC version 3.0 or -later. - -

- - -

- -

-
- Next: Assembling the matrix - Up: Applications - Previous: Applications - -

-Wolfgang Bangerth -
2000-04-20 -

- -