The major changes of this release are:
%
\begin{itemize}
- \item
+ \item Substantial improvements to the multigrid two-level transfer
+ operators, including matrix-free based versions and versions for
+ p-refinement (Section~\ref{sec:mf}).
+ \item A class FERemoteEvaluation that provides support for evaluating
+ finite element shape functions and solutions on parts of the
+ domain stored by other MPI processes (Section~\ref{sec:remote-eval}).
+ \item Much work has gone into writing wrappers for Trilinos' Tpetra
+ stack of linear algebra classes. Tpetra is Trilinos' Kokkos-based
+ replacement for the now-deprecated Epetra stack. See
+ Section~\ref{sec:tpetra} for more on this.
+ \item Tool classes \texttt{TaskResult} and \texttt{Lazy} that provide ways to compute
+ values on a separate thread or only when first accessed (Section~\ref{sec:tools}).
+ \item There are six new tutorial programs, on checkpointing simulations
+ (step-83), integrating time-dependent solvers with external time
+ stepping libraries (step-86, using PETSc's TS library), advanced
+ point evaluation techniques (step-87), non-matching grids
+ (step-89), and trace-based methods for PDEs on embedded surfaces
+ (step-90). See Section~\ref{subsec:steps} for more details.
\end{itemize}
%
+\todo[inline]{If we keep the section on relicensing
+ (Section~\ref{sec:license}), mention it in the list above.}
+
While all of these major changes are discussed in detail in
Section~\ref{sec:major}, there
are a number of other noteworthy changes in the current \dealii release,
which we briefly outline in the remainder of this section:
%
\begin{itemize}
-\item changes of interface of AffineConstraints
+ \item deal.II now requires and uses of C++17.
+ \item We have continued to make progress in supporting simplex and
+ mixed meshes -- mesh types that \dealii{} has traditionally not
+ supported at all. Specifically, the current release uses
+ better strategies for refinement of tetrahedra that result in
+ better-shaped child cells. It also contains
+ support for cubic finite elements on simplices.
+ \item The \texttt{FE\_NedelecSZ} class that contains our
+ implementation of the N\'ed\'elec element using the orientation
+ scheme of \cite{Zag06} now supports the computation of hanging
+ node constraints for locally refined meshes.
+ \todo[inline]{Sebastian: Is there anything more to say here? If
+ you want, we can also list this point in the list above, and add
+ a whole section 2.x for it.}
+ \item changes of interface of AffineConstraints
+ \todo[inline]{Wolfgang to write}
\item clean up: orientation
-\item various performance improvements throughout the library
-\item the \texttt{SolverGMRES} class now offers a third orthogonalization
+ \todo[inline]{DavidW: Please describe some details here.}
+\item Various performance improvements throughout the library.
+ \todo[inline]{Everyone: This sentence would be more useful if we can
+ provide a few examples.}
+\item The \texttt{SolverGMRES} class now offers a third orthogonalization
method, the classical Gram--Schmidt method with delayed orthogonalization
\cite{Bielich2022}. Furthermore, the solver now specifies the maximal basis
- size of the Arnoldi basis, rather than the number of auxiliary vectors.
+ size of the Arnoldi basis, rather than the number of auxiliary
+ vectors. Some changes have also been made to the GMRES and F-GMRES
+ implementations to make them more similar.
+ \todo[inline]{Martin: Does this capture the relevant changes? Please
+ fix as appropriate, or if you'd like put them into a section of
+ its own.}
\end{itemize}
%
-The changelog lists more than X other features and bugfixes.
+The changelog lists more than 180 other features and bugfixes.
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\label{sec:major}
This release of \dealii contains a number of large and significant changes,
-which will be discussed in this section.
+which we will discuss in this section.
It of course also includes a
vast number of smaller changes and added functionality; the details of these
can be found
-\href{https://dealii.org/developer/doxygen/deal.II/changes_between_9_4_2_and_9_5_0.html}
-{in the file that lists all changes for this release}; see \cite{changes95}.
+\href{https://dealii.org/developer/doxygen/deal.II/changes_between_9_5_2_and_9_6_0.html}
+{in the file that lists all changes for this release}; see \cite{changes96}.
-\begin{itemize}
-\item Hermite element
-\item simplices: cubic simplex elements, improved refinement of tetrahedron cells
-\item Tpetra advances
-\item Nedelec for meshes with hanging nodes
-\end{itemize}
-
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\subsection{Relicensing to Apache 2.0}\label{sec:license}
-
+\subsection{Updates to multigrid and matrix-free algorithms}\label{sec:mf}
-
-%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\subsection{Updates to matrix-free algorithms}\label{sec:mf}
+\todo[inline]{Martin/Peter: The list in the introduction also mentions the two-level
+ operators. Is this the right place to also discuss these?}
\begin{itemize}
\item VectorizedArray for ARM Neon
in several scenarios, especially multi-component systems.
\end{itemize}
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\subsection{The \texttt{FERemoteEvaluation} class}\label{sec:remote-eval}
+
+\todo[inline]{Peter/Martin/...: Would you be willing to describe this
+ class and its uses? Or, if you don't think it's important enough,
+ remove this section and move the part in the top list of major
+ changes to the bottom list?}
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{Advances in non-matching support}\label{sec:nonmatching}
+\todo[inline]{All: If you contributed to this functionality, please
+ help flesh out this section.}
+
\begin{itemize}
\item non-nested multigrid: new features and performance improvement
\item FERemoteEvaluation
\end{itemize}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
-\subsection{Interface to the \trilinos \tpetra stack}\label{sec:tpetra}
+\subsection{Interface to the \trilinos{} \tpetra stack}\label{sec:tpetra}
+\dealii{}'s parallel linear algebra facilities are largely built on
+wrappers around functionality provided by the \petsc{} and \trilinos{} libraries.
As a software project, \trilinos{} is designed as a stack of semi-independent
packages building upon each other and working together.
As a foundation, \teuchos{} offers a set of basic tools,
as well as more advanced techniques like automatic differentiation,
time integration and discretization.
-For now, there are two choices for the second layer
-with \epetra{} providing \MPI{} only parallelism
-and the newer \tpetra{} (templated Petra) providing additional shared memory
-parallelism and GPU capabilities through \kokkos{}, referred to as \MPIx.
+Historically, \trilinos{} has implemented \petra{} objects -- such as
+parallel vector, sparsity pattern, and sparse matrix classes -- in its
+\epetra{} package that uses \MPI{} as its only source of
+parallelism. The \dealii{} interfaces to \trilinos{} are therefore all
+built using \epetra{}.
+However, several years ago, \trilinos also introduced the
+newer \tpetra{} (templated Petra) package that provides additional shared memory
+parallelism and GPU capabilities by building on \kokkos{}. \tpetra{}
+is slated to eventually replace \epetra{}, and the latter is indeed
+now deprecated. As a consequence, we will eventually have to switch
+all of our \trilinos{} interfaces to \tpetra{} and \tpetra{}-based sub-packages.
+
+In the current release, we have put substantial work into this switch.
Since the interface of \tpetra{} was overhauled and modernized
many of the packages in the third layer now have two generations as well.
Therefore, the \MPI-only set of packages is also referred to as the
\begin{itemize}
\item Short overview capabilities of \epetra (MPI parallel, fixed types) [Thiele]
\item Short overview capabilities of \tpetra (MPI+X via Kokkos, auto diff types, Templated petra) [Thiele]
- \item Current state of the interface (Tpertra based linear algebra, direct solver from Amesos2, preconditioners from Ifpack2) [Thiele \& Kinnewig]
+ \item Current state of the interface (Tpetra based linear algebra, direct solver from Amesos2, preconditioners from Ifpack2) [Thiele \& Kinnewig]
\item User relevant changes from \epetra: incompatibilities, removed options, changes in the constructor. [Thiele \& Kinnewig]
\end{itemize}
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\subsection{More support for advanced programming idioms}\label{sec:tools}
+
+\todo[inline]{Wolfgang to write. Mention Lazy and TaskResult.}
+
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
\subsection{New and improved tutorials and code gallery programs}
\label{subsec:steps}
\end{itemize}
Furthermore, we added an example to the \texttt{libCEED}
-library\footnote{\url{https://github.com/CEED/libCEED}}. \texttt{libCEED} is a library
+library.\footnote{\url{https://github.com/CEED/libCEED}} \texttt{libCEED} is a library
that provides matrix-free evaluation routines that work on different hardware. The
examples show how to interface the \dealii data structures to the \texttt{libCEED} ones.
-The BP1-BP6 benchmarks (scalar/vector Laplace/mass matrix are solved with
-regular integration and over-integration) are solved.
+The example solves the BP1-BP6 benchmarks (scalar/vector Laplace/mass matrix with
+regular integration and over-integration).
+
+
+%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
+\subsection{Relicensing to Apache 2.0}\label{sec:license}
+
+\todo[inline]{Matthias: Where are we with this, and what of it is
+ worth describing here?}
+
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
The 9.6 release includes
\href{https://dealii.org/developer/doxygen/deal.II/changes_between_9_5_2_and_9_6_0.html}
- {around X incompatible changes};
+ {around 40 incompatible changes};
see \cite{changes96}. Many of these
incompatibilities change internal
interfaces that are not usually used in external
applications. That said, the following are worth mentioning since they
may have been more widely used:
\begin{itemize}
- \item CUDAWrappers, that enable the usage of cuSPARSE algorithms, have been deprecated and will be removed in the next release. Kokkos is now used for device-specific optimizations.
+ \item \dealii{} now requires compilers to support C++17, and has
+ started to extensively use C++17 features.
+ \item The \texttt{CUDAWrappers} namespace and its contents --
+ notably things that enable the usage of cuSPARSE algorithms --
+ have been deprecated and will be removed in the next
+ release. Kokkos is now used for device-specific optimizations.
\end{itemize}