ulfm readme: review comments from Jeff

Signed-off-by: Aurelien Bouteiller <[email protected]>

Author: abouteiller

docs/features/ulfm.rst

@@ -16,7 +16,7 @@ This is an extremely terse summary of how to use ULFM:
   shell$ ./configure --with-ft=ulfm [...options...]
   shell$ make [-j N] all install
   shell$ mpicc my-ft-program.c -o my-ft-program
-  shell$ mpiexec -n 4 --with-ft ulfm my-ft-program
+  shell$ mpirun -n 4 --with-ft ulfm my-ft-program
 
 Features
 --------
@@ -144,14 +144,15 @@ Running your application
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
 You can launch your application with fault tolerance by simply using
-the normal Open MPI ``mpiexec`` launcher, with the
+the normal Open MPI ``mpirun`` launcher, with the
 ``--with-ft ulfm`` CLI option (or its synonym ``--with-ft mpi``):
 
 .. code-block::
 
    shell$ mpirun --with-ft ulfm ...
 
-  .. important:: by default, fault tolerance is not active.
+  .. important:: By default, fault tolerance is not active at run time.
+                 It must be enabled via `--with-ft ulfm`.
 
 Running under a batch scheduler
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
@@ -160,14 +161,14 @@ ULFM can operate under a job/batch scheduler, and is tested routinely
 with ALPS, PBS, and Slurm. One difficulty comes from the fact that
 many job schedulers will "cleanup" the application as soon as any
 process fails. In order to avoid this problem, it is preferred that
-you use ``mpiexec`` within an allocation (e.g., ``salloc``,
+you use ``mpirun`` within an allocation (e.g., ``salloc``,
 ``sbatch``, ``qsub``) rather than a direct launch (e.g., ``srun``).
 
 * SLURM is tested and supported with fault tolerance.
 
   .. important:: Do not use ``srun``, or your application gets killed
                  by the scheduler upon the first failure.  Instead,
-                 use ``mpirun`` in an ``salloc/sbatch`` allocation.
+                 use ``mpirun`` in an ``salloc`` or ``sbatch`` allocation.
 
 * LSF is untested with fault tolerance.
 
@@ -186,8 +187,8 @@ errmgr_detector_bar <value>`` for PRTE options.
 
   .. important:: The main control for enabling/disabling fault tolerance
                  at runtime is the ``--with-ft ulfm`` (or its synomym
-                 ``--with-ft mpi``) ``mpiexec`` CLI option. This option
-                 setup multiple subsystems of Open MPI to enable fault
+                 ``--with-ft mpi``) ``mpirun`` CLI option. This option
+                 sets up multiple subsystems in Open MPI to enable fault
                  tolerance. The options described below are best used to
                  overide the default behavior after the ``--with-ft ulfm``
                  opion is used.
@@ -197,7 +198,7 @@ PRTE level options
 
 * ``prrte_enable_ft <true|false> (default: false)`` controls
   automatic cleanup of apps with failed processes within
-  mpirun. This option is automatically set to ``true`` when using
+  ``mpirun``. This option is automatically set to ``true`` when using
   ``--with-ft ulfm``.
 * ``errmgr_detector_priority <int> (default 1005``) selects the
   PRRTE-based failure detector. Only available when
@@ -216,17 +217,17 @@ PRTE level options
 Open MPI level options
 ~~~~~~~~~~~~~~~~~~~~~~
 
-Some default values are applied to some Open MPI parameters when using
-``mpiexec --with-ft ulfm``. These defaults are obtained from the ``ft-mpi``
+Default values are applied to some Open MPI parameters when using
+``mpirun --with-ft ulfm``. These defaults are obtained from the ``ft-mpi``
 aggregate MCA param file
 ``$installdir/share/openmpi/amca-param-sets/ft-mpi``. You can tune the
-runtime behavior with ULFM by either setting or unsetting variables in
+runtime behavior of ULFM by either setting or unsetting variables in
 this file, or by overiding the variable on the command line (e.g.,
 ``--mca btl ofi,self``).
 
   .. important:: Note that if fault tolerance is disabled at runtime,
-                 that is, when not using ``--with-ft ulfm``), the
-                 ``ft-mpi`` MCA param file is not loaded, thus
+                 (that is, when not using ``--with-ft ulfm``), the
+                 ``ft-mpi`` AMCA param file is not loaded, thus
                  components that are unsafe for fault tolerance will
                  load normally (this may change observed performance
                  when comparing with and without fault tolerance).
@@ -260,16 +261,16 @@ this file, or by overiding the variable on the command line (e.g.,
   latency (typically 1us increase).* You may want to **enable this
   option if you experience false positive** processes incorrectly
   reported as failed with the Open MPI failure detector.
-  This option is only relevant when `mpi_ft_detector` is `true`.
+  This option is only relevant when ``mpi_ft_detector`` is ``true``.
 * ``mpi_ft_detector_period <float> (default: 3e0 seconds)`` heartbeat
   period. Recommended value is 1/3 of the timeout. _Values lower than
   100us may impart a noticeable effect on latency (typically a 3us
   increase)._
-  This option is only relevant when `mpi_ft_detector` is `true`.
+  This option is only relevant when ``mpi_ft_detector`` is ``true``.
 * ``mpi_ft_detector_timeout <float> (default: 1e1 seconds)`` heartbeat
   timeout (i.e. failure detection speed). Recommended value is 3 times
   the heartbeat period.
-  This option is only relevant when `mpi_ft_detector` is `true`.
+  This option is only relevant when ``mpi_ft_detector`` is ``true``.
 
 Known Limitations in ULFM
 -------------------------
@@ -282,24 +283,20 @@ Known Limitations in ULFM
 Modified, Untested and Disabled Components
 ------------------------------------------
 
-Frameworks and components which are not listed in the following list
-are unmodified and support fault tolerance. Listed frameworks may be
-**modified** (and work after a failure), **untested** (and work before
-a failure, but may malfunction after a failure), or **disabled** (they
-cause unspecified behavior all around when FT is enabled).
+Frameworks and components are listed below and categorized into one of
+three classifications:
 
-All runtime disabled components are listed in the ``ft-mpi`` aggregate
-MCA param file
-``$installdir/share/openmpi/amca-param-sets/ft-mpi``. You can tune the
-runtime behavior with ULFM by either setting or unsetting variables in
-this file (or by overiding the variable on the command line (e.g.,
-``--mca btl ofi,self``).
+1. **Modified:** This framework/component has been specifically modified
+                 such that it will continue to work after a failure.
+2. **Untested:** This framework/component has not been modified and/or
+                 tested with fault tolerance scenarios, and _may_ malfunction
+                 after a failure.
+3. **Disabled:** This framework/component will cause unspecified behavior when
+                 fault tolerance is enabled.
 
-  .. important:: Note that if fault tolerance is disabled at runtime,
-                 the ``ft-mpi`` MCA param file is not loaded, thus
-                 components that are unsafe for fault tolerance will
-                 load normally (this may change observed performance
-                 when comparing with and without fault tolerance).
+Any framework or component not listed below are categorized as **Unmodified**,
+meaning that it is unmodified for fault tolerance, but will continue to work
+correctly after a failure.
 
 * ``pml``: MPI point-to-point management layer
 
@@ -343,8 +340,7 @@ this file (or by overiding the variable on the command line (e.g.,
 
 * ``vprotocol``: Checkpoint/Restart components
 
-  * These components have not been modified to handle faults, and are
-    **untested**.
+  * All ``vprotocol`` components are **untested**
 
 * ``threads``, ``wait-sync``: Multithreaded wait-synchronization
   object