Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions .github/workflows/dependencies/documentation.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
#
# Copyright 2020 The AMReX Community
#
# License: BSD-3-Clause-LBNL
# Authors: Andrew Myers

set -eu -o pipefail

# `man apt.conf`:
# Number of retries to perform. If this is non-zero APT will retry
# failed files the given number of times.
echo 'Acquire::Retries "3";' | sudo tee /etc/apt/apt.conf.d/80-retries

sudo apt-get update

sudo apt-get install -y --no-install-recommends\
build-essential \
pandoc \
doxygen \
texlive \
texlive-latex-extra \
texlive-lang-cjk \
tex-gyre \
latexmk

30 changes: 30 additions & 0 deletions .github/workflows/docs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
name: Build and Deploy

on:
push:
branches-ignore:
- 'dependabot/**'
pull_request:

concurrency:
group: ${{ github.ref }}-${{ github.head_ref }}-docs
cancel-in-progress: true

jobs:
build-and-deploy:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v6
with:
persist-credentials: false

- name: Dependencies
run: |
.github/workflows/dependencies/documentation.sh
python3 -m pip install --upgrade pip
python3 -m pip install sphinx==5.0.0 sphinx_rtd_theme

- name: Install and Build
run: |
./build_docs.sh
1 change: 1 addition & 0 deletions Docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Online documentation is available at https://amrex-codes.github.io/amrex/docs_html/.
51 changes: 51 additions & 0 deletions Docs/Readme.sphinx
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
To update the online AMReX documentation using sphinx:

1. Checkout the development branch of AMReX

2. Go to amrex/Docs/sphinx_documentation/source and edit the relevant .rst file(s).

3. Commit your changes.

The GitHub action script will build the html pages and commit them to AMReX's gh-pages repo for you. If you would like to
preview the html locally, you can:

4. Go back into amrex/Docs/sphinx_documentation.

5. Type "make html". This will build new html files in amrex/Docs/sphinx_documentation/build/html

**********************************************************************************

If you would like to build the web pages locally, you will need Python, the Sphinx software, and the "Read the Docs" theme.

If you have conda, you can install the necessary packages as follows:

1. Type "conda install sphinx"

2. Type "conda install sphinx_rtd_theme"

If you don't have a conda Python installation, you get one by doing the following:

1. Go to https://conda.io/miniconda.html and download the Python 3.6 64-bit (bash installer), "Miniconda3-latest-Linux-x86_64.sh". Save this script to your hard drive.

2. Type "bash Miniconda3-latest-Linux-x86_64.sh" and follow the installation prompts.

3. The install script will prompt you to add some commands to your .bashrc that add the miniconda install location to your PATH environment variable.
If Bash is your default shell, choose "yes". Otherwise, you will need to manually do the same for your shell of choice.

4. Either open a new terminal, or re-source the configuration file you just added to. E.g., for bash:
source ~/.bashrc

You should now be able to successfully "make html" in amrex/Docs/sphinx_documentation.

If you would like to make a pdf document from the *rst files, first

sudo apt-get install latexmk
(if you are using macOS, latexmk is installed via the TexLive Utility)

Then in the amrex/Docs/sphinx_documentation directory, type

make latexpdf
(if you have the slimmed-down latex install, then you need to also install the following packages via TexLive Utility: tabulary, varwidth, framed, wrapfig, capt-of, needspace, courier)


This will create amrex/Docs/sphinx_documentation/build/latex/amrex.pdf
26 changes: 26 additions & 0 deletions Docs/sphinx_documentation/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Minimal makefile for Sphinx documentation
#

# You can set these variables from the command line.
PYTHON ?= python
SPHINXOPTS =
SPHINXBUILD = $(PYTHON) -msphinx
SPHINXPROJ = amrex
SOURCEDIR = source
BUILDDIR = build

# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

.PHONY: help Makefile

clean:
rm -rf build ../Doxygen/xml source/class source/file
rm -f source/classlist.rst source/filelist.rst
rm -rf source/*_files.rst

# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
56 changes: 56 additions & 0 deletions Docs/sphinx_documentation/source/LoadBalancing.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
.. role:: cpp(code)
:language: c++

.. role:: fortran(code)
:language: fortran

.. _sec:load_balancing:

Load Balancing
--------------

The process of load balancing is typically independent of the process of grid creation;
the inputs to load balancing are a given set of grids with a set of weights
assigned to each grid. (The exception to this is the KD-tree approach in which the
grid creation process is governed by trying to balance the work in each grid.)

Single-level load balancing algorithms are sequentially applied to each AMR level independently,
and the resulting distributions are mapped onto the ranks taking into account the weights
already assigned to them (assign heaviest set of grids to the least loaded rank). Note that the
load of each process is measured by how much memory has already been allocated, not how much memory
will be allocated. Therefore the following code is not recommended because it tends to generate
non-optimal distributions.

.. highlight:: c++

::

for (int lev = 0; lev < nlevels; ++lev) {
// build DistributionMapping for Level lev
}
for (int lev = 0; lev < nlevels; ++lev) {
// build MultiFabs for Level lev
}

Instead, one should do,

.. highlight:: c++

::

for (int lev = 0; lev < nlevels; ++lev) {
// build DistributionMapping for Level lev
// build MultiFabs for Level lev
}

Distribution options supported by AMReX include the following; the default is SFC:

- Knapsack: the default weight of a grid in the knapsack algorithm is the number of grid cells,
but AMReX supports the option to pass an array of weights -- one per grid -- or alternatively
to pass in a MultiFab of weights per cell which is used to compute the weight per grid.

- SFC: enumerate grids with a space-filling Z-morton curve, then partition the
resulting ordering across ranks in a way that balances the load.

- Round-robin: sort grids and assign them to ranks in round-robin fashion -- specifically
FAB i is owned by CPU i%N where N is the total number of MPI ranks.
43 changes: 43 additions & 0 deletions Docs/sphinx_documentation/source/ManagingGridHierarchy_Chapter.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
.. role:: cpp(code)
:language: c++

.. _Chap:ManagingGridHierarchy:

Gridding and Load Balancing
===========================

AMReX provides a great deal of generality when it comes to how to decompose the
computational domain into individual logically rectangular grids, and how to distribute
those grids to MPI ranks. We use the phrase "load balancing" here to refer to the combined process
of grid creation (and re-creation when regridding) and distribution of grids to MPI ranks.

Even for single-level calculations, AMReX provides the flexibility to have different size grids,
more than one grid per MPI rank, and different strategies for distributing the grids to MPI ranks.

For multi-level calculations, the same principles for load balancing apply as in single-level calculations,
but there is additional complexity in how to tag cells for refinement and how to create the
union of grids at levels > 0 where that union most likely does not cover the computational domain.

See :ref:`sec:grid_creation` for grids are created, i.e. how the :cpp:`BoxArray` on which
:cpp:`MultiFabs` will be built is defined at each level.

See :ref:`sec:load_balancing` for the strategies AMReX supports for distributing
grids to MPI ranks, i.e. defining the :cpp:`DistributionMapping` with which
:cpp:`MultiFabs` at that level will be built.

We also note that we can create separate grids, and map them in different ways to MPI ranks, for
different types of data in a single calculation. We refer to this as the "dual grid approach"
and the most common usage is to load balance mesh and particle data separately. See :ref:`sec:dual_grid`
for more about this approach.

When running on multicore machines with OpenMP, we can also control the distribution of
work by setting the size of grid tiles (by defining :cpp:`fabarray_mfiter.tile_size`), and if relevant, of
particle tiles (by defining :cpp:`particle.tile_size`). We can also specify the strategy for assigning
tiles to OpenMP threads. See :ref:`sec:basics:mfiter:tiling` for more about tiling.

.. toctree::
:maxdepth: 1

GridCreation
DualGrid
LoadBalancing
Loading