Parallelization and localwork
=============================

Details on the parallelization of MW, in particular of the long-range, short-range interactions is given 
in the pdf file *./doc/parallelization/parallelization.pdf*.
Here we give some explanation about the parallelization information that is printed in the *run.out* file.

### Ewald parameters

```
ewald
-----
|ewald| alpha:                                 6.76064E-02
|ewald| real space summation tolerance:        1.00000E-05
|ewald| real space cut-off distance:           4.12900E+01
|ewald| erfc(alpha*|rcut|)/|rcut|:             1.91073E-06
|ewald| exp(-alpha^2*|rcut|^2)/|rcut|:         1.00000E-05
|ewald| reciprocal space summation tolerance:        1.00000E-07
|ewald| reciprocal space cut-off distance |kcut|^2:  5.42844E-01
|ewald| exp(-|kcut|^2/(4*alpha^2))/|kcut|^2:         3.39352E-07
|ewald| reciprocal space image count (kmax):             7            7           30
|ewald| Z integral points:                              61
|ewald| Z integral range:                     -5.44027E-01  5.44027E-01
|ewald| Z integral weights:                    1.78370E-02
|ewald| Short-Range interaction matrices:
|ewald| ion <-> ion
|ewald| X X X X X X
|ewald| X X X X X X
|ewald| X X X X X X
|ewald| X X X X X X
|ewald| X X X X X X
|ewald| X X X X X X
|ewald| electrode <-> electrode
|ewald| X O
|ewald| O X
|ewald| ion -> electrode
|ewald| X X
|ewald| X X
|ewald| X X
|ewald| X X
|ewald| X X
|ewald| X X
|ewald| electrode -> ion
|ewald| X X X X X X
|ewald| X X X X X X
```

The short-range interaction matrices indicates whether electrostatic interaction should be computed between "ion" and electrode species. 
Each row/column corresponds to a species (in the same order as given in the input file) and a X means that there will be interaction 
while a O means there will not be interaction. 
If one of the species is mobile, the code assumes there will be interaction. 
If both species are fixed, the code computes the minimum distance between all particles of the 2 species and if this distance is 
greater than the real space cut-off, then the computation is ommitted because the contribution will always be 0. 
In the example above the 2 electrodes are too far apart to interact with each other.

#### Local work assignment

The work is distributed among the different processors. The localwork section details what is assigned to the process doing 
the output (rank 0 here). Each process will output different values in this section.

The k-mode assignment corresponds to the work distribution for the reciprocal space Ewald summation. 
The outer loop on the global number of k-mode is distributed among the different process, each process working on a local 
number of k-mode. The start k-mode index gives the triplet (kx, ky, kz) of the first k-mode for the local process. 
The loop on kz is the fastest and the loop on kx is the slowest.

The pair interaction (short-range interactions) are distributed by blocks. Since the pair interactions are symmetric, 
only half of them are distributed. In the output we distinguish two types of blocks. 
Diagonal blocks are blocks on the diagonal of the interaction matrix. Only half of the interaction are taken into account 
in these blocks. Full blocks are off-diagonal blocks. Each table gives the number of blocks assigned to the process and the 
offset of the first block in the global matrix for a pair of species types.

Some computations are done on vector of species or molecules. Hence the vectors are distributed among the processes. 
The distribution is done by type, for each type the output gives the number of local species/molecules and the offset 
in the vector of the first species/molecules.

```
local work
----------
|localwork| k-modes assignment
|localwork|    number of global k-modes:   6832
|localwork|    number of local  k-modes:      9
|localwork|    start k-mode index      :      0,      1,    -30

|localwork| electrode to electrode pair diagonal block assignment:
|localwork| itype |  count | offset |
|localwork|     1 |      1 |      0 |
|localwork|     2 |      0 |      0 |

|localwork| electrode to electrode pair block assignment:
|localwork| itype | jtype |  count | offset |
|localwork|     1 |     1 |     17 |   2645 |
|localwork|     1 |     2 |     35 |   4445 |
|localwork|     2 |     1 |      0 |      0 |
|localwork|     2 |     2 |     17 |   5841 |

|localwork| ion to ion pair diagonal block assignment:
|localwork| itype |  count | offset |
|localwork|     1 |      1 |      0 |
|localwork|     2 |      1 |    109 |
|localwork|     3 |      1 |    218 |
|localwork|     4 |      0 |      0 |
|localwork|     5 |      0 |      0 |
|localwork|     6 |      0 |      0 |

|localwork| ion to ion pair block assignment:
|localwork| itype | jtype |  count | offset |
|localwork|     1 |     1 |     98 |  29835 |
|localwork|     1 |     2 |    197 |  51417 |
|localwork|     1 |     3 |    197 |  47477 |
|localwork|     1 |     4 |      3 |    955 |
|localwork|     1 |     5 |      3 |   1222 |
|localwork|     1 |     6 |     10 |    130 |
|localwork|     2 |     1 |      0 |      0 |
|localwork|     2 |     2 |     98 |  27875 |
|localwork|     2 |     3 |    197 |  52008 |
|localwork|     2 |     4 |      4 |     84 |
|localwork|     2 |     5 |      4 |    440 |
|localwork|     2 |     6 |      9 |   2621 |
|localwork|     3 |     1 |      0 |      0 |
|localwork|     3 |     2 |      0 |      0 |
|localwork|     3 |     3 |     98 |  28169 |
|localwork|     3 |     4 |      4 |    728 |
|localwork|     3 |     5 |      3 |   1036 |
|localwork|     3 |     6 |      9 |   1550 |
|localwork|     4 |     1 |      0 |      0 |
|localwork|     4 |     2 |      0 |      0 |
|localwork|     4 |     3 |      0 |      0 |
|localwork|     4 |     4 |      0 |      0 |
|localwork|     4 |     5 |      0 |      0 |
|localwork|     4 |     6 |      1 |     40 |
|localwork|     5 |     1 |      0 |      0 |
|localwork|     5 |     2 |      0 |      0 |
|localwork|     5 |     3 |      0 |      0 |
|localwork|     5 |     4 |      0 |      0 |
|localwork|     5 |     5 |      0 |      0 |
|localwork|     5 |     6 |      0 |      0 |
|localwork|     6 |     1 |      0 |      0 |
|localwork|     6 |     2 |      0 |      0 |
|localwork|     6 |     3 |      0 |      0 |
|localwork|     6 |     4 |      0 |      0 |
|localwork|     6 |     5 |      0 |      0 |
|localwork|     6 |     6 |      0 |      0 |

|localwork| electrode to ion pair block assignment:

|localwork| itype | jtype |  count | offset |
|localwork|     1 |     1 |     83 |      0 |
|localwork|     1 |     2 |     83 |   9130 |
|localwork|     1 |     3 |     82 |  18177 |
|localwork|     1 |     4 |      1 |    521 |
|localwork|     1 |     5 |      1 |    523 |
|localwork|     1 |     6 |      4 |   1336 |
|localwork|     2 |     1 |     82 |  20391 |
|localwork|     2 |     2 |     82 |  29411 |
|localwork|     2 |     3 |     83 |   6889 |
|localwork|     2 |     4 |      2 |    278 |
|localwork|     2 |     5 |      2 |    282 |
|localwork|     2 |     6 |      3 |   1468 |

|localwork| ion to electrode pair block assignment:
|localwork| itype | jtype |  count | offset |
|localwork|     1 |     1 |     83 |      0 |
|localwork|     1 |     2 |     83 |   1328 |
|localwork|     2 |     1 |     82 |  20391 |
|localwork|     2 |     2 |     82 |  21703 |
|localwork|     3 |     1 |     83 |   9130 |
|localwork|     3 |     2 |     83 |  10458 |
|localwork|     4 |     1 |      1 |    548 |
|localwork|     4 |     2 |      1 |    564 |
|localwork|     5 |     1 |      2 |    332 |
|localwork|     5 |     2 |      2 |    364 |
|localwork|     6 |     1 |      3 |   1420 |
|localwork|     6 |     2 |      3 |   1468 |

|localwork| electrode vector assignment
|localwork|    number of global electrode atoms:   7298
|localwork|    number of local electrode atoms:     20
|localwork|    electrode atoms local offset:          0,   5559
|localwork|    electrode atoms local count:          10,     10

|localwork| ion vector assignment
|localwork|    number of global ions:  27120
|localwork|    number of local ions:     71
|localwork|    ions local offset:          0,   9536,  19072,  26496,      0,  26896,
|localwork|    ions local count :         23,     23,     23,      1,      0,      1,

|localwork| molecules vector assignment
|localwork|    number of global molecules:   8800
|localwork|    number of local molecules:     23
|localwork|    molecules local istart:          1
|localwork|    molecules local iend:           23
```
