Introduce create_identity_partitioner for refined mesh partitioning

[schnellerhase]

Introduces create_identity_partitioner which facilitates the exchange of ghost cells of a refined mesh whilst maintaining the partition of the coarse mesh. This introduces the necessity of a new sentinel variable redistribute for the python refine functionality to distinguish between the C++ cases of nullptr and nullopt correctly.

This fixes (partially) #3443.

Additionally, this changes

1. move compute_destination_ranks from anonymous namespace to dolfinx::graph,
2. python dolfinx.mesh.refine 's partitioner default argument value was not aligning with cpp layer, fixed,
3. default argument for refine's option arg changed from Option::none to Option::parent_cell,
4. partitioner argument of refine is now std::optional, allows for handling of case of not provided (which is different from providing nullptr as callable).

[schnellerhase]

Verification code:

import febug
import pyvista
import dolfinx
import dolfinx.mesh
from mpi4py import MPI

mesh = dolfinx.mesh.create_unit_square(MPI.COMM_WORLD, 4, 4)
mesh.topology.create_entities(2 - 1)
mesh.topology.create_connectivity(2 - 1, 2)
mesh.topology.create_entities(1)
[refined_mesh, _, _] = dolfinx.mesh.refine(
    mesh, option=dolfinx.mesh.RefinementOption.parent_cell
)
subplotter = pyvista.Plotter(shape=(2, 1))

subplotter.subplot(0, 0)
febug.plot_mesh(mesh, plotter=subplotter)

subplotter.subplot(1, 0)
febug.plot_mesh(refined_mesh, plotter=subplotter)
# subplotter.show()
subplotter.save_graphic(f"out_{MPI.COMM_WORLD.rank}.pdf")

Output ($np=2$):
out_0.pdf
out_1.pdf

Output ($np=3$):
out_0.pdf
out_1.pdf
out_2.pdf

[schnellerhase]

Tests fail due to refinement routine now not being able to handle the case of no partitioner and no parent_cell flag being set anymore. This originates from the need to have parent_cell computed before the partitioner may be constructued. Should we disallow this combination?

Also, I have tested the implementation with the above code snippet - not sure how to extend that to a reasonable unit test though.

[schnellerhase]

I have a problem with the std::optional here in the Python layer. The export of the partitioner callback now has signature std::optional<CellPartitionFunction> (to allow for handling the different cases of nullptr and std::nullopt). Passing None results in a std::nullopt and no longer the nullptr as before . However I am not aware of another way to produce a nullptr in the C++ layer other than by passing None.

It comes down to: how can a std::optional<CellPartitionFunction> = nullptr case be produced form the python caller side?

[garth-wells]

My first reaction is that supporting nullptr and std::nullopt is confusing. What's the difference?

If there are three options, perhaps an enum is better.

[schnellerhase]

The three cases are

1. custom callback - caller provides a partitioner,
2. nullptr - a default partitioner will be created and used for the refined mesh,
3. nullopt - maintain the partition of the coarse mesh and redistribute ghosts according to new fine mesh structure (creating a identity_partitioner)

Problem is that the identity_partitioner relies on the parent_cell computation internally, so it can not be created before the refine call and passed as a custom partitioner.

[schnellerhase]

How about we introduce aliases for nullopt and nullptr in this use case? Something along the lines of placeholder_identity_partitioner and a no_repartitioning. That would allow to stick to the current type management with an optional, which fits nicely the problem necessities in typing, but allows for more verbose option selection on the caller side.

[garth-wells]

I don't see how aliases would work. Python None maps to nullptr and nullopt.

A std::variant<Enum, std::function<...>> could work.

[schnellerhase]

I don't see how aliases would work. Python None maps to nullptr and nullopt.

Would only have worked on the C++ layer.

A std::variant<Enum, std::function<...>> could work.

That's a fix - also realigns the python interface. No longer requires the additional redistribute flag. Only opted for a struct over an enum, since we only have one state.

[jhale]

This looks good to me - expensive repartitioning is now the default option, and it was always a bit odd that repartitioning would throw your mesh across all processes in the default case.

It would be interesting to look at providing examples of other useful custom partitioners or calls into ParMETIS with non-default arguments.