Remove ghost papercuts

[Tormod Landet]

Below are some issues that should be fixed to make working with ghosted meshes nicer.

Papercuts:

Vectors:

• get_local() should take a flag to include ghost dofs
• set_local() should be checked to ensure that it can gracefully handle getting ghost dof values at the end of the array (by ignoring them). This may be more controversial?
• local_size() should take a flag to include ghost dofs

Function space:

• tabulate_dof_coordinates() should take a flag to include coordinates of ghost dofs

Mesh:

• num_cells() and num_entities should take a flag to include ghost entities or not

General:

• Just by reading a Python or C++ code that uses the DOLFIN API it should be immediately obvious whether the API calls are addressing global, local "regular", local "ghost" or local "all" data. This is currently not the case. Something like num_cells("regular") is (to me) considered immediately obvious.

Explanation:

This list is a work in progress. Feel free to comment, especially if you disagree about the proposed API changes. The goal is that new flags are added in a backwards compatible way.

The above "papercuts" have been identified while implementing various DG slope limiters and other Function post-processors. A typical implementation of a slope limiter is "loop over owned cells and change owned cell DOF values based on the geometry and DOF values of vertex neighbors (including ghosted cells)".