Mesh Pre-processing Library

This library implements functions and classes for mesh registration, data augmentation, and data normalisation. All methods can operate on a single mesh or on all the meshes contained in a given directory. The library can also be used via Command Line Interface (CLI). See "Command Line Interface Usage" section below.

Registration

• Landmark-based Procrustes registration: see registration.ProcrustesLandmarkRegisterer class or use procrustes_landmark_registration in CLI. The registration is performed by scaling, translating, and rotating a mesh according to the transformation estimated performing a Procrustes alignment between a set of landmarks on the mesh and the corresponding landmarks on a reference mesh.

• Axis of Inertia alignment followed by ICP registration: see registration.InertiaAxesAndIcpRegisterer class or use inertia_axes_and_icp_registration in CLI. Align a mesh with another mesh or a point cloud using the principal axes of inertia as a starting point which is refined by iterative closest point.

• Landmark-based Procrustes alignment followed by ICP registration: see registration.ProcrustesLandmarkAndIcpRegisterer class or use procrustes_landmark_and_icp_registration in CLI. Perform ICP after an initial landmark-based Procrustes alignment.

• Landmark-based Procrustes alignment followed by NonRigid-ICP registration: see registration.ProcrustesLandmarkAndNicpRegisterer class or use procrustes_landmark_and_nicp_registration in CLI. After an initial landmark-based Procrustes alignment, iteratively deform a reference mesh to the new mesh. This can be used to put meshes in dense point correspondence with respect to a template. When operating on heads/faces, consider using data_weights computed with utils.create_data_weights().

Augmentation

• Random linear interpolations: see augmentation.RandomLinearInterpolation class or use random_linear_interpolation in CLI. New meshes are created by linearly interpolating the vertex coordinates of two meshes. The amount of the interpolation is the same for all vertices.

• Random spectral interpolations: see augmentation.RandomSpectralInterpolation class or use random_spectral_interpolation in CLI. New meshes are created by interpolating the spectral components of two meshes. This technique was introduced in Latent Disentanglement in Mesh Variational Autoencoders Improves the Diagnosis of Craniofacial Syndromes and Aids Surgical Planning by Foti et al. (2023).

• Random spectral perturbation: see augmentation.RandomSpectralPerturbation class or use random_spectral_perturbation in CLI. New meshes are created by randomly perturbing the spectral components of a single mesh. his technique was introduced in Intraoperative Liver Surface Completion with Graph Convolutional VAE by Foti et al. (MICCAI-GRAIL 2020).

Normalisation or Encryption

• Normalisation/encryption: see normalisation.Normaliser class or use one of the following equivalent commands in the CLI: normalisation, normalization, or encryption. Normalisation class that can be very useful to prepare data for training in case they need to be normalised. Alternatively, it can be used to encrypt sensitive meshes (e.g. faces of real people). Obviously, if this class is used for encryption, the normalisation_dictionary behaves as the encryption key and needs to be transferred separately. All the normalised meshes can be considered to be encrypted. Anyone who visualise them will see random shapes unless they are un-normalised first.

---

Installation

This library has only a limited number of requirements. However, additional libraries can be installed to significantly speed-up NonRigid-ICP. We recommend starting with the installation of the optional requirements as pykeops can use torch when available.

Optional requirements

• sksparse
• pytorch

The optional requirements can be installed as follows:

• Activate you environment (if it wasn't already active).

• Install the sksparse requirements (for Ubuntu): sudo apt-get install python-scipy libsuitesparse-dev

• Install sksparse: pip install scikit-sparse or conda install -c conda-forge scikit-sparse

• Install pytorch following the installation instructions available here.

Basic requirements

• trimesh
• numpy
• scipy
• pykeops
• fire

These requirements can be installed with:

pip install trimesh scipy pykeops fire

Note that pykeops can also be replaced with pytorch3d. Installation instructions can be found here

---

Command Line Interface Usage

The command line interface can be used as follows:

python mproc_cli.py <name_of_method_as_listed_above> --arguments=values

Additional functions within the class can be accessed adding the name of the function before the arguments. For instance, the register_all_and_save() method can be used as follows:

python mproc_cli.py <name_of_method_as_listed_above> register_all_and_save --arguments=values

An example of a command is:

python mproc_cli.py procrustes_landmark_registration --reference_path=/.../ref.ply --reference_landmarks_path=/.../ref_lms.txt --mesh_path=/.../mesh.ply  --mesh_landmarks_path=/.../mesh_lms.txt --show_results=True

All CLI commands are used to execute the __call__() function of the classes listed at the beginning of this document. The arguments specified in the CLI can belong either to their __call__() or __init__() functions.