#478 Parallelization of Lindemann Index Calculation

[N720720]

Description:
The current implementation of the Lindemann index calculation algorithm in the lindemann package is primarily executed in a serial manner, which limits its performance and scalability, especially for large datasets typically processed in high-performance computing (HPC) environments. The goal is to optimize the existing algorithm by introducing parallelization using Numba's just-in-time (JIT) compilation and parallel processing capabilities (numba prange).

Proposed Solution:
Enhance the performance of the Lindemann index calculation by implementing a parallel version of the algorithm using Numba. The key steps involved in this enhancement include:

1. Implementation of Parallel Variance Calculation:

   • Using Welford's algorithm to compute the mean and variance in a numerically stable way.
   • Using Numba to calculate the variance.

2. Chunk-wise Processing:

   • Splitting the frames into multiple chunks to enable parallel processing (prange)
   • Calculating the mean and variance for each chunk independently.

3. Combining Results:

   • Aggregating the results from all chunks to compute the final Lindemann index.
   • Using reduction to efficiently combine the results.

Code Implementation:
The following functions were developed in parallel_trj.py to achieve the parallelization:

• parallel_variance: Computes the variance in parallel using Welford's algorithm.
• calculate_chunk: Processes a chunk of frames to compute mean and variance distances.
• calculate: Divides the data into chunks and uses parallel processing to compute the Lindemann index.

Benefits:

• Significant reduction in computation time for large datasets.
• Efficient utilization of multi-core processors.
• Enhanced scalability and performance, making it suitable for HPC environments.
  

Next Steps:

• Parallelization of the -ot flag.