v0.14.2

Release v0.14.2 - June 2022

• |feature| |efficiency| (Windows-systems only) Removed the unorthodox default installation procedure of DeerLab based on the installation of Numpy and related packages linked against MKL via the Gohlke repository (:issue:322, :pr:330).

  • As a result the default performance of DeerLab can be affected in some Windows systems. To link the Numpy and related packages against MKL as in previous versions, an automated script upgrade_mkl.py is provided with the package.
  • Fixes the error appearing during installation if the git command was not installed or available in the system (:issue:326).
  • Allows the distribution of DeerLab as wheels.

• |feature| Implemented better options for automated and user-supplied noise estimates to improve bootstrapping approaches (:pr:334, :pr:343).

• |fix| Avoid the installation of (potentially unstable) pre-release versions of Numpy in systems with fresh Python installations (:pr:336).

• |fix| Improved the robustness of several function against non-numerical values due to division-by-zero errors (:pr:335).

• |fix| Corrected the behavior of regularization parameter selection with L-curve methods (:pr:340). Fixes the lc method in selgregparam which was seeking the optimal regularization parameter by minimizing curvature rather than by maximizing it. Prevents failure of the L-curve methods due to the appearance of non-numeric values when evaluating too large regularization parameter values.

• |fix| Fixes the error when specifying a limited excitation bandwidth in dipolarmodel via the excbandwidth argument (:pr:342).

• |fix| Fixes the navigation menu on the documentation that appeared empty on mobile phones or for partially minimized windows on computers, impeding navigation through the documentation (:pr:346).

• |fix| Minor corrections to the documentation and examples.

v0.14.1

Release v0.14.1 - June 2022

• |fix| Use Scipy's eigh instead of Numpy's to avoid convergence error numpy.linalg.LinAlgError: Eigenvalues did not converge during model uncertainty propagation (#310, #311).
• |fix| Refactored the code to avoid the use of lambda and nested functions. This enables the pickling DeerLab's objects with Python's pickle module without errors (#312).
• |feature| Added two new utility functions store_pickle and read_pickle that implement pickling with the dill package to be more robust against potential lambda functions defined by the users in scripts (#312).
• |fix| Fixed minor bug when printing fit results with many model parameters being frozen. The print command would return an error message (#329).
• |fix| Fixed bug when propagating bootstrapped uncertainty in presence of round-off errors (#325).
• |fix| |enhancement| Multiple minor improvements and corrections in the documentation.

v0.14.0

Release v0.14.0 - April 2022

Overall changes

• |feature| |api| Complete overhaul of the DeerLab modelling and fitting interface. Check the new documentation for help and details. (#218, #223, #228, #237, #225, #243).

  • A new modelling system has been introduced. DeerLab main interface runs on a new Model object class. Models implement and provide the distinction between linear and non-linear parameters.
  • Model parameters are no longer (solely) identified by their indexing inside a parameter vector, but are referenced by name. This avoids the need for a user to recall the ordering of the parameters. This is now all handled internally. For example, before paramA = parameters[idxA] is now model.paramA.
  • Any model parameter is accessible from the model object and its boundaries, start values and other properties can be easily modified. For example, to change the lower boundary of a parameter: model.paramA.lb = 0.
  • A new general fit function that fits arbitrary Model objects to single or multiple datasets has been implemented. The function automatically handles the selection of solvers to optimally fit the data to the model.
  • Implemented a new function link to link model parameters (setting equality constraints in models).
  • Implemented a new function merge to create a model returning more than one response (allowing the creation of global models).
  • Implemented a new function relate to define functional relationships between model parameters.
  • Implemented a new function lincombine to create a model whose response is a linear combination of the inputs' model responses.
  • Model parameters can now be frozen (set to a constant value and ignored during optimization) in the Model object and on the back-end snlls solver. For examples, to fix a parameter to a certain value: model.paramA.freeze(0.5).
  • Arbitrary normalization conditions can be imposed to the linear parameters.
  • Bootstrapping can now be requested directly from the fit function via the bootstrap keyword argument. The function will then return the bootstrap uncertainty quantification of all model parameters and of the model's response instead of the covariance-based uncertainty.
  • Implemented a new function dipolarmodel, which generates models based on the dipolar EPR multi-pathway theoretical model.
  • Added new examples, adapted existing ones, and removed unneeded examples.
  • Add many new tests and removed tests related to deprecated functionality.
  • All the built-in parametric models are now pre-compiled Model objects instead of just functions.
  • The function fitmodel has been deprecated and removed. The original has been substituted (and greatly expanded) by the new dipolarmodel and fit functions.
  • The function fitmultimodel has been deprecated and removed. The original functionality can be easily scripted with the new modelling system. An example of has been added, describing how to script the same functionality.

• |feature| Introduced the profile-likelihood methodology both for uncertainty quantification based on likelihood-confidence intervals, and for identifiability analysis (#222).

  • Added a new function profile_analysis to calculate the objective function profiles from model object parameters.
  • Implemented a new uncertainty quantification UQResult object type 'profile' for results obtained from profile_analysis.

• |feature| Implemented a system to specify arbitrary penalty functions to be included in the non-linear part of the objective function during optimization. The penalties can be custom-defined and constructed into a Penalty object that can be passed to the fit function. Outer optimization of the penalty weights can also be included based on certain information-based criteria (#197, #218, #225).

  • Implemented a new object Penalty that includes the penalty function, weight parameter (and its boundaries), and the selection functional for optimization.
  • Adds new outer optimization options for the penalty weights, based on hard-coded model selection functionals. For now, the ICC, AIC, AICc, and BIC functionals are available.
  • Implemented a new function dipolarpenalty that generates dipolar-EPR-specific penalties, e.g. to induce compactness and/or smoothness.

• |feature| Implemented masking of datasets during optimization (#250).

• |feature| Added a verbose option to display progress of the fit routines (#250).

• |feature| Added support for analyzing and fitting complex-valued models and data (#127, #218).

• |feature| Orientation selection in dipolar signals can now be simulated for arbitrary orientation weights distributions via the orisel keyword argument in the new dipolarmodel or the dipolarkernel functions (#183, #203).

• |feature| Re-purposed the ex_ models. Each of these function represents a specific dipolar EPR experiment. These now take the experimental time delays as input, and return a new ExperimentInfo object. This can be passed to dipolarmodel via the optional keyword argument experiment to refine the boundaries and start values of the dipolar pathway refocusing times and amplitudes based on the experimental setup (#225).

• |feature| Implemented masking of datasets during optimization (#250).

• |enhancement| Overhaul of the DeerLab documentation website (#235).

  • Full HTML/CSS overhaul. The new web design based on the PyData theme has a clearer design, with more readable pages and code blocks.
  • Deprecates the use of the RTD theme. This removes the hard constraint of using Sphinx 1.8.0. Now the documentation builds with the latest Sphinx release.
  • Add a user-guide for the new modelling and fitting system.
  • Re-organize all of the website content.
  • Improved the dipolar EPR starting guide, and adapted it to the new system.
  • Fixed some minor errors in the examples.
  • Redesigned all examples towards the use of actual experimental data files in BES3T format. Examples can now be taken and easily adapted by users to their experiment data files (#304).

• |enhancement| Added the functionality to print a FitResult object to display a summary of the fit goodness-of-fit statistics and a table of all estimated parameters and their uncertainties (#234).

• |enhancement| Added a new keyword argument regparamrange to snlls and fit to specify the search range of the optimal regularization parameter (#225).

• |enhancement| Noise levels of the datasets can be optionally specified in all functions taking datasets (#213).

• |enhancement| Added the option to include or exclude the edges of vector in regoperator via a new keyword argument includeedges (#204). The default for all functions in DeerLab has been set to includeedges=False (#205, #230).

• |enhancement| Generalized the regularization operator. Related functions no longer take regorder (regularization operator order) as an argument. Instead they now take regop (the full regularization operator) as an argument (#216).

• |enhancement| Generalized the regularized linear least-squares functionality. Now it can handle arbitrary bounds on linear parameters and adapts the linear LSQ solver back end accordingly (#216).

• |efficiency| Improved performance of post-optimization model evaluation/propagation for large datasets (#200, #238).

• |efficiency| Implemented (adaptable) memory limits for potentially memory-intense functions (#201, #239).

• |api| The function correctscale has been deprecated (#293). Its limited functionality is included in the now broader functionality provided by the new modelling and fitting system.

• |api| The functions fitregmodel and fitparamodel have been deprecated and their core functionality merged into snlls. The snlls function now handles any kind of least-squares problem and automatically employs optimal combinations of solvers to find the solution to the problems (#218).

• |api| Renamed the function bootan to bootstrap_analysis (#227).

• |api| Deprecated TV and Huber regularization. Accordingly the keyword arguments regtype, huberparameter have been removed throughout (#216).

• |api| Deprecated the nnlsbpp NNLS solver (#231).

• |api| Deprecated the regparamrange function (#232). It depended on home-written code for the GSVD, which (as shown in previous issues) was prone to LAPACK backend problems and other bugs. This function was still a derelict from DeerAnalysis methodology.

• |api| The function time2dist has been renamed to distancerange (#273, #274).- |api| The function time2dist has been renamed to distancerange (#273, #274).

• |api| The convergence control arguments of the fit functions have now been renamed for consistency with the least_squares function of the SciPy package (#296).

• |api| Changed the name of the parameter width to std in the dd_gauss, dd_gauss2, dd_gauss3, and dd_skewgaussmodels (#278, #280).

• |fix| When using the multistart keyword argument, no longer includes the parameter boundaries in the set of multiple start values (#218).

• |fix| Fixed error (manifesting as nan values in the confidence intervals) caused by a division-by-zero in the covariance matrix estimation (#218).

• |fix| Fix encoding error during installation (#252). This error could disrupt the installation in OS with default encoding different from CP-1252.

• |fix| Implement a new function to ensure that estimated Jacobians are positive semi-definite matrices. This fixes the appearance of warnings and bugs when calculating confidence intervals (#216).

• |fix| Corrected the scale invariance behavi...

v0.13.2

Release v0.13.2 - July 2021

Overall changes

• Fixed an error appearing during installation in Windows systems. If during installation a python executable alias was not created, the call to the pipwin manager returned an error and the installation failed to download and install Numpy, SciPy and cvxopt. (#187).
• Fixed the rendering of certain code-blocks in the documentation examples which were appearing as plain text (#179, #184).
• Fixed the execution and rendering of the model examples in the documentation (#189, #190).
• Fixed a bug in snlls where one of the linear least-squares solvers can return results that violate the boundary conditions due to float-point round-off errors (#177, #188).

v0.13.1

Release v0.13.1 - May 2021

Overall changes

• Fixed the behaviour of global weights throughout DeerLab fit functions. The keyword argument weights was not having any or the expected effect in the results in some fit functions. Also fixes the behaviour of built-in plots for global fits (#168, #171).
• Optimize default weights in global fitting according to the datasets noise levels (#169, #174).
• Fixed a bug in snlls that was causing the confidence intervals in snlls, fitmodel and fitmultimodel to vanish for large signal scales (#165, #166).

Specific changes

• deerload: Corrected a bug that happened in certain BES3T Bruker spectrometer files, when there are entries under the MANIPULATION HISTORY LAYER section at the end of the descriptor file. Also fixed the reading of .XGF partner files (#164).
• snlls: The keyword argument extrapenalty now requires a function that takes both non-linear and linear parameters. Corrected the name of the keyword in the documentation (#175).
• fitparamodel: Fixed the scaling of the output FitResult.model and FitResult.modelUncert (#173).
• ex_pseudotitration_parameter_free: Removed Ctot from second order term in the chemicalequalibrium polynomial (#163).

v0.13.0

Release v0.13.0 - April 2021

New features

• DeerLab is now distributed via the Anaconda repository and can be installed with the conda package manager (#12, #157). The installation instructions have been expanded to describe the Anaconda installation (#155).
• DeerLab now supports Python 3.9.
• The function fitsignal has been re-named to fitmodel for correctness and consistency with other functions (#102).
• Added new experiment models for RIDME on systems with one to seven harmonic pathways (S=1/2 to S=7/2) to include all higher harmonics (overtones) (#79).
• Bootstrapping is now embedded into fitmodel to automatically bootstrap all output quantities without the need to write additional script lines (#55). In fitmodel a new option uq allows to switch between covariance or bootstrapping uncertainty quantification (#88).
• The function fitmodel now returns Vmod and Vunmod, the modulated and unmodulated contributionsto the fitted dipolar signal, respectively, along their uncertainties as additional outputs (#78).
• Implemented several initialization strategies in fitmultimodel for multi-model components (#67). Three different new strategies 'spread', 'split' and 'merge' will initialize the parameter values of the N-component fit based on the results of the N-1/N+1 component fit to improve quality of results and speed.
• Added contribution guidelines to the documentation and automated list of DeerLab contributors.
• The function snlls now accepts additional custom penalties to include in the optimization (#76, #108).
• All fit functions now return the fit of the data along its uncertainty automatically as part of the FitResult object (#130, #134).
• Implemented a new method UQResult.join() to merge multiple uncertainty quantification objects (#154). This permits error propagation from multiple uncertainty sources to a common function.

Overall changes

• The performance of all fit functions has been considerably accelerated by removing call overheads in built-in DeerLab models (#100, #101, #143).
• Improved robustness of the installation from PyPI (#65):
  • The installer no longer assumes the alias pip to be setup on the system.
  • The installation will now handle cases when system-wide privileges are not available (#52).
  • Improved robustness of the installation in Windows systems to avoid missing DLL errors (#64).
  • The installer will now get the latest Numpy/Scipy releases in Windows systems available at the Gohlke repository.
• Adapted piece of code leading to a VisibleDeprecationWarning visible during execution of certain DeerLab functions.
• Improved interface of built-in plots in FitResult.plot(). The method now returns a Matplotlib figure object (matplotlib.figure.Figure) instead of an axes object (matplotlib.axes._subplots.AxesSubplot) which can be modified more freely to adjust graphical elements (#85). The method now takes an optional keyword FitResult.plot(show=True\False) to enable/disable rendering of the graphics upon calling the method (#87).
• The fit objective values returned in FitResult.cost are now correct (previous versions had an erroneous 1/2 factor) (#80). The value is now returned as a scalar value instead of a single-element list (#81).
• Removed the re-normalization conventions K(t=0,r)=1 and B(t=0)=1 and associated options renormalize and renormpaths in the dipolarkernel and dipolarbackground functions (#99) to avoid identifiability issues between dipolar pathway amplitudes and signal scales during fitting (#76).
• The fit convergence criteria tol (objective function tolerance) and maxiter (iteration limit) are now exposed as keyword argument in all fit functions (#111, #112).
• Multiple improvements and corrections to the documentation (#95, #96, #104, #106, #107, #115, #122)
• Corrections in the metadata of multiple dd_models. The key Parameters of some models contained the wrong names.
• The metadata of the built-in models is now accessible and manipulable via function attributes (e.g. dd_gauss.parameters) rather than trought a returned dictionary (e.g. dd_gauss()['Parameters']) (#143).
• The keyword argument to request uncertainty quantification has been unified across all fitting functions. It is now uq (#120).
• The UncertQuant class has been renamed into UQResult (#123).
• Uncertainty quantification is now tested numerically against an external package (lmfit) to ensue quality and accuracy (#121).
• Expanded the collection of examples in the documentation, and improved existing ones (#144, #148, #153).

Specific changes

• deerload:
  • Fixed behaviour of the function when loading certain 2D-datasets in the BES3T format (#82, #83).
  • In 2D-datasets, the abscissas are now returned as a list of abscissas instead of a single 2D-matrix (#83)).
• fitmodel:
  • Corrected the scaling behaviour of all outputs related to components of the dipolar signal to match the scaling of the original experimental data (#78).
  • The built-in plot method FitResult.plot() now plots the unmodulated component fit as well with its uncertainty (#78).
  • When plotting bootstrapped results with FitResult.plot(), the fit is substituted with the median of the bootstrapped distribution (#148).
  • Extended information included in the verbose summary (#78).
  • Simplified the interface for defining initial values and boundaries of parameters in fitsignal (#71). Now instead of defining, e.g., fitsignal(..., lb = [[],[50],[0.2, 0.5]]) one can define the individual vales/boundaries fitsignal(..., bg_lb = 50, ex_lb = [0.2,0.5]) by using the new keywords.
  • Removed the keyword argument uqanalysis=True/False. The uncertainty quantification can now be disabled via the new keyword uq=None (#98).
  • Corrected the behaviour of built-in start values when manually specifying boundaries (#73). If the built-in start values are outside of the user-specified boundaries the program will now automatically set the start values in the middle of the boundaries to avoid errors (#72).
  • Implemented the constraint Lam0+sum(lam)<=1 to ensure the structural-identifiability of Lam0 and V0 during SNLLS optimization of experiment models with more than one modulated dipolar pathway (i.e. does not affect ex_4pdeer) (#76, #108).
  • The function now accepts any sequence input (lists, arrays, tuples, etc.) instead of just lists (#152).
  • Removed the optional keyword argument regtype (#137).
  • Fixed a bug in the scaling of the distance distribution uncertainty quantification (#148).
• fitregmodel:
  • Corrected the behaviour of the uncertainty quantification when disabling the non-negativity constraint (#121).
• ``fi...

v0.12.2

Release v0.12.2 - October 2020

Hotfix

• regparamrange: The exception handling introduced in the previous release was still too specific. The function kept crashing due to SVD non-convergence errors during the GSVD. This has been fixed and the error will not lead to a crash. (#42).

Overall changes

• Fit functions using the multistart option are now fully deterministic. The functions was using now a random generator to define the different start points, this is now deterministic.

• Documentation UI has been re-designed for a more confortable reading. Minor errors and outdated information have been corrected throughout. Expanded reference documentation of several functions for better understanding.

Specific changes

• dd_skewgauss: corrected an error in the implementation that was leading to wrong distributions (#61).

• dd_models, ex_models: Adapted numerical boundaries and start values of some built-in models to reflect better the physical reality. Afected models: dd_skewgauss, dd_triangle, dd_gengauss, ex_5pdeer, ex_ovl4pdeer.

v0.12.1

Release v0.12.1 - October 2020

Overall changes

• The calculation of the Jacobian for covariance-based uncertainty analysis has been simplified providing a significant boost in performance for all fit functions (#55).

• The Jacobian computation is more robust, now taking into consideration parameter boundaries (#58). This solves errors such as the ones reported in #50.

• Broken examples in the documentation have been corrected (#57).

• When requesting attributes or method of a UncertQuant object under disabled uncertainty analysis (uqanalysis=False) now it will prompt an explanatory error instead of just crashing (#56).

Specific changes

• fitsignal: corected the behaviour of the scaling output (fit.scale). Now all fitted dipolar signals (fit.V) have the same scaling as the input signal (#53).

• regparamrange: relaxed the exception handling to catch errors occuring under certain conditions. The function seems to crash due to LAPACK or SVD non-convergence errors during the GSVD, now these are catched and the alpha-range is estimated using simple SVD as an approximation. This function might be deprecated in a future release (#42).

v0.12.0

Release Notes v0.12.0 - October 2020

Overall changes

• The regularization operator matrices regoperator now include the edges of the distribution (#38). Now the smoothness penalty is imposed on the distribution edges avoiding the accumulation of distribution mass at the edges of r.

• The interface for defining dipolar pathways has been simplified (#41). For example, a signal with two dipolar pathways had to be defined as pathways = [[Lam0,np.nan], [lam1,T0]]. Now the unmodulated pathway can just be defined by its amplitude and does not require the use of np.nan, e.g. pathways = [Lam0, [lam1,T0]].

• The project version control has been switched from the Git-flow to the GitHub-flow design. The default branch has been switched from master to main, which is now always production-ready. All new contributions are merged into main exclusively by pull requests.

• The dependency on the lambda parameter has been removed from all phenomenological background models, and kept only for physical models (#43). Their interface with dipolarbackground and dipolarkernel have been updated accordingly.

New features:

• diststats: New function to calculate different statistical quantities of the distance distribution and their corresponding uncertainties (#38).

• bootan: Introduced the option cores to parallelize the bootstrapping using multiple CPU.

Specific changes

• bg_homfractal:

  • Corrected behavior of the model. For d=3 the model returned wrong values, and for d~=3 the model resulted in an error.

• UncertQuant:

  • Fixed bug when propagating uncertainty to scalar functions.

• deerload:

  • Fixed UTF-8 error when loading certain spectrometer files in MacOS (#30)

• fitsignal:

  • The fitted scale of the signal is now properly calculated when fitting fully parametric signals.
  • Fixed error occuring when fitting a dipolar evolution function with a non-parametric distribution.

• selregparam:

  • Fixed bug occuring when requesting the lc or lr selection methods.

• regparamange:

  • An error occuring at the BLAS/LAPLACK error ocurring under certain conditions in MacOS and Ubuntu is now handled to avoid a crash.

v0.11.0

Version 0.11.0 - September 2020

Overall changes

• All Gauss models (dd_gauss,etc.) now use the standard deviation sigma instead of the FWHM as the width parameter for consistency with other method such as Rice distributions. (#19)

• All hard-wired random seeds have been removed.

• A new method plot() has been added to the FitResult class returned by all fit functions. This will create a basic plot of the fit results. (#7)

Specific changes

• snlls:

  • Renamed option penalty as reg and improved its interface (#13).
  • The regularization parameter of the optimal solution is returned now (#20).

• whitegaussnoise: Added a seed option to select static noise realizations.

• correctzerotime:

  • Fixed bug when zero-time is at start/end of array (#24).
  • Function no longer rescales the experimental data passed on to the function.

• fitsignal:

  • The regularization parameter of the optimal solution is returned now (#20).
  • Bug fixed when fitting dipolar evolution functions (no background and no experiment models) with a parametric distance distribution.

• fitmultimodel: Start points are now spread over constrained parameter space grid instead of being randomble initiated (#22).

• deerload:

  • Now returns the time axis in microseconds instead of nanoseconds (#21).
  • The bug appearing when loading certain BES3T files has been fixed (#14).

• fitregmodel: Now returns the fitted dipolar signal in the FitResult output

• correctscale: The parameter fit ranges have been adjusted.