Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

API: Add to_matfile which saves all the FISSA parameters and outputs #249

Merged
merged 9 commits into from
Jul 13, 2021
Merged

API: Add to_matfile which saves all the FISSA parameters and outputs #249

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8a3d763
API: Add Experiment.to_matfile
scottclowe Jun 21, 2021
10d3d35
DEP: Deprecate save_to_matlab
scottclowe Jul 13, 2021
38b9273
TST:DOC: Label legacy matlab comparisons as such
scottclowe Jul 13, 2021
78fb6a5
TST: Add tests for new matlab output
scottclowe Jul 13, 2021
d1ce28e
API: Change run_fissa to use new to_matfile method
scottclowe Jul 13, 2021
b7d72c0
TST: save_to_matlab() -> to_matfile(legacy=True)
scottclowe Jul 13, 2021
d504435
JNB: Fix typo
scottclowe Jul 13, 2021
46cd5ac
JNB: exp.save_to_matlab -> exp.to_matfile and update MATLAB example
scottclowe Jul 13, 2021
4ac0614
JNB: Add mean image and roi_polys example for MATLAB
scottclowe Jul 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 26 additions & 11 deletions examples/Basic usage - Functional.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -435,7 +435,7 @@
"\n",
"The results can easily be exported to a MATLAB-compatible matfile as follows.\n",
"\n",
"We can either use `export_to_matlab=True`, which will include export to matlab with the default filename, `'matlab.mat'`, within the cache directory (the cache directory being set by our `output_folder` variable)."
"We can either use `export_to_matlab=True`, which will include export to matlab with the default filename, `'separated.mat'`, within the cache directory (the cache directory being set by our `output_folder` variable)."
]
},
{
Expand Down Expand Up @@ -473,19 +473,34 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Loading the generated file, `<output_folder>/matlab.mat` or your custom filename, in MATLAB will give you three structs, `ROIs`, `raw`, and `result`.\n",
"Loading the generated file, `output_folder/separated.mat` or your custom filename, in MATLAB will provide you with all of FISSA's outputs.\n",
"\n",
"These interface similarly as `experiment.ROIs`, `experiment.raw`, and `experiment.result` described above. However, Matlab counts from 1 (as opposed to Python counting from 0), such that the ROI, raw trace, and decontaminated trace are all found for cell 0 trial 0 as:\n",
"These interface similarly as `experiment.raw`, and `experiment.result` described above, with a few small differences.\n",
"\n",
"With the python interface, the outputs are 2d numpy.ndarrays each element of which is itself a 2d numpy.ndarrays.\n",
"In comparison, when the output is loaded into MATLAB this becomes a 2d cell-array each element of which is a 2d matrix.\n",
"\n",
"Additionally, whilst Python indexes from 0, MATLAB indexes from 1 instead.\n",
"As a consequence of this, the results seen on Python for a given roi and trial `experiment.result[roi, trial]` correspond to the index `S.result{roi + 1, trial + 1}` on MATLAB.\n",
"\n",
"Our first plot in this notebook can be replicated in MATLAB as follows:\n",
"```octave\n",
"% data = load('fissa-example/matlab.mat') % load matfile saved to cache output dir\n",
"data = load('experiment_results.mat') % load matfile save to custom output\n",
"data.ROIs.cell0.trial0{1} % polygon for the ROI\n",
"data.ROIs.cell0.trial0{2} % polygon for first neuropil subregion\n",
"data.result.cell0.trial0(1, :) % final extracted cell signal\n",
"data.result.cell0.trial0(2, :) % contaminating signal\n",
"data.raw.cell0.trial0(1, :) % raw measured celll signal\n",
"data.raw.cell0.trial0(2, :) % raw signal from first neuropil subregion\n",
"% Load the FISSA output data\n",
"% S = load('fissa-example/separated.mat')\n",
"S = load('experiment_results.mat')\n",
"% Select the second trial\n",
"% On Python, this is equivalent to trial = 1\n",
"trial = 2;\n",
"% Plot the result traces for each ROI\n",
"figure; hold on;\n",
"for i_roi = 1:size(S.result, 1);\n",
" plot(S.result{i_roi, trial}(1, :));\n",
"end\n",
"xlabel('Time (frame number)');\n",
"ylabel('Signal intensity (candela per unit area)');\n",
"grid on;\n",
"box on;\n",
"set(gca,'TickDir','out');\n",
"```"
]
},
Expand Down
52 changes: 43 additions & 9 deletions examples/Basic usage.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -633,24 +633,58 @@
"metadata": {},
"outputs": [],
"source": [
"experiment.save_to_matlab()"
"experiment.to_matfile()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Loading `output_folder/matlab.mat` in MATLAB will give you three structs, `ROIs`, `raw`, and `result`.\n",
"Loading `output_folder/separated.mat` in MATLAB will give you structs for all of FISSA's outputs.\n",
"\n",
"These interface similarly as `experiment.ROIs`, `experiment.raw`, and `experiment.result` described above. However, Matlab counts from 1 (as opposed to Python counting from 0), such that the ROI, raw trace, and decontaminated trace are all found for cell 0 trial 0 as:\n",
"These interface similarly as `experiment.raw`, and `experiment.result` described above, with a few small differences.\n",
"\n",
"With the python interface, the outputs are 2d numpy.ndarrays each element of which is itself a 2d numpy.ndarrays.\n",
"In comparison, when the output is loaded into MATLAB this becomes a 2d cell-array each element of which is a 2d matrix.\n",
"\n",
"Additionally, whilst Python indexes from 0, MATLAB indexes from 1 instead.\n",
"As a consequence of this, the results seen on Python for a given roi and trial `experiment.result[roi, trial]` correspond to the index `S.result{roi + 1, trial + 1}` on MATLAB.\n",
"\n",
"Our first plot in this notebook can be replicated in MATLAB as follows:\n",
"```octave\n",
"% Load the FISSA output data\n",
"S = load('fissa-example/separated.mat')\n",
"% Select the third ROI, second trial\n",
"% On Python, this is equivalent to roi = 2; trial = 1;\n",
"roi = 3; trial = 2;\n",
"% Plot the raw and result traces for the ROI signal\n",
"figure; hold on;\n",
"plot(S.raw{roi, trial}(1, :));\n",
"plot(S.result{roi, trial}(1, :));\n",
"title(sprintf('ROI %d, Trial %d', roi, trial));\n",
"xlabel('Time (frame number)');\n",
"ylabel('Signal intensity (candela per unit area)');\n",
"legend({'Raw', 'Result'});\n",
"grid on;\n",
"box on;\n",
"set(gca,'TickDir','out');\n",
"```\n",
"\n",
"Assuming all ROIs are contiguous and described by a single contour, the mean image and ROI locations can be plotted in MATLAB as follows:\n",
"```octave\n",
"ROIs.cell0.trial0{1} % polygon for the ROI\n",
"ROIs.cell0.trial0{2} % polygon for first neuropil subregion\n",
"result.cell0.trial0(1, :) % final extracted cell signal\n",
"result.cell0.trial0(2, :) % contaminating signal\n",
"raw.cell0.trial0(1, :) % raw measured celll signal\n",
"raw.cell0.trial0(2, :) % raw signal from first neuropil subregion\n",
"% Load the FISSA output data\n",
"S = load('separated.mat')\n",
"trial = 1;\n",
"figure; hold on;\n",
"% Plot the mean image\n",
"imagesc(squeeze(S.means(trial, :, :)));\n",
"colormap('gray');\n",
"% Plot ROI locations\n",
"for i_roi = 1:size(S.result, 1);\n",
" contour = S.roi_polys{i_roi, trial}{1};\n",
" plot(contour(:, 2), contour(:, 1));\n",
"end\n",
"set(gca, 'YDir', 'reverse');\n",
"```"
]
},
Expand Down
210 changes: 170 additions & 40 deletions fissa/core.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1459,55 +1459,112 @@ def calc_deltaf(self, freq, use_raw_f0=True, across_trials=True):
if self.folder is not None:
self.save_separated()

def save_to_matlab(self, fname=None):
r"""
Save the results to a MATLAB file.

This will generate a .mat file which can be loaded into MATLAB to
provide structs: ROIs, result, raw.

If Δf/f\ :sub:`0` was calculated, these will also be stored as ``df_result``
and ``df_raw``, which will have the same format as ``result`` and
``raw``.
def to_matfile(self, fname=None, legacy=False):
r"""Save the results to a MATLAB file.

These can be interfaced with as follows, for cell 0, trial 0:
.. versionadded:: 1.0.0

``ROIs.cell0.trial0{1}``
Polygon outlining the ROI.
``ROIs.cell0.trial0{2}``
Polygon outlining the first (of ``nRegions``) neuropil region.
``result.cell0.trial0(1, :)``
Final extracted cell signal.
``result.cell0.trial0(2, :)``
Contaminating signal.
``raw.cell0.trial0(1, :)``
This will generate a MAT-file (.mat) which can be loaded into MATLAB.
The MAT-file contains structs for all the experiment output attributes
(:attr:`roi_polys`, :attr:`result`, :attr:`raw`, etc.)
and analysis parameters (:attr:`expansion`, :attr:`nRegions`,
:attr:`alpha`, etc.).
If Δf/f\ :sub:`0` was calculated with :meth:`calc_deltaf`,
:attr:`deltaf_result` and :attr:`deltaf_raw` are also included.

These can be interfaced with as illustrated below.

``result{1, 1}(1, :)``
The separated signal for the first cell and first trial.
This is equivalent to ``experiment.result[0, 0][0, :]`` when
interacting with the :class:`Experiment` object in Python.
``result{roi, trial}(1, :)``
The separated signal for the ``roi``-th cell and ``trial``-th trial.
This is equivalent to
``experiment.result[roi - 1, trial - 1][0, :]`` when
interacting with the :class:`Experiment` object in Python.
``result{roi, trial}(2, :)``
A contaminating signal.
``raw{roi, trial}(1, :)``
Raw measured cell signal, average over the ROI.
``raw.cell0.trial0(2, :)``
Raw signal from first (of ``nRegions``) neuropil region.
This is equivalent to ``experiment.raw[roi - 1, trial - 1][0, :]``
when interacting with the :class:`Experiment` object in Python.
``raw{roi, trial}(2, :)``
Raw signal from first neuropil region (of ``nRegions``).
``roi_polys{roi, trial}{1}``
Polygon outlining the ROI, as an n-by-2 array of coordinates.
``roi_polys{roi, trial}{2}``
Polygon outlining the first neuropil region (of ``nRegions``),
as an n-by-2 array of coordinates.

Examples
--------
Here are some example MATLAB plots.

Plotting raw and decontaminated traces:

.. code:: octave

% Load the FISSA output data
S = load('separated.mat')
% Separated signal for the third ROI, second trial
roi = 3; trial = 2;
% Plot the raw and result traces for the ROI signal
figure; hold on;
plot(S.raw{roi, trial}(1, :));
plot(S.result{roi, trial}(1, :));
title(sprintf('ROI %d, Trial %d', roi, trial));
xlabel('Time (frame number)');
ylabel('Signal intensity (candela per unit area)');
legend({'Raw', 'Result'});

If all ROIs are contiguous and described by a single contour,
the the mean image and ROI locations for one trial can be plotted as
follows:

.. code:: octave

% Load the FISSA output data
S = load('separated.mat')
trial = 1;
figure; hold on;
% Plot the mean image
imagesc(squeeze(S.means(trial, :, :)));
colormap('gray');
% Plot ROI locations
for i_roi = 1:size(S.result, 1);
contour = S.roi_polys{i_roi, trial}{1};
plot(contour(:, 2), contour(:, 1));
end
set(gca, 'YDir', 'reverse');

Parameters
----------
fname : str, optional
Destination for output file. Default is a file named
``"matlab.mat"`` within the cache save directory for the experiment
(the `folder` argument when the ``Experiment`` instance was created).
Destination for output file. The default is a file named
``"separated.mat"`` within the cache save directory for the
experiment (the :attr:`folder` argument when the
:class:`Experiment` instance was created).
legacy : bool, default=False
Whether to use the legacy format of :meth:`save_to_matlab`.
This also changes the default output name to ``"matlab.mat"``.
"""
default_name = "separated.mat"
if legacy:
default_name = "matlab.mat"

# define filename
if fname is None:
if self.folder is None:
raise ValueError(
"fname must be provided if experiment folder is undefined"
)
fname = os.path.join(self.folder, "matlab.mat")

if self.verbosity >= 1:
print("Exporting results to matfile {}".format(fname))
sys.stdout.flush()
fname = os.path.join(self.folder, default_name)

# initialize dictionary to save
M = collections.OrderedDict()

def reformat_dict_for_matlab(orig_dict):
def reformat_dict_for_legacy(orig_dict):
new_dict = collections.OrderedDict()
# loop over cells and trial
for cell in range(len(self.result)):
Expand All @@ -1522,13 +1579,39 @@ def reformat_dict_for_matlab(orig_dict):
new_dict[c_lab][t_lab] = orig_dict[cell][trial]
return new_dict

M["ROIs"] = reformat_dict_for_matlab(self.roi_polys)
M["raw"] = reformat_dict_for_matlab(self.raw)
M["result"] = reformat_dict_for_matlab(self.result)
if getattr(self, "deltaf_raw", None) is not None:
M["df_raw"] = reformat_dict_for_matlab(self.deltaf_raw)
if getattr(self, "deltaf_result", None) is not None:
M["df_result"] = reformat_dict_for_matlab(self.deltaf_result)
if legacy:
M["ROIs"] = reformat_dict_for_legacy(self.roi_polys)
M["raw"] = reformat_dict_for_legacy(self.raw)
M["result"] = reformat_dict_for_legacy(self.result)
if getattr(self, "deltaf_raw", None) is not None:
M["df_raw"] = reformat_dict_for_legacy(self.deltaf_raw)
if getattr(self, "deltaf_result", None) is not None:
M["df_result"] = reformat_dict_for_legacy(self.deltaf_result)
else:
fields = [
"alpha",
"deltaf_raw",
"deltaf_result",
"expansion",
"info",
"max_iter",
"max_tries",
"means",
"method",
"mixmat",
"nCell",
"nRegions",
"raw",
"result",
"roi_polys",
"sep",
"tol",
]
for field in fields:
x = getattr(self, field)
if x is None:
continue
M[field] = x

with warnings.catch_warnings():
warnings.filterwarnings(
Expand All @@ -1537,6 +1620,53 @@ def reformat_dict_for_matlab(orig_dict):
)
savemat(fname, M)

def save_to_matlab(self, fname=None):
r"""
Save the results to a MATLAB file.

.. deprecated:: 1.0.0
Use ``experiment.to_matfile(legacy=True)`` instead.

This will generate a .mat file which can be loaded into MATLAB to
provide structs: ROIs, result, raw.

If Δf/f\ :sub:`0` was calculated, these will also be stored as ``df_result``
and ``df_raw``, which will have the same format as ``result`` and
``raw``.

These can be interfaced with as follows, for cell 0, trial 0:

``ROIs.cell0.trial0{1}``
Polygon outlining the ROI.
``ROIs.cell0.trial0{2}``
Polygon outlining the first (of ``nRegions``) neuropil region.
``result.cell0.trial0(1, :)``
Final extracted cell signal.
``result.cell0.trial0(2, :)``
Contaminating signal.
``raw.cell0.trial0(1, :)``
Raw measured cell signal, average over the ROI.
``raw.cell0.trial0(2, :)``
Raw signal from first (of ``nRegions``) neuropil region.

Parameters
----------
fname : str, optional
Destination for output file. Default is a file named
``"matlab.mat"`` within the cache save directory for the experiment
(the `folder` argument when the ``Experiment`` instance was created).

See Also
--------
Experiment.to_matfile
"""
warnings.warn(
"The experiment.save_to_matlab() method is deprecated."
" Please use experiment.to_matfile(legacy=True) instead.",
DeprecationWarning,
)
return self.to_matfile(fname=fname, legacy=True)


def run_fissa(
images,
Expand Down Expand Up @@ -1602,7 +1732,7 @@ def run_fissa(
Whether to export the data to a MATLAB-compatible .mat file.
If `export_to_matlab` is a string, it is used as the path to the output
file. If ``export_to_matlab=True``, the matfile is saved to the
default path of ``"matlab.mat"`` within the `folder` directory, and
default path of ``"separated.mat"`` within the `folder` directory, and
`folder` must be set. If this is ``None``, the matfile is exported to
the default path if `folder` is set, and otherwise is not exported.
Default is ``False``.
Expand Down Expand Up @@ -1637,7 +1767,7 @@ def run_fissa(
# Save to matfile
if export_to_matlab:
matlab_fname = None if isinstance(export_to_matlab, bool) else export_to_matlab
experiment.save_to_matlab(matlab_fname)
experiment.to_matfile(matlab_fname)
# Return appropriate data
if return_deltaf:
return experiment.deltaf_result
Expand Down
Loading