Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[fix] Reduce saved model size #1425

Merged
merged 7 commits into from
Sep 21, 2023
Merged

[fix] Reduce saved model size #1425

merged 7 commits into from
Sep 21, 2023

Conversation

leoniewgnr
Copy link
Collaborator

@leoniewgnr leoniewgnr commented Sep 19, 2023

🔬 Background

See #1415

🔮 Key changes

removed attribute _trainer

@leoniewgnr
Copy link
Collaborator Author

I removed some attributes, but really unsure where the data is saved. In model.logger.history is saved some big tensor, but the model size is still not the same:

2023-09-19 13:04:06 [INFO]: size of saved fitted model: 288791 (test_utils.py:41)
2023-09-19 13:04:06 [INFO]: testing: save forecast model (test_utils.py:45)
2023-09-19 13:04:06 [INFO]: size of saved forecast model: 43850 (test_utils.py:47)

@codecov
Copy link

codecov bot commented Sep 19, 2023

Codecov Report

Merging #1425 (dcb55d7) into main (b04e141) will decrease coverage by 0.04%.
The diff coverage is 85.00%.

@@            Coverage Diff             @@
##             main    #1425      +/-   ##
==========================================
- Coverage   89.88%   89.85%   -0.04%     
==========================================
  Files          38       38              
  Lines        5071     5084      +13     
==========================================
+ Hits         4558     4568      +10     
- Misses        513      516       +3     
Files Changed Coverage Δ
neuralprophet/utils.py 79.55% <85.00%> (-0.10%) ⬇️

@github-actions
Copy link

github-actions bot commented Sep 19, 2023

Model Benchmark

Benchmark Metric main current diff
PeytonManning MAE_val 0.58162 0.58162 0.0%
PeytonManning RMSE_val 0.72218 0.72218 0.0%
PeytonManning Loss_val 0.01239 0.01239 0.0%
PeytonManning MAE 0.41671 0.41671 0.0%
PeytonManning RMSE 0.55961 0.55961 0.0%
PeytonManning Loss 0.00612 0.00612 0.0%
PeytonManning time 12.8696 13.27 3.11% ⚠️
AirPassengers MAE_val 13.0627 13.0627 0.0%
AirPassengers RMSE_val 15.9453 15.9453 0.0%
AirPassengers Loss_val 0.00131 0.00131 0.0%
AirPassengers MAE 9.88153 9.88153 0.0%
AirPassengers RMSE 11.7354 11.7354 0.0%
AirPassengers Loss 0.00052 0.00052 0.0%
AirPassengers time 5.46486 5.62 2.84%
YosemiteTemps MAE_val 1.34899 1.34899 0.0%
YosemiteTemps RMSE_val 2.00817 2.00817 0.0%
YosemiteTemps Loss_val 0.00078 0.00078 0.0%
YosemiteTemps MAE 1.32133 1.32133 0.0%
YosemiteTemps RMSE 2.13713 2.13713 0.0%
YosemiteTemps Loss 0.00064 0.00064 0.0%
YosemiteTemps time 61.0225 62.58 2.55%
Model training plots

Model Training

PeytonManning

YosemiteTemps

AirPassengers

@leoniewgnr
Copy link
Collaborator Author

I think I found it! In _trainer there was saved some unecessary information!
2023-09-19 13:51:06 [INFO]: size of saved fitted model: 18095 (test_utils.py:43)
2023-09-19 13:51:06 [INFO]: testing: save forecast model (test_utils.py:47)
2023-09-19 13:51:06 [INFO]: size of saved forecast model: 18180 (test_utils.py:50)

@leoniewgnr leoniewgnr self-assigned this Sep 19, 2023
@leoniewgnr leoniewgnr added the status: needs review PR needs to be reviewed by Reviewer(s) label Sep 19, 2023
@leoniewgnr leoniewgnr marked this pull request as ready for review September 19, 2023 20:58
Copy link
Owner

@ourownstory ourownstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good fix!

Do you think we should just include "_trainer" in attrs_to_remove = ["trainer"]?
Or is there a relevant scenario where "_trainer" needs to be saved, but "trainer" not?
If not, we can skip the 'minimal' option and keep the code more lean.

@leoniewgnr
Copy link
Collaborator Author

@ourownstory no probably not, I think so too, we should drop the minimal version and keep the code lean, will change that soon

@leoniewgnr leoniewgnr added status: ready PR is ready to be merged and removed status: needs review PR needs to be reviewed by Reviewer(s) labels Sep 21, 2023
@ourownstory ourownstory merged commit 50d5e41 into main Sep 21, 2023
12 of 14 checks passed
@ourownstory ourownstory deleted the fix-saved-model-size branch September 21, 2023 19:39
@ourownstory ourownstory changed the title [bug] Fix saved model size [fix] Reduce saved model size Dec 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: ready PR is ready to be merged
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Size of saved model file changes dramatically after different calls of predict
2 participants