Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove default settings (None for calibration df and 0.1 for alpha) #1079

Merged
merged 3 commits into from
Dec 21, 2022

Conversation

christymctse
Copy link
Collaborator

🔬 Background

Make calibration dataset and alpha mandatory when the conformal predict function is being called

🔮 Key changes

  • Removed the default setting for calibration_df and alpha under theconformal_predict function
  • Updated the comment for parameter as well

📋 Review Checklist

  • I have performed a self-review of my own code.
  • I have commented my code, added docstrings and data types to function definitions.
  • I have added pytests to check whether my feature / fix works.

Please make sure to follow our best practices in the Contributing guidelines.

@christymctse christymctse marked this pull request as draft December 20, 2022 21:19
@christymctse christymctse marked this pull request as ready for review December 20, 2022 21:25
@christymctse christymctse marked this pull request as draft December 20, 2022 21:25
@christymctse christymctse marked this pull request as ready for review December 20, 2022 22:00
@christymctse christymctse marked this pull request as draft December 21, 2022 01:08
@Kevin-Chen0 Kevin-Chen0 self-requested a review December 21, 2022 17:50
@Kevin-Chen0 Kevin-Chen0 added this to the Release 0.5.1 milestone Dec 21, 2022
@Kevin-Chen0 Kevin-Chen0 linked an issue Dec 21, 2022 that may be closed by this pull request
3 tasks
@Kevin-Chen0 Kevin-Chen0 marked this pull request as ready for review December 21, 2022 17:51
@Kevin-Chen0 Kevin-Chen0 added the status: needs review PR needs to be reviewed by Reviewer(s) label Dec 21, 2022
@codecov-commenter
Copy link

codecov-commenter commented Dec 21, 2022

Codecov Report

Merging #1079 (d5f83a9) into main (ae341cd) will not change coverage.
The diff coverage is 100.00%.

@@           Coverage Diff           @@
##             main    #1079   +/-   ##
=======================================
  Coverage   89.98%   89.98%           
=======================================
  Files          21       21           
  Lines        4784     4784           
=======================================
  Hits         4305     4305           
  Misses        479      479           
Impacted Files Coverage Δ
neuralprophet/forecaster.py 87.87% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

Copy link
Collaborator

@Kevin-Chen0 Kevin-Chen0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Kevin-Chen0 Kevin-Chen0 merged commit 2f2907f into ourownstory:main Dec 21, 2022
@github-actions
Copy link

2f2907f

Model Benchmark

Benchmark Metric main current diff
AirPassengers MAE_val 15.2698 15.2698 0.0%
AirPassengers RMSE_val 19.4209 19.4209 0.0%
AirPassengers Loss_val 0.00195 0.00195 0.0%
AirPassengers MAE 9.82902 9.82902 0.0%
AirPassengers RMSE 11.7005 11.7005 0.0%
AirPassengers Loss 0.00056 0.00056 0.0%
AirPassengers time 5.28483 6.33 19.78%
YosemiteTemps MAE_val 1.72949 1.72948 -0.0%
YosemiteTemps RMSE_val 2.27386 2.27386 -0.0%
YosemiteTemps Loss_val 0.00096 0.00096 -0.0%
YosemiteTemps MAE 1.45189 1.45189 -0.0%
YosemiteTemps RMSE 2.16631 2.16631 -0.0%
YosemiteTemps Loss 0.00066 0.00066 0.0%
YosemiteTemps time 115.93 143.79 24.03%
PeytonManning MAE_val 0.64636 0.64636 -0.0%
PeytonManning RMSE_val 0.79276 0.79276 -0.0%
PeytonManning Loss_val 0.01494 0.01494 -0.0%
PeytonManning MAE 0.42701 0.42701 0.0%
PeytonManning RMSE 0.57032 0.57032 -0.0%
PeytonManning Loss 0.00635 0.00635 0.0%
PeytonManning time 14.9977 17.72 18.15%
Model training plots

Model Training

PeytonManning

YosemiteTemps

AirPassengers

@christymctse christymctse deleted the remove_default_setting branch February 1, 2023 09:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority:P2 Medium priority status: needs review PR needs to be reviewed by Reviewer(s)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Conformal Prediction V1.1 tasks
3 participants