Time in model training benchmark

[karl-richter]

Added training time to the benchmarks + only hide the figures, but not the metrics.

[karl-richter]

@Kevin-Chen0 @ourownstory @noxan
Can someone review and merge this pr asap pls? Need the time in the benchmarks to check that the Lightning speedup works.

[github-actions]

52bd14d

Model Benchmark

Benchmark	Metric	main	current	diff
YosemiteTemps	MAE_val	1.72949	1.72949	0.0%	✅
YosemiteTemps	RMSE_val	2.27386	2.27386	0.0%	✅
YosemiteTemps	Loss_val	0.00096	0.00096	0.0%	✅
YosemiteTemps	RegLoss_val	0	0	0.0%	✅
YosemiteTemps	epoch	84	84	0.0%	✅
YosemiteTemps	MAE	1.45189	1.45189	0.0%	✅
YosemiteTemps	RMSE	2.16631	2.16631	0.0%	✅
YosemiteTemps	Loss	0.00066	0.00066	0.0%	✅
YosemiteTemps	RegLoss	0	0	0.0%	✅
YosemiteTemps	time	nan	103.3	0.0%	✅
PeytonManning	MAE_val	0.64636	0.64636	0.0%	✅
PeytonManning	RMSE_val	0.79276	0.79276	0.0%	✅
PeytonManning	Loss_val	0.01494	0.01494	0.0%	✅
PeytonManning	RegLoss_val	0	0	0.0%	✅
PeytonManning	epoch	37	37	0.0%	✅
PeytonManning	MAE	0.42701	0.42701	0.0%	✅
PeytonManning	RMSE	0.57032	0.57032	0.0%	✅
PeytonManning	Loss	0.00635	0.00635	0.0%	✅
PeytonManning	RegLoss	0	0	0.0%	✅
PeytonManning	time	nan	13.13	0.0%	✅
AirPassengers	MAE_val	15.2698	15.2698	0.0%	✅
AirPassengers	RMSE_val	19.4209	19.4209	0.0%	✅
AirPassengers	Loss_val	0.00195	0.00195	0.0%	✅
AirPassengers	RegLoss_val	0	0	0.0%	✅
AirPassengers	epoch	89	89	0.0%	✅
AirPassengers	MAE	9.82902	9.82902	0.0%	✅
AirPassengers	RMSE	11.7005	11.7005	0.0%	✅
AirPassengers	Loss	0.00056	0.00056	0.0%	✅
AirPassengers	RegLoss	0	0	0.0%	✅
AirPassengers	time	nan	5.11	0.0%	✅

Model training plots

Model Training

PeytonManning

YosemiteTemps

AirPassengers

[codecov-commenter]

Codecov Report

Merging #995 (52bd14d) into main (fe1d759) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #995   +/-   ##
=======================================
  Coverage   89.81%   89.81%           
=======================================
  Files          18       18           
  Lines        4632     4632           
=======================================
  Hits         4160     4160           
  Misses        472      472           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[noxan]

Looks good to me, one minor comment on collapsible metrics - ready to merge from my side.

[noxan]

I'd prefer to have an option to also collapse the regular metrics - no hard feelings and no blocker for sure 👍

[karl-richter]

Totally, agree, in the current setting there are too many metrics. Lets agree in our weekly meeting on the few key metrics that we always display and make all the others collabsable. I am just afraid that people do not check their metrics if its not visible that their pr reduces performance.