Skip to content

matbench_v0.1: MODNet (v0.1.10)

Algorithm description:

MODNet, the Materials Optimal Descriptor Network (v0.1.10). A feed-forward neural network, using all compatible matminer features and a relevance-redundancy based feature selection algorithm. Hyperparameter optimisation is performed with a nested grid search for the 9 smaller tasks, and with a genetic algorithm for the 4 larger tasks (matbench_perovskites, matbench_mp_gap, matbench_mp_is_metal, matbench_mp_eform. Benchmark results were loaded from https://github.com/ml-evs/modnet-matbench/releases/tag/v0.3.0, archived at 10.5281/zenodo.5562338.

Notes:

None

Raw data download and example notebook available on the matbench repo.

References (in bibtex format):

('@article{De_Breuck_2021, doi = {10.1088/1361-648x/ac1280}, url = '
 '{https://doi.org/10.1088/1361-648x/ac1280}, year = 2021, month = {jul}, '
 'publisher = {{IOP} Publishing}, volume = {33}, number = {40}, pages = '
 '{404002}, author = {Pierre-Paul De Breuck and Matthew L Evans and Gian-Marco '
 'Rignanese}, title = {Robust model benchmarking and bias-imbalance in '
 'data-driven materials science: a case study on {MODNet}}, journal = {Journal '
 'of Physics: Condensed Matter}, abstract = {As the number of novel '
 'data-driven approaches to material science continues to grow, it is crucial '
 'to perform consistent quality, reliability and applicability assessments of '
 'model performance. In this paper, we benchmark the Materials Optimal '
 'Descriptor Network (MODNet) method and architecture against the recently '
 'released MatBench v0.1, a curated test suite of materials datasets. MODNet '
 'is shown to outperform current leaders on 6 of the 13 tasks, while closely '
 'matching the current leaders on a further 2 tasks; MODNet performs '
 'particularly well when the number of samples is below 10\xa0000. Attention '
 'is paid to two topics of concern when benchmarking models. First, we '
 'encourage the reporting of a more diverse set of metrics as it leads to a '
 'more comprehensive and holistic comparison of model performance. Second, an '
 'equally important task is the uncertainty assessment of a model towards a '
 'target domain. Significant variations in validation errors can be observed, '
 'depending on the imbalance and bias in the training set (i.e., similarity '
 'between training and application space). By using an ensemble MODNet model, '
 'confidence intervals can be built and the uncertainty on individual '
 'predictions can be quantified. Imbalance and bias issues are often '
 'overlooked, and yet are important for successful real-world applications of '
 'machine learning in materials science and condensed matter.}}, '
 '@article{DeBreuck2021, doi = {10.1038/s41524-021-00552-2}, url = '
 '{https://doi.org/10.1038/s41524-021-00552-2}, year = {2021}, month = jun, '
 'publisher = {Springer Science and Business Media {LLC}}, volume = {7}, '
 'number = {1}, author = {Pierre-Paul De Breuck and Geoffroy Hautier and '
 'Gian-Marco Rignanese}, title = {Materials property prediction for limited '
 'datasets enabled by feature selection and joint learning with {MODNet}}, '
 'journal = {npj Computational Materials}}')

User metadata:

{}

Metadata:

tasks recorded 13/13
complete?
composition complete?
structure complete?
regression complete?
classification complete?

Software Requirements

{'python': ['modnet==0.1.10', 'matbench==0.2.0']}

Task data:

matbench_dielectric

Fold scores
fold mae rmse mape* max_error
fold_0 0.1939 0.7043 0.0657 13.9549
fold_1 0.2669 1.0559 0.0897 19.4132
fold_2 0.4138 2.9360 0.0873 58.9519
fold_3 0.2880 2.2447 0.0593 52.4648
fold_4 0.3223 1.6518 0.1040 28.0662
Fold score stats
metric mean max min std
mae 0.2970 0.4138 0.1939 0.0720
rmse 1.7185 2.9360 0.7043 0.8039
mape* 0.0812 0.1040 0.0593 0.0164
max_error 34.5702 58.9519 13.9549 17.9539
Fold parameters
fold params dict
fold_0 {'std': [[0.06733036786317825], [0.0744570940732956], [0.4551847279071808], [0.6979547739028931], [0.05083160847425461], [0.09550821781158447], [0.19389964640140533], [0.45796462893486023], [0.0457437...
fold_1 {'std': [[0.072625070810318], [0.20568081736564636], [0.06122368574142456], [0.1985706239938736], [0.13508345186710358], [0.5729472637176514], [0.18508531153202057], [0.36901697516441345], [0.15902499...
fold_2 {'std': [[0.07455231994390488], [0.40818431973457336], [0.17178542912006378], [0.1756463497877121], [0.05205323547124863], [0.18280482292175293], [0.029979035258293152], [0.13881000876426697], [0.1849...
fold_3 {'std': [[0.07038000971078873], [0.052069056779146194], [0.11985469609498978], [0.2917916774749756], [0.07153098285198212], [0.16248461604118347], [0.03178466856479645], [0.06994788348674774], [0.0775...
fold_4 {'std': [[0.062172286212444305], [0.1020524799823761], [0.046170447021722794], [0.2582769989967346], [0.5205304622650146], [0.09285475313663483], [0.04396134242415428], [0.1190856322646141], [0.159720...

matbench_expt_gap

Fold scores
fold mae rmse mape* max_error
fold_0 0.3272 0.7062 0.3510 6.3096
fold_1 0.3594 0.7340 0.3187 6.3544
fold_2 0.3845 0.8563 0.3841 9.8567
fold_3 0.3259 0.6888 0.3231 5.1081
fold_4 0.3382 0.7334 0.4075 6.5141
Fold score stats
metric mean max min std
mae 0.3470 0.3845 0.3259 0.0222
rmse 0.7437 0.8563 0.6888 0.0588
mape* 0.3569 0.4075 0.3187 0.0345
max_error 6.8286 9.8567 5.1081 1.5952
Fold parameters
fold params dict
fold_0 {'std': [[0.3934377431869507], [0.6812934875488281], [0.46057939529418945], [0.21048687398433685], [0.39122024178504944], [2.184469699859619], [1.0323148965835571], [1.1202787160873413], [0.8973723649...
fold_1 {'std': [[0.42153677344322205], [0.10141757875680923], [0.2717689573764801], [0.0055972738191485405], [0.12942852079868317], [0.19773989915847778], [0.1537284404039383], [0.152150496840477], [0.127241...
fold_2 {'std': [[0.5310537219047546], [0.09472547471523285], [0.6016039848327637], [0.7606176137924194], [0.2108703851699829], [0.3892253637313843], [0.8048807382583618], [0.058867525309324265], [0.236589178...
fold_3 {'std': [[0.4957612454891205], [0.24328213930130005], [1.1489912271499634], [0.4026401937007904], [0.38004472851753235], [0.235760897397995], [0.2802310287952423], [0.23525512218475342], [0.9116547703...
fold_4 {'std': [[0.17024394869804382], [1.0889294147491455], [0.0037015366833657026], [0.45974406599998474], [0.7000935673713684], [1.2791191339492798], [1.037060260772705], [1.1216192245483398], [1.26752507...

matbench_expt_is_metal

Fold scores
fold accuracy balanced_accuracy f1 rocauc
fold_0 0.9269 0.9269 0.9255 0.9269
fold_1 0.9136 0.9136 0.9121 0.9136
fold_2 0.9177 0.9177 0.9173 0.9177
fold_3 0.9177 0.9177 0.9169 0.9177
fold_4 0.9045 0.9045 0.9049 0.9045
Fold score stats
metric mean max min std
accuracy 0.9161 0.9269 0.9045 0.0073
balanced_accuracy 0.9161 0.9269 0.9045 0.0072
f1 0.9153 0.9255 0.9049 0.0068
rocauc 0.9161 0.9269 0.9045 0.0072
Fold parameters
fold params dict
fold_0 {'std': [[0.17845642566680908, 0.1784563958644867], [0.38880154490470886, 0.38880154490470886], [0.11527526378631592, 0.11527524888515472], [0.29507318139076233, 0.29507318139076233], [0.4294841885566...
fold_1 {'std': [[0.250985711812973, 0.250985711812973], [0.15300564467906952, 0.15300562977790833], [0.11072004586458206, 0.11072004586458206], [0.07669822871685028, 0.07669822126626968], [0.1074658855795860...
fold_2 {'std': [[0.16936197876930237, 0.16936197876930237], [0.28381362557411194, 0.2838136553764343], [0.3199393153190613, 0.3199393153190613], [0.14618311822414398, 0.1461830586194992], [0.1249608173966407...
fold_3 {'std': [[0.13702887296676636, 0.13702887296676636], [0.1351342797279358, 0.1351342648267746], [0.40080583095550537, 0.40080583095550537], [0.1579451709985733, 0.1579451709985733], [0.1655253022909164...
fold_4 {'std': [[0.28618890047073364, 0.28618890047073364], [0.27044594287872314, 0.27044594287872314], [0.29565274715423584, 0.29565274715423584], [0.29736053943634033, 0.29736053943634033], [0.237021714448...

matbench_glass

Fold scores
fold accuracy balanced_accuracy f1 rocauc
fold_0 0.8759 0.8262 0.9153 0.8262
fold_1 0.8539 0.7783 0.9030 0.7783
fold_2 0.8565 0.8063 0.9016 0.8063
fold_3 0.8856 0.8402 0.9217 0.8402
fold_4 0.8662 0.8023 0.9102 0.8023
Fold score stats
metric mean max min std
accuracy 0.8676 0.8856 0.8539 0.0119
balanced_accuracy 0.8107 0.8402 0.7783 0.0212
f1 0.9104 0.9217 0.9016 0.0075
rocauc 0.8107 0.8402 0.7783 0.0212
Fold parameters
fold params dict
fold_0 {'std': [[0.1912706196308136, 0.1912706196308136], [0.3220626413822174, 0.3220626413822174], [0.38618433475494385, 0.38618433475494385], [0.3689897954463959, 0.36898982524871826], [0.296833336353302, ...
fold_1 {'std': [[0.3554098606109619, 0.3554098606109619], [0.19303181767463684, 0.19303181767463684], [0.2967971861362457, 0.2967972159385681], [0.3302050232887268, 0.3302050232887268], [0.2770985960960388, ...
fold_2 {'std': [[0.3961077332496643, 0.39610767364501953], [0.19110238552093506, 0.19110238552093506], [0.22688446938991547, 0.22688449919223785], [0.34065985679626465, 0.3406599164009094], [0.25127366185188...
fold_3 {'std': [[0.18866945803165436, 0.18866944313049316], [0.17435620725154877, 0.17435620725154877], [0.18012933433055878, 0.1801293045282364], [0.11812251806259155, 0.11812251806259155], [0.1196716576814...
fold_4 {'std': [[0.16088004410266876, 0.16088005900382996], [0.2377340942621231, 0.2377340942621231], [0.3177623748779297, 0.3177623748779297], [0.3061956465244293, 0.3061956465244293], [0.2921837270259857, ...

matbench_jdft2d

Fold scores
fold mae rmse mape* max_error
fold_0 27.5769 49.7512 21.3632 243.2504
fold_1 27.9722 63.3103 0.2282 364.1909
fold_2 51.3402 142.7963 0.6111 845.7528
fold_3 26.9141 52.8447 0.2724 311.7558
fold_4 38.8806 152.4413 0.4853 1534.9797
Fold score stats
metric mean max min std
mae 34.5368 51.3402 26.9141 9.4959
rmse 92.2288 152.4413 49.7512 45.5508
mape* 4.5920 21.3632 0.2282 8.3868
max_error 659.9859 1534.9797 243.2504 486.3231
Fold parameters
fold params dict
fold_0 {'std': [[9.497343063354492], [15.862295150756836], [74.97210693359375], [25.96040916442871], [47.26897048950195], [14.80854606628418], [22.77548599243164], [10.362432479858398], [8.255328178405762], ...
fold_1 {'std': [[5.382687568664551], [25.67997932434082], [16.605792999267578], [7.763948917388916], [10.631340026855469], [36.21831512451172], [12.867671012878418], [3.2303359508514404], [38.958377838134766...
fold_2 {'std': [[71.73993682861328], [18.688243865966797], [7.084332466125488], [16.097488403320312], [83.72747802734375], [12.528894424438477], [16.004690170288086], [14.574416160583496], [7.346397399902344...
fold_3 {'std': [[2.2033019065856934], [17.148666381835938], [6.929365634918213], [3.3733177185058594], [19.175621032714844], [9.659783363342285], [2.456592321395874], [13.089242935180664], [44.94028091430664...
fold_4 {'std': [[24.92951202392578], [17.333660125732422], [10.269680976867676], [4.752265453338623], [3.5876128673553467], [4.854499340057373], [12.900960922241211], [6.644251823425293], [9.120869636535645]...

matbench_log_gvrh

Fold scores
fold mae rmse mape* max_error
fold_0 0.0731 0.1089 0.0576 0.9014
fold_1 0.0738 0.1111 0.0579 1.1745
fold_2 0.0731 0.1101 0.0587 0.9076
fold_3 0.0738 0.1115 0.0567 0.9225
fold_4 0.0718 0.1101 0.0560 0.8007
Fold score stats
metric mean max min std
mae 0.0731 0.0738 0.0718 0.0007
rmse 0.1103 0.1115 0.1089 0.0009
mape* 0.0574 0.0587 0.0560 0.0009
max_error 0.9413 1.1745 0.8007 0.1243
Fold parameters
fold params dict
fold_0 {'std': [0.06334669888019562, 0.04876908287405968, 0.0713210254907608, 0.06149518862366676, 0.05233978480100632, 0.053833525627851486, 0.045166339725255966, 0.09107258170843124, 0.08312246203422546, 0...
fold_1 {'std': [0.04058562591671944, 0.09664303809404373, 0.06196340546011925, 0.07074710726737976, 0.05361659824848175, 0.05300111323595047, 0.04533914476633072, 0.060226064175367355, 0.15155699849128723, 0...
fold_2 {'std': [0.04066888242959976, 0.05564378947019577, 0.05513373762369156, 0.03629153221845627, 0.08530019223690033, 0.0363982692360878, 0.07014258950948715, 0.07834821194410324, 0.056601572781801224, 0....
fold_3 {'std': [0.06376504898071289, 0.2838374972343445, 0.025865282863378525, 0.04888685792684555, 0.18576562404632568, 0.045733798295259476, 0.047175027430057526, 0.04196206107735634, 0.06469003856182098, ...
fold_4 {'std': [0.06425822526216507, 0.07589271664619446, 0.04857879504561424, 0.07567392289638519, 0.07976284623146057, 0.05443073436617851, 0.0474713109433651, 0.08143744617700577, 0.10169852524995804, 0.0...

matbench_log_kvrh

Fold scores
fold mae rmse mape* max_error
fold_0 0.0536 0.1013 0.0356 1.5366
fold_1 0.0559 0.1079 0.0366 1.2998
fold_2 0.0510 0.0949 0.0340 1.1808
fold_3 0.0585 0.1126 0.0418 1.1355
fold_4 0.0549 0.1046 0.0370 1.3202
Fold score stats
metric mean max min std
mae 0.0548 0.0585 0.0510 0.0025
rmse 0.1043 0.1126 0.0949 0.0060
mape* 0.0370 0.0418 0.0340 0.0026
max_error 1.2946 1.5366 1.1355 0.1397
Fold parameters
fold params dict
fold_0 {'std': [0.03473027050495148, 0.05344022810459137, 0.04392522946000099, 0.09693300724029541, 0.0621185339987278, 0.0515923835337162, 0.034392938017845154, 0.0368841215968132, 0.09843463450670242, 0.03...
fold_1 {'std': [0.037284620106220245, 0.0660589188337326, 0.05483892932534218, 0.05504067987203598, 0.045397065579891205, 0.053156472742557526, 0.04068203642964363, 0.04492218419909477, 0.1503378003835678, 0...
fold_2 {'std': [0.04053986072540283, 0.039209164679050446, 0.04213540256023407, 0.036292385309934616, 0.06385202705860138, 0.032488591969013214, 0.0784469023346901, 0.0694998949766159, 0.050309233367443085, ...
fold_3 {'std': [0.04948587343096733, 0.11705353856086731, 0.025648461654782295, 0.03585298731923103, 0.11334579437971115, 0.03046250157058239, 0.040365662425756454, 0.03331249952316284, 0.038164108991622925,...
fold_4 {'std': [0.04571979492902756, 0.0366676039993763, 0.036114685237407684, 0.06556463986635208, 0.07480020076036453, 0.03638936206698418, 0.05547630041837692, 0.10959770530462265, 0.16662324965000153, 0....

matbench_mp_e_form

Fold scores
fold mae rmse mape* max_error
fold_0 0.0402 0.0817 0.3786 4.0438
fold_1 0.0497 0.1018 0.3121 4.8803
fold_2 0.0475 0.0905 0.2562 1.6230
fold_3 0.0464 0.0889 0.3515 1.5189
fold_4 0.0400 0.0812 0.2882 3.3787
Fold score stats
metric mean max min std
mae 0.0448 0.0497 0.0400 0.0039
rmse 0.0888 0.1018 0.0812 0.0075
mape* 0.3173 0.3786 0.2562 0.0436
max_error 3.0889 4.8803 1.5189 1.3281
Fold parameters
fold params dict
fold_0 {'std': [[0.0636366754770279], [0.02924380451440811], [0.11916627734899521], [0.11677506566047668], [0.1760452687740326], [0.1811746507883072], [0.08742818236351013], [0.1322329342365265], [0.14015400...
fold_1 {'std': [[0.10467445850372314], [0.2568114399909973], [0.08591523766517639], [0.11847899109125137], [1.0217572450637817], [0.2770746350288391], [0.20971281826496124], [0.19037210941314697], [0.0730839...
fold_2 {'std': [[0.07034026086330414], [0.07515157759189606], [0.1308293640613556], [0.23308764398097992], [0.2118426114320755], [0.1338074803352356], [0.17896589636802673], [0.09289371967315674], [0.0988285...
fold_3 {'std': [[0.17922396957874298], [0.21060164272785187], [0.04639369249343872], [0.0925942063331604], [0.06210273131728172], [0.28422462940216064], [0.2840571105480194], [0.2760363817214966], [0.1231188...
fold_4 {'std': [[0.1517249494791031], [0.13391436636447906], [0.40770843625068665], [0.13683228194713593], [0.124815434217453], [0.042988162487745285], [0.13916344940662384], [0.06709353625774384], [0.053676...

matbench_mp_gap

Fold scores
fold mae rmse mape* max_error
fold_0 0.2147 0.4441 2.8966 5.0558
fold_1 0.2161 0.4484 2.6899 6.2874
fold_2 0.2165 0.4433 4.1912 7.5685
fold_3 0.2309 0.4705 4.6749 6.9325
fold_4 0.2211 0.4564 4.9590 4.9406
Fold score stats
metric mean max min std
mae 0.2199 0.2309 0.2147 0.0059
rmse 0.4525 0.4705 0.4433 0.0101
mape* 3.8823 4.9590 2.6899 0.9248
max_error 6.1570 7.5685 4.9406 1.0299
Fold parameters
fold params dict
fold_0 {'std': [[0.2779600918292999], [0.1588134467601776], [0.0013879217440262437], [0.0013879217440262437], [0.0013879217440262437], [0.15315547585487366], [0.4301016926765442], [0.19215451180934906], [0.0...
fold_1 {'std': [[0.0023871688172221184], [0.0023871688172221184], [0.0023871688172221184], [0.0023871688172221184], [0.0023871688172221184], [0.0023871688172221184], [0.35058045387268066], [0.439932823181152...
fold_2 {'std': [[0.17874807119369507], [0.0015997957671061158], [0.0015997957671061158], [0.35170578956604004], [0.0015997957671061158], [0.0015997957671061158], [0.0015997957671061158], [0.00156632636208087...
fold_3 {'std': [[0.004137647803872824], [0.004137647803872824], [0.004137647803872824], [0.004137647803872824], [0.004137647803872824], [0.004137647803872824], [0.004137647803872824], [0.004137647803872824],...
fold_4 {'std': [[0.4364481568336487], [0.0038157568778842688], [0.003807253669947386], [0.003807792905718088], [0.003815052565187216], [0.0038899907376617193], [0.0038147747982293367], [0.19303250312805176],...

matbench_mp_is_metal

Fold scores
fold accuracy balanced_accuracy f1 rocauc
fold_0 0.8515 0.8415 0.8175 0.8415
fold_1 0.8824 0.8709 0.8526 0.8709
fold_2 0.5650 0.5000 0.0000 0.5000
fold_3 0.8575 0.8447 0.8200 0.8447
fold_4 0.8588 0.8453 0.8203 0.8453
Fold score stats
metric mean max min std
accuracy 0.8031 0.8824 0.5650 0.1195
balanced_accuracy 0.7805 0.8709 0.5000 0.1406
f1 0.6621 0.8526 0.0000 0.3313
rocauc 0.7805 0.8709 0.5000 0.1406
Fold parameters
fold params dict
fold_0 {'std': [[0.0416233129799366, 0.041623327881097794], [0.06803157180547714, 0.06803156435489655], [0.008189204148948193, 0.008189203217625618], [0.0037693644408136606, 0.003769365604966879], [0.0107376...
fold_1 {'std': [[0.2837996482849121, 0.2837996482849121], [0.28376519680023193, 0.28376519680023193], [0.2669283449649811, 0.2669283449649811], [0.2666773200035095, 0.2666773200035095], [0.3405170142650604, ...
fold_2 {'std': [[0.1777520626783371, 0.1777520775794983], [0.1777520626783371, 0.1777520775794983], [0.1777520626783371, 0.1777520775794983], [0.1777520626783371, 0.1777520775794983], [0.1777520626783371, 0....
fold_3 {'std': [[0.22140955924987793, 0.22140958905220032], [0.2370903342962265, 0.2370903342962265], [0.2370903342962265, 0.2370903342962265], [0.2370903342962265, 0.2370903342962265], [0.2370903342962265, ...
fold_4 {'std': [[0.09707242995500565, 0.09707243740558624], [0.259949266910553, 0.2599492371082306], [0.09707242995500565, 0.09707243740558624], [0.23583407700061798, 0.2358340471982956], [0.2358340770006179...

matbench_perovskites

Fold scores
fold mae rmse mape* max_error
fold_0 0.0932 0.1304 0.0970 0.8705
fold_1 0.0939 0.1283 0.1058 1.0063
fold_2 0.0861 0.1216 0.0939 0.9432
fold_3 0.0892 0.1274 0.0995 0.8501
fold_4 0.0914 0.1310 0.0894 1.1780
Fold score stats
metric mean max min std
mae 0.0908 0.0939 0.0861 0.0028
rmse 0.1277 0.1310 0.1216 0.0033
mape* 0.0971 0.1058 0.0894 0.0055
max_error 0.9696 1.1780 0.8501 0.1180
Fold parameters
fold params dict
fold_0 {'std': [[0.10714199393987656], [0.0770525336265564], [0.07103670388460159], [0.047520048916339874], [0.09854762256145477], [0.054435212165117264], [0.06670265644788742], [0.12760771811008453], [0.110...
fold_1 {'std': [[0.06410787999629974], [0.09309504926204681], [0.07608579844236374], [0.08194778114557266], [0.11951383203268051], [0.07481898367404938], [0.04808051139116287], [0.08761747926473618], [0.0603...
fold_2 {'std': [[0.10084810853004456], [0.10042519122362137], [0.10863561928272247], [0.11654899269342422], [0.08363119512796402], [0.11726558208465576], [0.12616018950939178], [0.104669950902462], [0.083491...
fold_3 {'std': [[0.0883394405245781], [0.0751652866601944], [0.07409299165010452], [0.12206761538982391], [0.10416710376739502], [0.11867869645357132], [0.15680250525474548], [0.07212385535240173], [0.066893...
fold_4 {'std': [[0.15121375024318695], [0.09383570402860641], [0.10639220476150513], [0.09952569007873535], [0.060146454721689224], [0.0721314549446106], [0.09489388763904572], [0.0831003338098526], [0.09796...

matbench_phonons

Fold scores
fold mae rmse mape* max_error
fold_0 40.2218 99.9366 0.0661 1031.8168
fold_1 41.1190 83.0600 0.0680 721.2376
fold_2 38.8526 70.0409 0.0705 452.0254
fold_3 37.1039 78.3636 0.0710 662.8152
fold_4 36.4648 59.7092 0.0665 342.3226
Fold score stats
metric mean max min std
mae 38.7524 41.1190 36.4648 1.7732
rmse 78.2220 99.9366 59.7092 13.4507
mape* 0.0684 0.0710 0.0661 0.0020
max_error 642.0435 1031.8168 342.3226 238.5648
Fold parameters
fold params dict
fold_0 {'std': [[26.26407814025879], [175.91537475585938], [15.736849784851074], [16.808921813964844], [18.036314010620117], [16.40987777709961], [31.393617630004883], [26.229381561279297], [16.6152362823486...
fold_1 {'std': [[32.547828674316406], [18.21637535095215], [34.24558639526367], [27.42135238647461], [23.881690979003906], [151.67642211914062], [18.94878578186035], [19.094982147216797], [16.469425201416016...
fold_2 {'std': [[21.542694091796875], [21.627588272094727], [15.593945503234863], [67.26934814453125], [11.872613906860352], [11.879145622253418], [12.56555461883545], [24.557519912719727], [22.6404819488525...
fold_3 {'std': [[30.841398239135742], [17.576093673706055], [20.379390716552734], [24.910297393798828], [13.184524536132812], [24.256025314331055], [26.51211166381836], [18.35163116455078], [17.3094120025634...
fold_4 {'std': [[11.17216968536377], [16.963390350341797], [32.19032287597656], [16.677236557006836], [27.273052215576172], [28.90708351135254], [25.442333221435547], [21.135835647583008], [16.36865615844726...

matbench_steels

Fold scores
fold mae rmse mape* max_error
fold_0 112.2905 189.8130 0.0707 931.3261
fold_1 81.9908 115.9188 0.0604 404.5644
fold_2 99.3739 139.4921 0.0699 411.7195
fold_3 93.2877 152.1443 0.0672 827.5305
fold_4 94.1265 152.3995 0.0709 672.9292
Fold score stats
metric mean max min std
mae 96.2139 112.2905 81.9908 9.8352
rmse 149.9535 189.8130 115.9188 23.9473
mape* 0.0678 0.0709 0.0604 0.0039
max_error 649.6139 931.3261 404.5644 213.6365
Fold parameters
fold params dict
fold_0 {'std': [[181.11865234375], [192.23825073242188], [42.135902404785156], [54.896053314208984], [164.36167907714844], [53.29085159301758], [41.6890869140625], [68.6667709350586], [96.96005249023438], [3...
fold_1 {'std': [[69.18898010253906], [72.29112243652344], [54.083961486816406], [66.05774688720703], [51.890892028808594], [41.604408264160156], [207.0948028564453], [56.53154373168945], [111.53943634033203]...
fold_2 {'std': [[179.28602600097656], [267.37554931640625], [125.63412475585938], [117.67745971679688], [43.56736755371094], [56.37009811401367], [37.02374267578125], [116.51256561279297], [162.7328186035156...
fold_3 {'std': [[264.0724182128906], [320.3443603515625], [69.12999725341797], [98.54374694824219], [102.41849517822266], [85.94804382324219], [39.834190368652344], [159.82240295410156], [59.77300262451172],...
fold_4 {'std': [[172.47418212890625], [83.7674560546875], [355.5929260253906], [119.87616729736328], [59.057350158691406], [119.39688873291016], [50.491127014160156], [79.44507598876953], [93.9663314819336],...