TY - JOUR
T1 - Soft target and functional complexity reduction
T2 - A hybrid regularization method for genetic programming
AU - Vanneschi, Leonardo
AU - Castelli, Mauro
N1 - info:eu-repo/grantAgreement/FCT/3599-PPCDT/DSAIPA%2FDS%2F0113%2F2019/PT#
info:eu-repo/grantAgreement/FCT/3599-PPCDT/DSAIPA%2FDS%2F0022%2F2018/PT#
info:eu-repo/grantAgreement/FCT/3599-PPCDT/PTDC%2FCCI-INF%2F29168%2F2017/PT#
Vanneschi, L., & Castelli, M. (2021). Soft target and functional complexity reduction: A hybrid regularization method for genetic programming. Expert Systems with Applications, 177, 1-11. [114929]. https://doi.org/10.1016/j.eswa.2021.114929
PY - 2021/9/1
Y1 - 2021/9/1
N2 - Regularization is frequently used in supervised machine learning to prevent models from overfitting. This paper tackles the problem of regularization in genetic programming. We apply, for the first time, soft target regularization, a method recently defined for artificial neural networks, to genetic programming. Also, we introduce a novel measure of functional complexity of the genetic programming individuals, aimed at quantifying their degree of curvature. We experimentally demonstrate that both the use of soft target regularization, and the minimization of the complexity during learning, are often able to reduce overfitting, but they are never able to eliminate it. On the other hand, we demonstrate that the integration of these two strategies into a novel hybrid genetic programming system can completely eliminate overfitting, for all the studied test cases. Last but not least, consistently with what found in the literature, we offer experimental evidence of the fact that the size of the genetic programming models has no correlation with their generalization ability.
AB - Regularization is frequently used in supervised machine learning to prevent models from overfitting. This paper tackles the problem of regularization in genetic programming. We apply, for the first time, soft target regularization, a method recently defined for artificial neural networks, to genetic programming. Also, we introduce a novel measure of functional complexity of the genetic programming individuals, aimed at quantifying their degree of curvature. We experimentally demonstrate that both the use of soft target regularization, and the minimization of the complexity during learning, are often able to reduce overfitting, but they are never able to eliminate it. On the other hand, we demonstrate that the integration of these two strategies into a novel hybrid genetic programming system can completely eliminate overfitting, for all the studied test cases. Last but not least, consistently with what found in the literature, we offer experimental evidence of the fact that the size of the genetic programming models has no correlation with their generalization ability.
KW - Functional complexity
KW - Genetic programming
KW - Hybrid system
KW - Regularization
KW - Soft target
UR - http://www.scopus.com/inward/record.url?scp=85103934730&partnerID=8YFLogxK
UR - http://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcAuth=Alerting&SrcApp=Alerting&DestApp=WOS_CPL&DestLinkType=FullRecord&UT=WOS:000652669700005
U2 - 10.1016/j.eswa.2021.114929
DO - 10.1016/j.eswa.2021.114929
M3 - Article
AN - SCOPUS:85103934730
VL - 177
SP - 1
EP - 11
JO - Expert Systems with Applications
JF - Expert Systems with Applications
SN - 0957-4174
M1 - 114929
ER -