Effects of the Random Forests Hyper-Parameters in Surrogate Models for Multi-Objective Combinatorial Optimization: A Case Study using MOEA/D-RFTS
Keywords:
Decomposition-based optimization, Evolutionary Algorithms, Expensive Objective Functions, Machine Learning, Online LearningAbstract
Surrogate models are techniques to approximate the objective functions of expensive optimization problems. Recently, Random Forests have been studied as a surrogate model technique for combinatorial optimization problems. Nonetheless, Random Forests contain several hyper-parameters that are used to control the prediction process. Despite their importance, research on the effects of these hyper-parameters is scarce. Therefore, this paper performs a systematic investigation of the effects of different combinations of values for the Random Forest hyper-parameters on the approximation of well-known multi-objective combinatorial benchmark problems. The results show that the number of samples to consider when building each tree and the minimum number of samples to be at the leaf node are the two most important hyper-parameters in this context.
Downloads
References
C. A. Coello Coello, S. González Brambila, J. Figueroa Gamboa, M. G. Castillo Tapia, and R. Hernández Gómez, “Evolutionary multiobjective optimization: open research areas and some challenges lying ahead,” Complex & Intelligent Systems, vol. 6, no. 2, pp. 221–236, 2020.
Z. Tan and H. Wang, “A Kriging-Assisted Evolutionary Algorithm Using Feature Selection for Expensive Sparse Multi-Objective optimization,” 2020 IEEE Congress on Evolutionary Computation, CEC 2020 - Conference Proceedings, no. 61976165, 2020.
R. G. Regis, “Evolutionary programming for high-dimensional constrained expensive black-box optimization using radial basis functions,” IEEE Transactions on Evolutionary Computation, vol. 18, no. 3, pp. 326–347, 2014.
Z. Zhou, Y. S. Ong, M. H. Nguyen, and D. Lim, “A study on polynomial regression and gaussian process global surrogate model in hierarchical surrogate-assisted evolutionary algorithm,” in 2005 IEEE Congress on Evolutionary Computation, vol. 3, pp. 2832–2839 Vol. 3, 2005.
H. Wang and Y. Jin, “A Random Forest-Assisted Evolutionary Algorithm for Data-Driven Constrained Multiobjective Combinatorial Optimization of Trauma Systems,” IEEE Transactions on Cybernetics, vol. 50, no. 2, pp. 536–549, 2018.
L. Breiman, “Random Forests,” Machine Learning, vol. 45, pp. 5–32, 2001.
Y. Zheng, X. Fu, and Y. Xuan, “Data-driven optimization based on random forest surrogate,” in 2019 6th International Conference on Systems and Informatics (ICSAI), pp. 487–491, 2019.
Q. Gu, Q. Wang, X. Li, and X. Li, “A surrogate-assisted multi-objective particle swarm optimization of expensive constrained combinatorial optimization problems,” Knowledge-Based Systems, vol. 223, p. 107049, 2021.
Q. Gu, Q. Wang, N. N. Xiong, S. Jiang, and L. Chen, “Surrogate-assisted evolutionary algorithm for expensive constrained multi-objective discrete optimization problems,” Complex & Intelligent Systems, jan 2021.
L. Han and H. Wang, “A random forest assisted evolutionary algorithm using competitive neighborhood search for expensive constrained combinatorial optimization,” Memetic Computing, vol. 13, no. 1, pp. 19–30, 2021.
M. B. De Moraes and G. P. Coelho, “A random forest-assisted decomposition-based evolutionary algorithm for multi-objective combinatorial optimization problems,” in 2022 IEEE Congress on Evolutionary Computation (CEC), pp. 1–8, 2022.
S. K. Dasari, A. Cheddad, and P. Andersson, “Random forest surrogate models to support design space exploration in aerospace use-case,” in Artificial Intelligence Applications and Innovations (J. MacIntyre,
I. Maglogiannis, L. Iliadis, and E. Pimenidis, eds.), (Cham), pp. 532–544, Springer International Publishing, 2019.
E. Zitzler and L. Thiele, “Multiobjective evolutionary algorithms: A comparative case study and the strength Pareto approach,” IEEE Transactions on Evolutionary Computation, vol. 3, no. 4, pp. 257–271, 1999.
Q. Zhang and H. Li, “MOEA/D: A multiobjective evolutionary algorithm based on decomposition,” IEEE Transactions on Evolutionary Computation, vol. 11, no. 6, pp. 712–731, 2007.
C. A. Coello and M. R. Sierra, “A study of the parallelization of a coevolutionary multi-objective evolutionary algorithm,” Lecture Notes in Artificial Intelligence (Subseries of Lecture Notes in Computer Science), vol. 2972, pp. 688–697, 2004.
C. J. Willmott and K. Matsuura, “Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance,” Climate Research, vol. 30, no. 1, pp. 79–82, 2005.
Y. Tian, C. He, R. Cheng, and X. Zhang, “A Multistage Evolutionary Algorithm for Better Diversity Preservation in Multiobjective Optimization,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, no. November, pp. 1–15, 2019.
K. Deb, A. Pratap, S. Agarwal, and T. Meyarivan, “A fast and elitist multiobjective genetic algorithm: NSGA-II,” IEEE Transactions on Evolutionary Computation, vol. 6, pp. 182–197, apr 2002.
F. Wilcoxon, “Individual Comparisons by Ranking Methods,” in Breakthroughs in Statistics, pp. 196–202, 1992.
M. B. de Moraes and G. P. Coelho, “A diversity preservation method for expensive multi-objective combinatorial optimization problems using Novel-First Tabu Search and MOEA/D,” Expert Systems with Applications, vol. 202, p. 117251, sep 2022.
M. B. de Moraes and G. P. Coelho, “Replication data for: Effects of the random forests hyper-parameters in surrogate models for multi-objective combinatorial optimization - A Case Study using MOEA/D-RFTS.”
Repositório de Dados de Pesquisa da Unicamp.
