Time-series failure prediction on small datasets using machine learning
Keywords:
Machine Learning, Remaining Useful Life, Time-Series, Empirical Mode Decomposition, Predictive ModelsAbstract
Condition-based maintenance is a decision-making strategy using condition monitoring information to optimize the availability of operational plants. In this context, machine learning techniques are useful and have been used in predicting the remaining useful life (RUL) of equipment to ensure the overall safety and reliability of the system through maintenance policies and, consequently, reducing costs arising from the failure. These databases are not large which is tricky for data-driven models. In this study, we consider five different databases containing the failure times from distinct real-world equipment. Here, four different regression algorithms were compared for RUL prediction, namely: Support Vector Regression (SVR), Decision Tree (DT), Multilayer Perceptron (MLP) and K-Nearest Neighbors (KNN). Furthermore, aiming to improve the data quality, the Empirical Mode Decomposition (EMD) was used, which is responsible for pre-processing the input data used on the predictive modeling. We optimize the models’ hyperparameters using grid-search cross-validation algorithm and the performance of each model is compared using the normalized root mean squared error (NRMSE). Considering the datasets analyzed, KNN model proves to be the most promising to perform the prognostic task in small datasets, adapting itself to the distinct characteristics of the different databases. In addition, we mention the better performance after optimizing the hyperparameters, which avoided overfitting problems and had a low computational cost for the problems analyzed here.
Downloads
References
1016/S0360-8352(02)00036-0.
Z. Tian, L. Wong, and N. Safaei, “A neural network approach for remaining useful life prediction utilizing both failure and suspension histories,” Mech Syst Signal Process, vol. 24, no. 5, pp. 1542–1555, Jul. 2010, doi: 10.1016/j.ymssp.2009.11.005.
C. Ordóñez, F. Sánchez Lasheras, J. Roca-Pardiñas, and F. J. de C. Juez, “A hybrid ARIMA–SVM model for the study of the remaining useful life of aircraft engines,” J Comput Appl Math, vol. 346, pp. 184–191, Jan. 2019, doi: 10.1016/j.cam.2018.07.008.
P. J. García Nieto, E. García-Gonzalo, F. Sánchez Lasheras, and F. J. de Cos Juez, “Hybrid PSO–SVM-based method for forecasting of the remaining useful life for aircraft engines and evaluation of its reliability,” Reliab Eng Syst Saf, vol. 138, pp. 219–231, Jun. 2015, doi: 10.1016/j.ress.2015.02.001.
L. Wang, D. Zhou, H. Zhang, W. Zhang, and J. Chen, “Application of Relative Entropy and Gradient Boosting Decision Tree to Fault Prognosis in Electronic Circuits,” Symmetry (Basel), vol. 10, no. 10, p. 495, Oct. 2018, doi: 10.3390/sym10100495.
T. Wuest, D. Weimer, C. Irgens, and K.-D. Thoben, “Machine learning in manufacturing: advantages, challenges, and applications,” Prod Manuf Res, vol. 4, no. 1, pp. 23–45, Jan. 2016, doi: 10.1080/21693277.2016.1192517.
N. E. Huang et al., “The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis,” Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences, vol. 454, no. 1971, pp. 903–995, Mar. 1998, doi: 10.1098/rspa.1998.0193.
B. Sales da Cunha, M. das Chagas Moura, C. Souto Maior, A. Cláudia Negreiros, and I. Didier Lins, “A comparison between computer vision- and deep learning-based models for automated concrete crack detection,” Proc Inst Mech Eng O J Risk Reliab, p. 1748006X2211409, Dec. 2022, doi: 10.1177/1748006X221140966.
F.-J. Yang, “An Extended Idea about Decision Trees,” in 2019 International Conference on Computational Science and Computational Intelligence (CSCI), IEEE, Dec. 2019, pp. 349–354. doi: 10.1109/CSCI49370.2019.00068.
T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning. in Springer Series in Statistics. New York, NY: Springer New York, 2009. doi: 10.1007/978-0-387-84858-7.
F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” Journal of Machine Learning Research, vol. 12, no. 85, pp. 2825–2830, 2011, doi: 10.1007/s13398-014-0173-7.2.
S. Zhang, D. Cheng, Z. Deng, M. Zong, and X. Deng, “A novel k NN algorithm with data-driven k parameter computation,” Pattern Recognit Lett, vol. 109, pp. 44–54, Jul. 2018, doi: 10.1016/j.patrec.2017.09.036.
R. Rojas, “The Backpropagation Algorithm,” in Neural Networks, Berlin, Heidelberg: Springer Berlin Heidelberg, 1996, pp. 149–182. doi: 10.1007/978-3-642-61068-4_7.
C. M. Bishop, Pattern Recognition and Machine Learning. New York: Springer, 2006.
A. J. Smola and B. Schölkopf, “A tutorial on support vector regression,” Stat Comput, vol. 14, no. 3, pp. 199–222, Aug. 2004, doi: 10.1023/B:STCO.0000035301.49549.88.
V. Kecman, “Support Vector Machines – An Introduction,” in Support Vector Machines: Theory and Applications, 2005, pp. 1–47. doi: 10.1007/10984697_1.
M. J. L. Orr, “Introduction to Radial Basis Function Networks,” 1996.
Y. Zhou, M. Huang, and M. Pecht, “Remaining useful life estimation of lithium-ion cells based on k-nearest neighbor regression with differential evolution optimization,” J Clean Prod, vol. 249, p. 119409, Mar. 2020, doi: 10.1016/j.jclepro.2019.119409.
M. Khazaee, A. Banakar, B. Ghobadian, M. A. Mirsalim, and S. Minaei, “Remaining useful life (RUL) prediction of internal combustion engine timing belt based on vibration signals and artificial neural network,” Neural Comput Appl, vol. 33, no. 13, pp. 7785–7801, Jul. 2021, doi: 10.1007/s00521-020-05520-3.
P. Kundu, A. K. Darpe, and M. S. Kulkarni, “An ensemble decision tree methodology for remaining useful life prediction of spur gears under natural pitting progression,” Struct Health Monit, vol. 19, no. 3, pp. 854–872, May 2020, doi: 10.1177/1475921719865718.
Z. Kang, C. Catal, and B. Tekinerdogan, “Remaining Useful Life (RUL) Prediction of Equipment in Production Lines Using Artificial Neural Networks,” Sensors, vol. 21, no. 3, p. 932, Jan. 2021, doi: 10.3390/s21030932.
Z. Chen, S. Cao, and Z. Mao, “Remaining Useful Life Estimation of Aircraft Engines Using a Modified Similarity and Supporting Vector Machine (SVM) Approach,” Energies (Basel), vol. 11, no. 1, p. 28, Dec. 2017, doi: 10.3390/en11010028.
M. Yan, X. Wang, B. Wang, M. Chang, and I. Muhammad, “Bearing remaining useful life prediction using support vector machine and hybrid degradation tracking model,” ISA Trans, vol. 98, pp. 471–482, Mar. 2020, doi: 10.1016/j.isatra.2019.08.058.
Y. Zhou and M. Huang, “Lithium-ion batteries remaining useful life prediction based on a mixture of empirical mode decomposition and ARIMA model,” Microelectronics Reliability, vol. 65, pp. 265–273, Oct. 2016, doi: 10.1016/j.microrel.2016.07.151.
J. Ben Ali, N. Fnaiech, L. Saidi, B. Chebel-Morello, and F. Fnaiech, “Application of empirical mode decomposition and artificial neural network for automatic bearing fault diagnosis based on vibration signals,” Applied Acoustics, vol. 89, pp. 16–27, Mar. 2015, doi: 10.1016/j.apacoust.2014.08.016.
H. Ascher and H. Feingold, Repairable Systems Reliability: Modeling, Inference, Misconceptions and Their Causes. New York: CRC Press/Marcel Dekker, 1984.
G. R. Weckman, R. L. Shell, and J. H. Marvel, “Modeling the reliability of repairable systems in the aviation industry,” Comput Ind Eng, vol. 40, no. 1–2, pp. 51–63, 2001, doi: 10.1016/S0360-8352(00)00063-2.
Q. Fan and H. Fan, “Reliability Analysis and Failure Prediction of Construction Equipment with Time Series Models,” Journal of Advanced Management Science, vol. 3, no. 3, pp. 203–210, 2015, doi: 10.12720/joams.3.3.203-210.
A. Quinn, V. Lopes-dos-Santos, D. Dupret, A. Nobre, and M. Woolrich, “EMD: Empirical Mode Decomposition and Hilbert-Huang Spectral Analyses in Python,” J Open Source Softw, vol. 6, no. 59, p. 2977, Mar. 2021, doi: 10.21105/joss.02977.
M. M. Aquino, C. B. S. Maior, N. A. E. Lins, C. C. B. O. Mota, P. L. A. Nascimento, and A. S. L. Gomes, “Using optical coherence tomography images to evaluate fungal growth in reline resins,” J Innov Opt Health Sci, Jan. 2023, doi: 10.1142/S1793545822500377.
I. Alsyouf, M. Shamsuzzaman, G. Abdelrahman, and M. Al Taha, “Improving reliability of repairable systems using preventive maintenance and time-between-failures monitoring,” European J. of Industrial Engineering, vol. 10, no. 5, p. 596, 2016, doi: 10.1504/EJIE.2016.078798.
C. B. S. Maior, M. J. das C. Moura, J. M. M. Santana, and I. D. Lins, “Real-time classification for autonomous drowsiness detection using eye aspect ratio,” Expert Syst Appl, vol. 158, p. 113505, Nov. 2020, doi: 10.1016/j.eswa.2020.113505.
