Remaining useful life prediction based on BiLSTM and attention mechanism
ZHAO Zhihong1,2,LI Qing1,YANG Shaopu2,LI Lehao1
1.School of Information Science and Technology, Shijiazhuang Tiedao University, Shijiazhuang 050043, China;
2.State Key Laboratory of Structural Mechanics Behavior and System Safety of Traffic Engineering, Shijiazhuang Tiedao University, Shijiazhuang 050043, China
摘要剩余寿命(remaining useful life,RUL)预测在现代工业中占有重要地位,如何提高剩余寿命预测的准确性已经成为当今研究的热点。传统的剩余寿命预测方式是采用基于模型的方法进行预测,需要人工提取特征,不能自动地学习特征信息,无法获得原始数据与剩余使用寿命之间的复杂映射关系。本文提出一种基于双向长短期记忆网络(bi-directional long short term memory ,BiLSTM)与注意力机制的剩余寿命预测模型,与已有的剩余寿命预测方法不同之处在于:直接将获取的原始时间序列输入到BiLSTM神经网络中,通过BiLSTM自动地提取设备状态特征信息;然后利用注意力机制对特征分配不同的权重,这样可以更准确地提取设备的健康状态信息。进行了发动机和轴承剩余寿命预测实验,并与长短期记忆网络(long short-term memory,LSTM)模型和BiLSTM剩余寿命预测模型进行比较,实验结果表明提出的BiLSTM与注意力机制相结合的模型能够更准确地进行剩余寿命预测,具有一定的应用价值。
Abstract:Remaining useful life (RUL) prediction occupies an important position in modern industry, and in recent years improving the accuracy of RUL prediction has become a widely researched topic. The traditional RUL prediction methods generally use model-based prediction methods wherein features are manually extracted. Furthermore, these traditional methods cannot learn feature information automatically, and obtains the complex mapping relationship between the original data and the RUL. In this study, a RUL prediction model based on bidirectional long short term memory (BiLSTM) and an attention mechanism is proposed. In contrast to traditional prediction models, the proposed model directly inputs the obtained original time series into the BiLSTM neural network, and the device status is automatically extracted. Then, the attention mechanism is used to assign different weights to the features, so that the health status information of a device or machine can be extracted more accurately. We carry out various RUL prediction experiments on engines and bearings, and compare the results with the long short-term memory(LSTM) and BiLSTM RUL prediction models. The experimental results show that the proposed BiLSTM integrated with an attention mechanism can predict the RUL more accurately and can be used in real-world settings.
赵志宏1,2,李晴1,杨绍普2,李乐豪1. 基于BiLSTM与注意力机制的剩余寿命预测研究[J]. 振动与冲击, 2022, 41(6): 44-50.
ZHAO Zhihong1,2,LI Qing1,YANG Shaopu2,LI Lehao1. Remaining useful life prediction based on BiLSTM and attention mechanism. JOURNAL OF VIBRATION AND SHOCK, 2022, 41(6): 44-50.
[1] A. Elsheikh, S. Yacout, M.S. Ouali, Bidirectional handshaking LSTM for remaining useful life prediction, Neurocomputing 323 (2019) 148–156.
[2] Y. Peng, Y. Wang, Y. Zi, Switching state-space degradation model with recursive filter/smoother for prognostics of remaining useful life, IEEE Transactions on Industrial Informatics,15 (2) (2018) 822–832.
[3] J. Zhu, N. Chen, W. Peng, Estimation of bearing remaining useful life based on multiscale convolutional neural network, IEEE Transactions on Industrial Electronics,66(4) (2019) 3208–3216.
[4] Yuanhang Chen,Gaoliang Peng,Zhiyu Zhu,Sijue Li. A novel deep learning method based on attention mechanism for bearing remaining useful life prediction[J]. Applied Soft Computing Journal,2020,86.
[5] 李少鹏. 结合CNN和LSTM的滚动轴承剩余寿命预测方法研究[D].哈尔滨理工大学,2019.
Li Shaopeng. Research on Rolling Bearing Remaining Service Life Prediction Method Combining CNN and LSTM[D]. Harbin University of Science and Technology, 2019.
[6] GEBRAEEL N,LAWLEY M,LIU R,et al. Residual life
predictions from vibration-based degradation signals:A neural network approach[J].IEEE Transactions on Industrial Electronics,2004,51(3):694-700.
[7] 王慧,井伟川,赵国超,金鑫.基于灰色系统模型GM(1,1)改进Miner准则的液压支架底座疲劳寿命预测[J].上海交通大学学报,2020,54(01):106-110.
Wang Hui, Jing Weichuan, Zhao Guochao, Jin Xin. Fatigue Life Prediction of Hydraulic Support Base Based on Grey System Model GM(1,1) Improved Miner Criterion[J].Journal of Shanghai Jiaotong University,2020,54(01):106-110 .
[8] 韦国歆,李志鹏,陆涛,席伟韬,袁瑶.基于马尔可夫模型的蓄电池寿命预测方法研究[J].自动化与仪器仪表,2019(11):44-47.
Wei Guoxin, Li Zhipeng, Lu Tao, Xi Weitao, Yuan Yao. Research on battery life prediction method based on Markov model[J].Automation and Instrumentation,2019(11):44-47.
[9] 徐国平. 基于支持向量机的动调陀螺仪寿命预测方法研究[D].上海交通大学,2008.
Xu Guoping. Research on life prediction method of dynamic gyro based on support vector machine[D]. Shanghai Jiaotong University, 2008.
[10] Giduthuri Sateesh Babu,Peilin Zhao,Xiao-Li Li. Deep Convolutional Neural Network Based Regression Approach for Estimation of Remaining Useful Life . Springer , 2016.
[11] 杨宇,张娜,程军圣.全参数动态学习深度信念网络在滚动轴承寿命预测中的应用[J].振动与冲击,2019,38(10):199-205+249.
Yang Yu,Zhang Na,Cheng Junsheng.The application of full parameter dynamic learning deep belief network in rolling bearing life prediction[J].Vibration and Shock,2019,38(10):199-205+249.
[12] 唐旭,徐卫晓,谭继文,王彦松.基于LSTM的滚动轴承剩余寿命预测[J].机械设计,2019,36(S1):117-119.
Tang Xu, Xu Weixiao, Tan Jiwen, Wang Yansong. Prediction of remaining service life of rolling bearings based on LSTM[J]. Mechanical Design, 2019, 36(S1): 117-119.
[13] Ren,L., Sun, Y., Cui, J., & Zhang, L. Bearing remaining useful life prediction based on deep auto encoder and deep neural networks.Journal of Manufacturing Systems.
[14] Mnih,Volodymyr,Heess H,GRAVES A,et al. Recurrent Models of Visual Attention[J].arXiv preprint arXiv:1406-6247,2014.
[15]朱张莉,饶元,吴渊,祁江楠,张钰.注意力机制在深度学习中的研究进展[J].中文信息学报,2019,33(06):1-11.
Zhu Zhangli, Rao Yuan, Wu Yuan, Qi Jiangnan, Zhang Yu. Research progress of attention mechanism in deep learning[J].Journal of Chinese Information Processing,2019,33(06):1-11.
[16 ]葛阳,郭兰中,牛曙光,窦岩.基于t-SNE和LSTM的旋转机械剩余寿命预测[J].振动与冲击,2020,39(07):223-231+273.
Ge Yang, Guo Lanzhong, Niu Shuguang, Dou Yan. Remaining life prediction of rotating machinery based on t-SNE and LSTM [J]. Vibration and Shock, 2020, 39(07): 223-231+273.
[17] 韩林洁,石春鹏,张建超.基于BiLSTM的滚动轴承剩余寿命预测[J].制造业自动化,2020,42(05):47-50.
Han Linjie, Shi Chunpeng, Zhang Jianchao. BiLSTM-based rolling bearing remaining life prediction [J]. Manufacturing Automation, 2020, 42(05): 47-50.
[18] 汪琪,段湘煜.基于注意力卷积的神经机器翻译[J].计算机科学,2018,45(11):226-230.
Wang Qi, Duan Xiangyu. Neural machine translation based on attention convolution[J]. Computer Science, 2018, 45(11): 226-230.
[19]石磊,王毅,成颖,魏瑞斌.自然语言处理中的注意力机制研究综述[J].数据分析与知识发现,2020,4(05):1-14.
Shi Lei, Wang Yi, Cheng Ying, Wei Ruibin. A review of attention mechanism research in natural language processing [J]. Data Analysis and Knowledge Discovery, 2020, 4(05): 1-14.
[20] Structured attention networks. Kim Y,Denton C,Hoang L,Alexander M Rush. Proceedings of International Conference on Learning Representations . 2017
[21] Saxena, A., Goebel, K.: PHM08 challenge data set, NASA AMES prognostics data repository. Tech. rep., Moffett Field, CA (2008)
[22] Saxena, A., Goebel, K., Simon, D., Eklund, N.: Damage propagation modeling for aircraft engine run-to-failure simulation. In: International Conference on Prognostics and Health Management, 2008. PHM 2008. pp. 1–9 (Oct 2008)
[23] Wang, P ., Y oun, B.D., Hu, C.: A generic probabilistic framework for structural health prognostics and uncertainty management. Mechanical Systems and Signal Processing 28, 622–637 (2012)
[24] 范永全,杜亚军.基于加权相似度的用户协同过滤方法[J].计算机工程与应用,2016,52(22):222-225+259.
Fan Yongquan, Du Yajun. User collaborative filtering method based on weighted similarity[J]. Computer Engineering and Applications, 2016, 52(22): 222-225+259.
[25] G. Babu, P. Zhao, X. Li, Deep convolutional neural network based regression approach for estimation of remaining useful life, Proceedings of the International Conference on Database Systems for Advanced Applications, vol. 9642 (2016) 214–228.
[26] C. Zhang, P. Lim, A. Qin, K. Tan, Multiobjective deep belief networks ensemble for remaining useful life estimation in prognostics, IEEE Transactions on Neural Networks and Learning Systems ,28 (10) (2017) 2306–2318.
[27] Nectoux P, Gouriveau R, Medjaher K, Ramasso E, Morello B, Zerhouni N, et al. PRONOSTIA: an experimental platform for bearings accelerated degradation tests. In: Proceedings of the Conference on Prognostics and Health Management. Denver, Colorado, USA: IEEE, 2012. 1–8.
[28] 康守强,周月,王玉静,谢金宝,V.I.MIKULOVICH.基于改进SAE和双向LSTM的滚动轴承RUL预测方法[J/OL].自动化学报:1-11[2020-10-27].
Kang Shouqiang, Zhou Yue, Wang Yujing, Xie Jinbao, V.I.MIKULOVICH. Rolling bearing RUL prediction method based on improved SAE and bidirectional LSTM[J/OL].Acta Automatica Sinica: 1-11[2020-10-27].