Multi-label industrial fault diagnosis method based on the Transformer network model
HUO Jiuyuan1,2,3,LI Chaojie1,YU Chunxiao1
1.School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China;
2.National Cryosphere Desert Data Center (NCDC), Lanzhou 730000, China;
3.Lanzhou Ruizhiyuan Information Technology Co., Ltd., Lanzhou 730070, China
Abstract:The multi-dimensionality, class imbalance and concurrency of industrial fault data bring three major challenges to industrial fault diagnosis: first, the extraction of fault features from multi-dimensional sensor data is excessively dependent on expert knowledge; second, the extreme class imbalance between different types of fault samples seriously limits the performance of the classifier; third, multiple types of faults may occur at the same time to increase the complexity of fault diagnosis. In order to meet these challenges, a multi-label fault diagnosis model based on the improved Transformer of multiple self-attention is proposed. Combined with adaptive synthetic sampling (ADASYN) and Borderline-SMOTE1 oversampling methods, making full use of the advantages of Transformer encoder-decoder structure and attention mechanism, features can be automatically extracted from multi-dimensional sensor data, the complex mapping relationship between multi-dimensional sensor data and multiple fault labels can be fully mined. The results show that this method can still diagnose multiple faults occurred at the same time in the extreme imbalanced industrial fault data for the PHM2015 Plant data set.
火久元1,2,3,李超杰1,于春潇1. 基于Transformer的多标签工业故障诊断方法研究[J]. 振动与冲击, 2023, 42(18): 88-99.
HUO Jiuyuan1,2,3,LI Chaojie1,YU Chunxiao1. Multi-label industrial fault diagnosis method based on the Transformer network model. JOURNAL OF VIBRATION AND SHOCK, 2023, 42(18): 88-99.
[1] 金晓航,王宇,ZHANG Bin. 工业大数据驱动的故障预测与健康管理[J]. 计算机集成制造系统,2022,28(05):1314-1336.
JIN Xiaohang, WANG Yu, ZHANG Bin. Industrial big data-driven fault prognostics and health management[J]. Computer Integrated Manufacturing Systems, 2022, 28(05): 1314-1336.
[2] Atoui M A, Cohen A. Coupling data-driven and model-based methods to improve fault diagnosis[J]. Computers in Industry, 2021, 128(4): 103401.
[3] 吴定海,任国全,王怀光,等. 基于卷积神经网络的机械故障诊断方法综述[J]. 机械强度,2020,42(05):1024-1032.
WU Dinghai, REN Guoquan, WANG Huaiguang, et al. The review of mechanical fault diagnosis methods based on convolutional neural network[J]. Journal of Mechanical Strength, 2020, 42(05): 1024-1032.
[4] Xiao W. A probabilistic machine learning approach to detect industrial plant faults[J]. International Journal of Prognostics and Health Management, 2016, 7(1).
[5] 朱丹宸,张永祥,潘洋洋,等. 基于多传感器信号和卷积神经网络的滚动轴承故障诊断[J]. 振动与冲击,2020,39(04):172-178.
ZHU Danchen, ZHANG Yongxiang, PAN Yangyang, et al. Fault diagnosis for rolling element bearings based on multi-sensor signals and CNN[J]. Journal of Vibration and Shock, 2020, 39(04): 172-178.
[6] 赵志宏,赵敬娇,魏子洋. 基于BiLSTM的滚动轴承故障诊断研究[J]. 振动与冲击,2021,40(01):95-101.
ZHAO Zhihong, ZHAO Jingjiao, WEI Ziyang. Rolling bearing fault diagnosis based on BiLSM network[J]. Journal of Vibration and Shock, 2021, 40(01): 95-101.
[7] 张立鹏,毕凤荣,程建刚,等. 基于注意力BiGRU的机械故障诊断方法研究[J]. 振动与冲击,2021,40(05):113-118.
ZHANG Lipeng, BI Fengrong, CHENG Jiangang, et al. Mechanical fault diagnosis method based on attention BiGRU[J]. Journal of Vibration and Shock, 2021, 40(05): 113-118.
[8] 张龙,甄灿壮,易剑昱,等. 双通道特征融合CNN-GRU齿轮箱故障诊断[J]. 振动与冲击,2021,40(19):239-245+294.
ZHANG Long, ZHEN Canzhuang, YI Jianyu, et al. Dual-channel feature fusion CNN-GRU gearbox fault diagnosis[J]. Journal of Vibration and Shock, 2021, 40(19): 239-245+294.
[9] Liu T, Zhu X, Pedrycz W, et al. A design of information granule-based under-sampling method in imbalanced data classification[J]. Soft Computing, 2020, 24(22): 17333-17347.
[10] Mohammed R, Rawashdeh J, Abdullah M. Machine learning with oversampling and undersampling techniques: overview study and experimental results[C]//2020 11th International Conference on Information and Communication Systems. Irbid: IEEE, 2020.
[11] Li J, Zhu Q, Wu Q, et al. A novel oversampling technique for class-imbalanced learning based on SMOTE and natural neighbors[J]. Information Sciences, 2021, 565: 438-455.
[12] Alotaibi R, Flach P. Multi-label thresholding for cost-sensitive classification[J]. Neurocomputing, 2021, 436: 232-247.
[13] 雷亚国,贾峰,孔德同,等. 大数据下机械智能故障诊断的机遇与挑战[J]. 机械工程学报,2018,54(05):94-104.
LEI Yaguo, JIA Feng, KONG Detong, et al. Opportunities and challenges of machinery intelligent fault diagnosis in big data era[J]. Journal of Mechanical Engineering, 2018, 54(05): 94-104.
[14] Zhang W, Li X, Jia X D, et al. Machinery fault diagnosis with imbalanced data using deep generative adversarial networks[J]. Measurement, 2020, 152: 107377.
[15] Elreedy D, Atiya A F. A comprehensive analysis of synthetic minority oversampling technique (SMOTE) for handling class imbalance[J]. Information Sciences, 2019, 505: 32-64.
[16] Tay Y, Dehghani M, Bahri D, et al. Efficient transformers: A survey[J]. ACM Computing Surveys (CSUR), 2020. https://www.xueshufan.com/publication/3085139254.
[17] 李贞莹. 基于多标签深度学习的阵列故障诊断方法研[D]. 成都:电子科技大学,2021.
[18] Tarekegn A N, Giacobini M, Michalak K. A review of methods for imbalanced multi-label classification[J]. Pattern Recognition, 2021, 118: 107965.
[19] Li L, Jamieson K, DeSalvo G, et al. Hyperband: A novel bandit-based approach to hyperparameter optimization[J]. The Journal of Machine Learning Research, 2017, 18(1): 6765-6816.
[20] Jamieson K, Talwalkar A. Non-stochastic best arm identification and hyperparameter optimization[C]// Proceedings of the 19th International Conference on Artificial Intelligence and Statistics. Cadiz: PMLR, 2016.
[21] 程建刚,毕凤荣,张立鹏,等. 基于多重注意力-卷积神经网络-双向门控循环单元的机械故障诊断方法研究[J]. 内燃机工程,2021,42(04):77-83+92.
CHENG Jiangang, BI Fengrong, ZHANG Lipeng, et al. Research on mechanical fault diagnosis method based on multiple attention-convolutional neural networks-bidirectional gated recurrent unit[J]. Chinese Internal Combustion Engine, 2021, 42(04): 77-83+92.
[22] Laurens V D M, Hinton G. Visualizing data using t-SNE[J]. Journal of Machine Learning Research, 2008, 9(2605): 2579-2605.