基于强化学习的随机振动主动控制策略

摘要
图/表
参考文献(30)
相关文章 (15)

全文: PDF (1105 KB) HTML (1 KB)
输出: BibTeX | EndNote (RIS)

摘要

针对被控系统的不确定性和非线性特征，提出了一种基于强化学习的随机振动主动控制策略。采用深度确定性策略梯度（DDPG）的强化学习算法设计振动控制器，该过程不涉及专家经验，完全由算法和数据自主交互学习完成。控制器是一个多层神经网络模型，这种由强化学习算法设计的控制器称为RL-NN控制器。通过两个数值仿真算例验证了RL-NN控制器的性能，其中考虑不确定性的单自由度系统主动控制效果达97%，考虑不确定性和非线性的车辆1/4悬架系统半主动控制效果达74%，结果表明RL-NN控制器对不确定性和非线性系统具有优异的振动控制能力。强化学习算法仅仅花费数小时设计的随机振动主动控制策略便优于专家经验数年来设计的控制策略，这为复杂系统振动主动/半主动控制器的设计提供了一种新的实现途径。

	服务

	把本文推荐给朋友
	加入我的书架
	加入引用管理器
	E-mail Alert
	RSS
	作者相关文章
	周嘉明1
	董龙雷1
	孟超2
	孙海亮2

关键词 ：强化学习, 神经网络, 不确定性, 非线性, 振动主动控制

Abstract：Concerning the uncertainty and nonlinearity of a controlled system, an active control strategy for random vibration based on reinforcement learning was proposed.The vibration controller was designed by a reinforcement learning algorithm-deep deterministic policy gradient (DDPG).This process does not involve expert experience and was entirely completed by the autonomous interactive learning of DDPG algorithms and data.The controller is a multi-layer neural network model, and this kind of controller designed by reinforcement learning algorithm is called neural network controller designed by reinforcement learning (RL-NN)controller.The performance of the RL-NN controller was verified through two numerical simulation examples: the active control effect of a single degree of freedom system with uncertainty reaches 97%.The semi-active control effect of the 1/4 vehicle suspension system with uncertainty and nonlinearity reaches 74%.The results show that the RL-NN controller has excellent vibration control capabilities for systems with uncertain and nonlinear.The random vibration active control strategy designed by the reinforcement learning algorithm in only a few hours is better than the control strategy designed by experts over few years.This provides a new approach to design active/semi-active controllers for complex systems.

Key words： reinforcement learning neural network uncertainty nonlinearity active vibration control

收稿日期: 2020-12-21 出版日期: 2021-08-28

引用本文:

周嘉明1，董龙雷1，孟超2，孙海亮2. 基于强化学习的随机振动主动控制策略[J]. 振动与冲击, 2021, 40(16): 281-286.
ZHOU Jiaming1, DONG Longlei1, MENG Chao2, SUN Hailiang2. A active vibration control strategy based on reinforcement learning. JOURNAL OF VIBRATION AND SHOCK, 2021, 40(16): 281-286.

链接本文:

http://jvs.sjtu.edu.cn/CN/ 或 http://jvs.sjtu.edu.cn/CN/Y2021/V40/I16/281

［1］MAHMOODI S N, AHMADIAN M.Modified acceleration feedback for active vibration control of aerospace structures［J］.Smart Materials and Structures,2010,19(6):065015.
［2］SUN W C, PAN H H, GAO H J.Filter-based adaptive vibration control for active vehicle suspensions with electrohydraulic actuators［J］.IEEE Transactions on Vehicular Technology, 2015, 65(6): 4619-4626.
［3］AMEZQUITA-SANCHEZ J P, DOMINGUEZ-GONZALEZ A, SEDAGHATI R, et al.Vibration control on smart civil structures: a review［J］.Mechanics of Advanced Materials and Structures, 2014, 21(1): 23-38.
［4］THENOZHI S, YU W.Advances in modeling and vibration control of building structures［J］.Annual Reviews in Control, 2013, 37(2): 346-364.
［5］WANG C X, XIE X L, CHEN Y H, et al.Investigation on active vibration isolation of a Stewart platform with piezoelectric actuators［J］.Journal of Sound and Vibration, 2016, 383: 1-19.
［6］PAN Y P, GUO Z, LI X, et al.Output-feedback adaptive neural control of a compliant differential SMA actuator［J］.IEEE Transactions on Control Systems Technology, 2017, 25(6): 2202-2210.
［7］GU X Y, YU Y, LI Y C, et al.Experimental study of semi-active magnetorheological elastomer base isolation system using optimal neuro fuzzy logic control［J］.Mechanical Systems and Signal Processing, 2019, 119: 380-398.
［8］WEBER F.Semi-active vibration absorber based on real-time controlled MR damper［J］.Mechanical Systems and Signal Processing, 2014, 46(2): 272-288.
［9］DANGOR M, DAHUNSI O A, PEDRO J O, et al.Evolutionary algorithm-based PID controller tuning for nonlinear quarter-car electrohydraulic vehicle suspensions［J］.Nonlinear Dynamics, 2014, 78(4): 2795-2810.
［10］JOVANOVIC＇ M M, SIMONOVIC＇ A M, ZORIC＇ N D, et al.Experimental studies on active vibration control of a smart composite beam using a PID controller［J］.Smart Materials and Structures, 2013, 22(11): 115038.
［11］ZHANG S Q, SCHMIDT R, QIN X S.Active vibration control of piezoelectric bonded smart structures using PID algorithm［J］.Chinese Journal of Aeronautics, 2015,28(1):305-313.
［12］SCHULZ S L, GOMES H M, AWRUCH A M.Optimal discrete piezoelectric patch allocation on composite structures for vibration control based on GA and modal LQR［J］.Computers & Structures, 2013, 128: 101-115.
［13］MIYAMOTO K, SATO D, SHE J.A new performance index of LQR for combination of passive base isolation and active structural control［J］.Engineering Structures, 2018, 157: 280-299.
［14］HO C C, MA C K.Active vibration control of structural systems by a combination of the linear quadratic Gaussian and input estimation approaches［J］.Journal of Sound and Vibration, 2007, 301(3/4/5): 429-449.
［15］GAN Z, HILLIS A J, DARLING J.Adaptive control of an active seat for occupant vibration reduction［J］.Journal of Sound and Vibration, 2015, 349: 39-55.
［16］HU Q, SU L, CAO Y G, et al.Decentralized simple adaptive control for large space structures［J］.Journal of Sound and Vibration, 2018, 427: 95-119.
［17］GEORGIOU G, FOUTSITZI G A, STAVROULAKIS G E.Nonlinear discrete-time multirate adaptive control of non-linear vibrations of smart beams［J］.Journal of Sound and Vibration, 2018, 423: 484-519.
［18］卜锋斌, 蒋爱华.自适应控制算法在振动主动控制中的应用［J］.噪声与振动控制，2014，34(2)：46-49.
BU Fengbin, JIANG Aihua.Application of an adaptive algorithm to active vibration control［J］.Noise and Vibration Control, 2014, 34(2): 46-49.
［19］FU J, BAI J F, LAI J J, et al.Adaptive fuzzy control of a magnetorheological elastomer vibration isolation system with time-varying sinusoidal excitations［J］.Journal of Sound and Vibration, 2019, 456: 386-406.
［20］TAIRIDIS G, FOUTSITZI G, KOUTSIANITIS P, et al.Fine tuning of a fuzzy controller for vibration suppression of smart plates using genetic algorithms［J］.Advances in Engineering Software, 2016, 101: 123-135.
［21］MARINAKI M, MARINAKIS Y, STAVROULAKIS G E.Fuzzy control optimized by a multi-objective differential evolution algorithm for vibration suppression of smart structures［J］.Computers & Structures,2015,147:126-137.
［22］SUN C Y, HE W, HONG J.Neural network control of a flexible robotic manipulator using the lumped spring-mass model［J］.IEEE Transactions on Systems, Man and Cybernetics: Systems, 2016, 47(8): 1863-1874.
［23］ABDELJABER O, AVCI O, INMAN D J.Active vibration control of flexible cantilever plates using piezoelectric materials and artificial neural networks［J］.Journal of Sound and Vibration, 2016, 363: 33-53.
［24］WANG N M, ADELI H.Self-constructing wavelet neural network algorithm for nonlinear control of large structures［J］.Engineering Applications of Artificial Intelligence, 2015, 41: 249-258.
［25］TAGHIZADEH M, YARMOHAMMADI M J.Development of a self-tuning PID controller on hydraulically actuated stewart platform stabilizer with base excitation［J］.International Journal of Control, Automation and Systems, 2018, 16(6): 2990-2999.
［26］MNIH V, KAVUKCUOGLU K, SILVER D, et al.Human-level control through deep reinforcement learning［J］.Nature, 2015, 518(7540): 529-533.
［27］MNIH V, BADIA A P, MIRZA M, et al.Asynchronous methods for deep reinforcement learning［C］∥International Conference on Machine Learning.NYC,USA: PMLR, 2016.
［28］IOFFE S, SZEGEDY C.Batch normalization:accelerating deep network training by reducing internal covariate shift［C］∥International Conference on Machine Learning.Lille: PMLR, 2015.
［29］WU J, ZHOU H L, LIU Z Y, et al.A load-dependent PWA-H∞ controller for semi-active suspensions to exploit the performance of MR dampers［J］.Mechanical Systems and Signal Processing, 2019, 127: 441-462.
［30］YAO G Z, YAP F F, CHEN G, et al.MR damper and its application for semi-active control of vehicle suspension system［J］.Mechatronics, 2002, 12(7): 963-973.