Abstract:A bending-torsion-axial coupled nonlinear dynamic model of spiral bevel, parallel-axes and planetary gear transmission system with 18 degrees of freedom is established, including the factors of time-varying mesh stiffness, mesh damping, transmission error and backlash. Fourth-fifth order Runge-Kutta method with variable time step is introduced to solve the dimensionless dynamic differential equations, and the coupled nonlinear vibration characteristic of gear system is studied. The results of simulation indicate that the response of system enters into chaos through period-doubling bifurcation along with the increase of backlash, and that the affection of backlash to dynamic characteristic is getting smaller with the increase of load; with the increasing of load, the response of system turns into single period from chaos through inverse period-doubling bifurcation, and the contact state tends to single-sided impact from double-sided, and finally turns to non-impact; the chaos region of displacement-load bifurcation diagram becomes smaller when the rotate speed decreases.