Abstract: A bending-torsion coupled nonlinear dynamic model is established, including complicating factors, such as time-varying mesh stiffness, mesh damping, tooth error, tooth friction, backlash and bearing clearance of face-gear drive system. Fourth-fifth order Runge-Kutta method with variable time step is introduced to solve the dimensionless dynamic differential equations. The results of simulation indicate that single period non-harmonic, multi-period sub-harmonic, quasi-periodic and chaotic responses appear with jumping phenomenon in different rotating speed; the response trend of system with the increasing load torque is from chaos, through multi-period sub-harmonic, to single period non-harmonic; single-sided and double-sided impact between gear pairs occur easily with light load, and non-impact occur when load increases enough; the nonlinear vibration characteristic of system changes slightly in low friction coefficient, and it becomes chaotic through a sequence of period doubling bifurcation with the increase of friction coefficient.