Meshing force model of misaligned spline coupling is derived, dynamic equation of rotor system considering the influence of misaligned spline coupling was established based on finite element analysis. The influence rules of meshing force on rotor-spline coupling system are simulated with numerical integral method. Theory analysis and simulation results show: meshing force of spline coupling is related to structure parameters of coupling, transfer torque, static misalignment, dynamic vibration displacement, and so on; stiffness of coupling which is not a constant, relates to dynamic vibration displacement and depends on static misalignment. Dynamic behavior of rotor-spline coupling system reveals that 1X -rotating speed is the main response of system when there is no misalignment while 1X -rotating speed appears with some misalignment; moreover, when misalignment increase, shaft orbit deviates origin and magnitudes of all frequency come larger, the vibration of system gets complex. This research results can provide important criterions for trouble shooting of rotor systems connected by spline coupling.