Based on the data-driven stochastic subspace identification (SSI) algorithm, an improved on-line SSI procedure is presented to automatically extract the operational modal parameters of structures. The Hankel matrix’s dimension parameters i and j are determined by experiential approach and the Mode Transfer Norm (MTN) is used to improve the stabilization diagrams. A singular entropy-based approach is implemented to determine the system so that the identification procedure becomes more automatic. To reduce the calculation work, the Choleshy factorization technique is used to handle the lower tri-matrix R. As a result, the calculation efficiency is much improved when the data are overlapped for two different time windows. The applicability and efficiency of proposed method are validated via both simulated 3-span continuous beam and full-size bridge that was tested on field under operational conditions. It is demonstrated that the approach has an advantage of calculation time and precision. The proposed method can be used to perform online operation modal parameter identification of large-scale structures.