To extract the sensitive damage features directly from the structural vibration responses, the singular value decomposition and the proper orthogonal decomposition are applied to decompose the structural vibration response matrix into a set of proper orthogonal mode, which can represent the structure natural properties. The singular value decomposition is firstly applied to the structural vibration power spectral matrix to figure out the modal frequencies, then the correlation matrix can be calculated at each modal frequency. The proper orthogonal decomposition method is applied on the correlation matrix, so that the proper orthogonal mode can be got, and the proper orthogonal mode converges to the normal structural mode, so the proper orthogonal modes can be used to construct a damage locating vector, and the damage can be located through the different stress distribution of each elements. The experimental results show that the proposed method can detect and locate the damage effectively.