Combining the features of Frequency Slice Wavelet Transform (FSWT) and denoising source separation (DSS), we propose an underdetermined DSS method based on FSWT. This method is used to deal with the blind source separation (BSS) problem of rotating machinery in the case of the number of observed mixtures being less than that of contributing sources. New mixed signals can be reconstructed through the FSWT method, which is effectively way to solve the problem of the dimension insufficient in underdetermined blind source separation. As the same time, the method can solve the problem of time-frequency analysis in FSWT. Applying FSWT-DSS method to the rotor fault detection, we have diagnosed the sudden unbalance phenomenon through the measured fault signals of the rotor. The simulation and experimental results, and their analysis show preliminarily that the FSWT-DSS method is indeed efficient in analyzing the fault diagnosis and it has an important engineering significance for fault detection of rotating machines.