An adaptive generalized S transform (AGST) is introduced in this paper. The simulation shows that the time-frequency resolution obtained by AGST is better than S transform and STFT. Then, the proposed method is introduced to analyze the vibration characteristics of the cylinder head of a four-cylinder IC engine. The result indicates that time-frequency representation obtained by AGST method has a better time-frequency resolution than S transform, and could exactly reveal the vibration characteristics of the internal combustion (IC) engine cylinder head. The research achievements could provide guidance for the following research of the vibration control and structural optimization for this IC engine.