An optimum installation method of viscous dampers in tall buildings is proposed based on relative fitness genetic algorithm, which considering different seismic waves and different optimal objectives. Compared with standard genetic algorithm (SGA), relative fitness genetic algorithm constructs fitness function based on the relative objective function value, which can fully embody the fitness difference in the population, and, promote effective evolution, obtain the global optimal solution as a result. Various coefficient combinations of the objective function were discussed. The objective function was normalized and different seismic waves were considered. The method is used to optimize damper installations in a 20 layers frame structure. The result demonstrated the efficiency and applicability of the method proposed in this paper.