Focusing on the problem of carbon nanotube structure optimization by molecule dynamics simulation, a novel algorithm is proposed which combines the genetic algorithm with simulated annealing and the clone select algorithm.Test results of five typical functions show that this algorithm has high stability and gives good global optimization. Applied to structure optimization of carbon nanotubes, it can accelerate the process of energy optimization and improve the quality of structure optimization.The simulation results show that the optimizing time increases linearly with the number of atoms.The time of structure optimization is reduced one order of maguitucle compared with the conjugate gradient methods.