The dynamic equation of a nonlinear relative rotation system with a triple-well Mathieu-Duffing oscillator is investigated. Firstly, a codimension three-bifurcation characteristic is deduced by combining with the multi-scale method and singularity theory under the condition of nonautonomy. Secondly, the threshold value of chaos about Smale horseshoe commutation is given from Melnikov method. Finally, the numerical simulation exhibits safe basins and chaos, and the erosion process of safe basins, which is closely related to the process, leading to chaos.