We proposed a new chaotic system using the closed loop self-feedback frequency modulation (FM) technique where the current sample of the output signal of voltage-controlled oscillator (VCO) is transformed and used as the next input of VCO. The dynamical behavior of the system is investigated by Bifurcation diagram and Lyapunov exponents. The performance of the system is analyzed in terms of peak to average power ratio (PAPR), frequency spectrum and the correlation function of the output signal. The results show that the developed system has complicated dynamical behaviors, including bifurcation, multi-stability and chaos, and the generated chaotic signals have low PAPR and wide bandwidth. We also discussed the synchronization of the system by FM code. In circuit experiments, VCO is implemented by digital frequency synthesis. The experimental results are consistent with our theoretical analysis.