The effective transmittance of the Fabry-Perot (F-P) etalon in the case of the atmospheric backscatter light incidence is deduced. In the single F-P etalon-based dual-frequency Doppler lidar system, the wind retrieval accuracy with Rayleigh-induced effect for average method is analyzed quantitatively. A nonlinear iterative algorithm is proposed which can retrieve both wind speed and backscatter ratio, and its effectiveness is verified by simulation test. At the same time, the specific expressions of the radial wind speed error and backscatter ratio error are deduced. According to these expressions, the radial wind speed error and backscatter ratio error are simulated. The simulation results show that on the assumption that the total number of backscattering photons received by telescope is 50000, the radial wind speed measurement error decreases rapidly with the increase of backscatter ratio; when Rβ >1.2, the radial wind speed error is below 3 m/s within the wind speed measurement dynamic range of ±25 m/s; the backscatter ratio measurement error increases with the increase of backscatter ratio, but it is almost irrelative to the magnitude of radial wind; when Rβ<10, the backscatter ratio relative error is less than 13%.