Atomic magnetometers can achieve high magnetic-field sensitivity in the spin-exchange relaxationfree (SERF) regime, while multi-species media can provide good long-term zero-bias stability. Their underlying spin system is intrinsically nonlinear. In this work, we study the nonlinear dynamical behavior of a feedback-assisted atomic magnetometer under periodic modulation of the longitudinal bias magnetic field. In the parameter range considered here, the
87Rb electron-spin polarization remains close to a quasi-steady state, so the long-time dynamics are dominated by the
129Xe nuclear-spin polarization and are described by a simplified Bloch model, Eqs. (1) to (3). By numerically integrating the Bloch equations, we determine the stable dynamical states reached after long-time evolution. To characterize the nature of these states, we combine fast Fourier transform spectrum, Poincaré section and Lyapunov exponents. Our results show that, under periodic driving, the system exhibits three distinct states: quasi-periodic orbits, chaos and limit cycles. The quasi-periodic state is characterized by multiple incommensurate frequencies in the spectrum. The chaotic state shows a positive Lyapunov exponent and a continuous broadband spectrum. The limit cycle state corresponds to a stable closed periodic orbit in phase space. By scanning the modulation amplitude
δ and modulation frequency
ωacT2 (see Eq. (4)), we construct the dynamical phase diagram in the modulation-parameter space. In particular, for (
ω0T2,
χ/
χc) = (1.3, 8), the phase diagram, Fig. 3 (also see the left figure below), is mainly composed of quasi-periodic, chaotic, and limitcycle regions, with a small isolated limit-cycle island near the boundary between the quasi-periodic and chaotic regions. Increasing
δ can drive the system from quasi-periodic to chaos and then to a limit cycle. For comparison, when (
ω0T2,
χ/
χc) = (1.2, 8), the quasi-periodic region shrinks markedly toward smaller
δ, which is shown in the phase diagram, Fig. 4 (also see the right figure below). Based on the phase diagram without modulation, Fig. 5, we give a qualitative argument for the origins of the quasi-periodic and chaotic dynamics, and comment that the limit cycles with intricate trajectories (see Fig. 2(e)) are emergent from the nonlinearity of the system. These results enrich the understanding of nonlinear spin dynamics in periodically modulated atomic magnetometers.