A new method of polarization modulation based triple-photoelastic-modulator (triple-PEM) is proposed as an key component of photo-elastic modulator-based imaging spectro-polarimeter (PEM-ISP) combined with acousto optic tunable filter. The basic principles of PEM-ISP and triple-PEM-based differential frequency polarization modulation are described, that is, the tandem PEMs are operated as an electro-optic circular retardance modulator in a high-performance reflective imaging system. Operating the PEMs at slightly different resonant frequencies generates a differential signal that modulates the polarized component of the incident light at a much lower heterodyne frequency. Then the basic equations for polarization measurement is derived by analyzing and calculating its Muller matrix. The simulation and experiments verify the feasibility and accuracy of polarization measurement by triple-PEM-based differential frequency polarization modulation. Finally, we analyze the influences of the setting of integral step and sampling interval of the detector polarization measurement, and a preliminary error analyses of field angle, phase retardation amplitude etc are also be carried out. The result shows that the measurement error of DoLP is less than 0.6% when the phase retardation error is 1%. This work provides the necessary theoretical basis for remote sensing of new PEM-ISP and for engineering implementation of Stokes parametric inversion.