Optical parametric frequency up-conversion and amplification, used for ultra-weak infrared image detection, is investigated both numerically and experimentally. A circum-circle model of spatial filtering is proposed to suppress the parametric fluorescence background noise. Combining the spatial filter, image relay module and band-pass filter, the fluorescence noise is suppressed by 70%. Accordingly, the image quality is significantly improved and the peak signal-to-noise ratio is increased by 22%. Based on the 10 Hz, mJ-level, 355 nm picosecond pump laser, the weak infrared image is amplified and up-converted into the visible region, and over 1.3108 (82 dB) optical gain is obtained at the same time. Our experiment demonstrates that the detection sensitivity of the conventional uncooled CCD can be as high as 7.4 photons per pixel when the high gain OPA is used as an image preamplifier. The scheme in this paper is expected to be used in single-photon-level sensitive infrared imaging.