The effective permittivity, effective permeability and the phase factor of a rectangular waveguide filled with an anisotropic metamaterial is derived through solving the wave equation. A group of important conclusions is obtained through discussing the relations between transmission characteristics of the waveguide and constitutive parameters of the filling metamaterial. It is shown that characteristics of the waveguide are determined by the transversal constitutive parameters when the operating frequency is higher than the cut_off frequency, and they are independent on the longitudinal constitutive parameters. However, below the cut off frequency, the characteristics of the waveguide are determined by both the transversal and longitudinal constitutive parameters. Two examples are calculated to demonstrate the above conclusions, and it is shown that calculated results are in a good agreement with the theoretical predictions. These results play important role in the applications of metamaterials to waveguide.