The fields in 3-dimensional tapered waveguides are unstable compared with the fields in the straight waveguides. In the case of waveguide-to-fiber coupling and fiber-to-waveguide coupling, a sequence of short straight waveguides has been modeled to approximate the 3-dimensional tapered waveguide; and the unstable incident and reflected fields, as well as their derivatives, were determined by the beam propagation method(BPM). Then free space radiation mode(FSRM) was employed to calculate the reflected and transmitted powers. Analysis results of the coupling of fiber with silicon-on-insulator(SOI) tapered rib waveguides showed the feasibility of the method.