This paper addresses the stabilization problem of a class of switched linear systems. Due to the presence of switching delay in the switching signal of the controller, the switching of the controller and the of the system are turned out to be asynchronous. By combining switching signal method with the multiple Lyapunov function method, some sufficient conditions are provided to guarantee the stability of the whole switched system by an adjacent mode-dependent average dwell time scheme. These conditions imply the relationship among the continuous dynamics, switching delay, and the average dwell time. Finally, a numerical example is presented to illustrate the effectiveness of the proposed method.