At present the limit equilibrium method is generally used to evaluate the stability of composite foundation. In this procedure, the composite foundation theory assumes the cohesion and friction angle of rigid pile are equivalent to that of the whole foundation, and then the safety factor of the whole foundation is obtained by circular slice method. This theory considers the shear failure at critical state for all rigid pile and the soil. However, according to the relevant research results, shear failure is not exclusive for group piles, and the safety factor based on the shear failure mode may tend to be overestimated. Based on the actual failure mode of rigid pile group, a strength reduction method is developed for rigid pile failure. That is, when the critical state of composite foundation system is searched by strength reduction method, the failure mode of rigid pile should meet the actual damage. At the same time, on the basis of the deformation and failure characteristics of the rigid pile, methods for identifying critical state are discussed. The feasibility and validity of the proposed method are verified through comparing the proposed method and the traditional method only considering shear failure.