The shallow granite residual soil from a constructional engineering in the coast of Southeast China is selected as the test samples, and then the ultrasonic technology of rock-soil damage detection is used to test and calculate longitudinal ultrasonic wave velocity of the samples under the different impact loads. The longitudinal wave velocity is selected as damage variable to calculate the damage degree, and the influence relationship of the impact frequency, as well as impulse on damage degree is analyzed. Based on the actual situation of sample damage, the characteristics of sample damage evolution under impact load have been analyzed at the same time. The results show that the damage degree of residual soil has a tendency to increase with the increase of impact frequency of impulse; and the impact damage evolution process of residual soil can be divided into several different stages, including small deformation, crack in the telos, bulging or crack in front of 1/3, crack propagation and surface spall in the telos with the increase of damage degree. The research results will provide the scientific basis and quantitative parameters for us to disclose the rule of damage evolution of the residual soil under dynamic loading.