To study characteristics of rock mass with non-persistent joints under loading in engineering structures, two conceptions are put forward, which are mesoscopic damage of loading and macroscopic damage with joints. The initial damage state of intact rock is defined as a basic state. A compound damage variable is deduced on the basis of the Lemaitre strain equivalence hypothesis, which considers the existence of macroscopic defect with joints, the damage propagation of mesoscopic defects, micro cracks, and the coupling actions of macro and meso-defects under loading. A new calculation formula of the macroscopic damage variable is derived simultaneously in terms of the specimen size, geometrical size of joints and mechanical properties of joints. Then, a damage constitutive model for rock mass with non-persistent joints is established based on coupling macroscopic and mesoscopic defects. This paper describes the evolution of mesoscopic damage and the behavior of macroscopic damage of rock mass with non-persistent joints under loading. The calculated results are in good agreement with the actual failure of rock mass. The results show that: (1) The stress-strain behaviors of fractured and intact rock samples show a significant difference prior to the peak strength. The difference gradually decreases after the peak strength. Finally, the residual stress tends to be equal. (2) The strength of fractured rock mass increases with the joint connectivity rate, and exhibits obvious anisotropy with the variation of joint inclination angle, and also is relevant to the internal friction angle of joints. (3) The peak strength of fractured rock samples is the highest at the joint inclination angle of 90°. While the peak strength is the lowest at the joint inclination angle of 45°for open-type fractured rock samples. (4) The mechanical properties of rock mass under loading in engineering structures are determined by the mesoscopic damage of loading, macroscopic damage with joints and their coupling effects. The coupling macroscopic and mesoscopic based compound damage variable may well characterize mechanical properties of rock mass with non-persistent joints.