Characterizing the microstructure of shales is challenging due to its extremely small scale, and generally involves manual interpretation of image data. Here, we present a completely automatic machine learning method for quantifying the preferential mineral-microfracture relationships in intact and deformed shales. This new method is innovative because it allows automated analysis of large volumes of image data, which is typically very time- and labor-intensive with existing techniques. Using automatic object detection algorithms, 225 images, including energy-dispersive X-ray spectroscopy images and backscattered electron images, were analyzed. These images were obtained from deformed and intact samples of a carbonate-rich organic shale and a siliceous organic shale. We quantitatively characterized the location and size of microfractures and their preferential association with particular minerals. The results show that compression created microfractures at the grain scale. More than 90% of microfractures developed within organic matter (OM), and that the microfractures tended to develop along grain/mineral boundaries. In addition, we found that the fabric of the rock plays an important role in microfracture generation, with laminated OM and clay tending to favor microfracture development while more massive minerals inhibited it. This quantitative analysis helps to improve understanding of the micromechanics of deformation during hydraulic fracturing. In addition, this approach is completely automatic, which could increase work efficiency and reduce the effects of subjective decision-making. The work presented here will greatly improve future studies of quantifying fracture and mineral properties, and can provide guidance for hydraulic fracturing and production strategies.