Automating Visual Inspection, Documentation, and Assessment of Masonry Structures Using Computer Vision and Deep-Learning Techniques

Masonry, valued for its durability and sustainability, is a common building material. However, many masonry structures have surpassed 100-year span, and show significant signs of deterioration. Therefore, inspecting and assessing these structures is crucial for their continued function and longevity. This research aims to streamline traditional inspection and assessment processes through automation. The primary contribution of this work is the development of advanced deep-learning and computer-vision techniques. These techniques enable precise detection, segmentation, and documentation of intricate masonry micro-geometry using digital data sources like images, point clouds, and reality-mesh objects. Additionally, this research involves creating precise numerical models for masonry assessment and studying how geometric accuracy affects numerical analysis. These objectives have been successfully achieved across various publications, encompassing both 2D and 3D environments.
More specifically, the initial phase of the research focused on automating the generation of geometric and numerical models of masonry structures from photographs, utilizing computer-vision methods. This process included generating CAD drawings for documentation and DEM (Discrete Element Method) models for structural assessment. The validity of this approach was confirmed by comparing it to idealized and precise numerical models of different structures. Subsequently, improvements were made by incorporating deep learning for the semantic segmentation of masonry micro-geometry into the existing workflow. This resulted in a more reliable method for detecting masonry features. This step also involved integrating an existing defect-detection model and developing a new block-detection model based on convolutional neural networks (CNNs). The enhanced workflow was further validated by comparing manually and automatically generated geometry using DEM. Finally, the research was extended to a 3D environment, where a realistic 3D model was used to generate a classified point cloud of any masonry structure. Feature detection in this 3D context benefited from CNN models for blocks and cracks, with additional classifications (mortar and other elements) estimated using image-processing techniques.
In conclusion, the developed workflows simplify a significant portion of the manual procedures involved in visual inspection, assessment, and even computer graphics generation. In most cases, the generated outputs meet the accuracy standards required for commercial use, assuming clear digital input with visible material changes. However, opportunities for improvement remain, primarily in refining detection techniques to enhance accuracy, addressing the mortar-mesh effect in numerical analysis, and achieving a solid 3D reconstruction of identified geometry to generate 3D numerical models for assessment.