TY - GEN
T1 - Automatic Pixel-Level Detection of Structural Steel Elements Using U-Net and Transfer Learning
AU - Jiang, Zhouqian
AU - Messner, John I.
N1 - Publisher Copyright:
© International Conference on Computing in Civil Engineering 2023.All rights reserved.
PY - 2024
Y1 - 2024
N2 - Leveraging deep learning models to automatically detect structural steel elements at the pixel level from visual data can support many potential use cases for construction projects, including automated construction progress monitoring, quality control, safety monitoring, and perception systems for construction robotics. Many researchers have been applying deep learning models to detect various construction entities on sites. However, limited research has focused on detecting structural steel elements at the pixel level. This research focused on implementing U-Net, a state-of-The-Art deep learning model for semantic segmentation, to segment steel beams, girders, and columns from construction images. All images were captured by a drone from a three-story steel building construction site. A data augmentation process was performed to increase the quantity of the training data. Two pre-Trained convolutional neural networks (CNNs), VGG16 and ResNet50, were applied to the U-Net as backbones, respectively, for transfer learning purposes. The U-Net models with different backbones were trained using different sizes of the training data, and the performance was analyzed and compared. Four metrics were used to quantitatively analyze the model performance: precision, recall, F1-score, and Intersection over Union (IoU). This research found that the transfer learning technique improved the model performance by 2%-6%, depending on the analyzed metric. The data augmentation also slightly boosted the model performance. However, both transfer learning and data augmentation increased the overfitting issue. The future work includes combining the segmented steel elements with the as-designed building information modeling (BIM) for automated construction progress monitoring.
AB - Leveraging deep learning models to automatically detect structural steel elements at the pixel level from visual data can support many potential use cases for construction projects, including automated construction progress monitoring, quality control, safety monitoring, and perception systems for construction robotics. Many researchers have been applying deep learning models to detect various construction entities on sites. However, limited research has focused on detecting structural steel elements at the pixel level. This research focused on implementing U-Net, a state-of-The-Art deep learning model for semantic segmentation, to segment steel beams, girders, and columns from construction images. All images were captured by a drone from a three-story steel building construction site. A data augmentation process was performed to increase the quantity of the training data. Two pre-Trained convolutional neural networks (CNNs), VGG16 and ResNet50, were applied to the U-Net as backbones, respectively, for transfer learning purposes. The U-Net models with different backbones were trained using different sizes of the training data, and the performance was analyzed and compared. Four metrics were used to quantitatively analyze the model performance: precision, recall, F1-score, and Intersection over Union (IoU). This research found that the transfer learning technique improved the model performance by 2%-6%, depending on the analyzed metric. The data augmentation also slightly boosted the model performance. However, both transfer learning and data augmentation increased the overfitting issue. The future work includes combining the segmented steel elements with the as-designed building information modeling (BIM) for automated construction progress monitoring.
UR - http://www.scopus.com/inward/record.url?scp=85184280733&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85184280733&partnerID=8YFLogxK
U2 - 10.1061/9780784485224.078
DO - 10.1061/9780784485224.078
M3 - Conference contribution
AN - SCOPUS:85184280733
T3 - Computing in Civil Engineering 2023: Data, Sensing, and Analytics - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2023
SP - 649
EP - 656
BT - Computing in Civil Engineering 2023
A2 - Turkan, Yelda
A2 - Louis, Joseph
A2 - Leite, Fernanda
A2 - Ergan, Semiha
PB - American Society of Civil Engineers (ASCE)
T2 - ASCE International Conference on Computing in Civil Engineering 2023: Data, Sensing, and Analytics, i3CE 2023
Y2 - 25 June 2023 through 28 June 2023
ER -