Automatic Pixel-Level Detection of Structural Steel Elements Using U-Net and Transfer Learning

Zhouqian Jiang, John I. Messner

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Leveraging deep learning models to automatically detect structural steel elements at the pixel level from visual data can support many potential use cases for construction projects, including automated construction progress monitoring, quality control, safety monitoring, and perception systems for construction robotics. Many researchers have been applying deep learning models to detect various construction entities on sites. However, limited research has focused on detecting structural steel elements at the pixel level. This research focused on implementing U-Net, a state-of-The-Art deep learning model for semantic segmentation, to segment steel beams, girders, and columns from construction images. All images were captured by a drone from a three-story steel building construction site. A data augmentation process was performed to increase the quantity of the training data. Two pre-Trained convolutional neural networks (CNNs), VGG16 and ResNet50, were applied to the U-Net as backbones, respectively, for transfer learning purposes. The U-Net models with different backbones were trained using different sizes of the training data, and the performance was analyzed and compared. Four metrics were used to quantitatively analyze the model performance: precision, recall, F1-score, and Intersection over Union (IoU). This research found that the transfer learning technique improved the model performance by 2%-6%, depending on the analyzed metric. The data augmentation also slightly boosted the model performance. However, both transfer learning and data augmentation increased the overfitting issue. The future work includes combining the segmented steel elements with the as-designed building information modeling (BIM) for automated construction progress monitoring.

Original languageEnglish (US)
Title of host publicationComputing in Civil Engineering 2023
Subtitle of host publicationData, Sensing, and Analytics - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2023
EditorsYelda Turkan, Joseph Louis, Fernanda Leite, Semiha Ergan
PublisherAmerican Society of Civil Engineers (ASCE)
Pages649-656
Number of pages8
ISBN (Electronic)9780784485224
DOIs
StatePublished - 2024
EventASCE International Conference on Computing in Civil Engineering 2023: Data, Sensing, and Analytics, i3CE 2023 - Corvallis, United States
Duration: Jun 25 2023Jun 28 2023

Publication series

NameComputing in Civil Engineering 2023: Data, Sensing, and Analytics - Selected Papers from the ASCE International Conference on Computing in Civil Engineering 2023

Conference

ConferenceASCE International Conference on Computing in Civil Engineering 2023: Data, Sensing, and Analytics, i3CE 2023
Country/TerritoryUnited States
CityCorvallis
Period6/25/236/28/23

All Science Journal Classification (ASJC) codes

  • General Computer Science
  • Civil and Structural Engineering

Cite this