Layer-Wise Adaptive Weighting for Faster Convergence in Federated Learning

Vedant S. Lanjewar, Hai Anh Tran, Truong X. Tran

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Federated learning (FL), a decentralized approach utilizing multiple devices for training machine learning algorithms, presents an innovative solution to the challenges associated with handling sensitive personal data in neural network training. This research explores the application of FL for training neural network models, emphasizing its benefits in safeguarding privacy, optimizing efficiency, and addressing communication costs. While FL offers advantages such as enhanced data efficiency, promotion of heterogeneity, and scalability, challenges arise from the presence of non-independent and non-identically distributed data in the real world. This research introduces the FedLayerWise algorithm, a novel approach that dynamically assigns weights to each layer of clients' models based on their contributions to the global model. The algorithm leverages the relationship between gradients of the client model and the global model to adaptively modify weights post-training, expediting convergence towards optimal loss. Experimental results on the MNIST dataset with non-identical distributions demonstrate the algorithm's effectiveness. FedLayerWise outperforms existing algorithms (FedAdp and FedAvg) under highly skewed non-IID data distribution, achieving faster convergence. The proposed modifications showcase improvements and converge faster to non-IID data distributions, reducing computation rounds by about 4 0 % compared to FedAdp and about 5 0 % compared to FedAvg in such scenarios.

Original languageEnglish (US)
Title of host publicationProceedings - 2024 IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages126-131
Number of pages6
ISBN (Electronic)9798350351187
DOIs
StatePublished - 2024
Event25th IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024 - San Jose, United States
Duration: Aug 7 2024Aug 9 2024

Publication series

NameProceedings - 2024 IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024

Conference

Conference25th IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024
Country/TerritoryUnited States
CitySan Jose
Period8/7/248/9/24

All Science Journal Classification (ASJC) codes

  • Computer Vision and Pattern Recognition
  • Information Systems
  • Information Systems and Management
  • Safety, Risk, Reliability and Quality

Fingerprint

Dive into the research topics of 'Layer-Wise Adaptive Weighting for Faster Convergence in Federated Learning'. Together they form a unique fingerprint.

Cite this