TY - GEN
T1 - Layer-Wise Adaptive Weighting for Faster Convergence in Federated Learning
AU - Lanjewar, Vedant S.
AU - Tran, Hai Anh
AU - Tran, Truong X.
N1 - Publisher Copyright:
© 2024 IEEE.
PY - 2024
Y1 - 2024
N2 - Federated learning (FL), a decentralized approach utilizing multiple devices for training machine learning algorithms, presents an innovative solution to the challenges associated with handling sensitive personal data in neural network training. This research explores the application of FL for training neural network models, emphasizing its benefits in safeguarding privacy, optimizing efficiency, and addressing communication costs. While FL offers advantages such as enhanced data efficiency, promotion of heterogeneity, and scalability, challenges arise from the presence of non-independent and non-identically distributed data in the real world. This research introduces the FedLayerWise algorithm, a novel approach that dynamically assigns weights to each layer of clients' models based on their contributions to the global model. The algorithm leverages the relationship between gradients of the client model and the global model to adaptively modify weights post-training, expediting convergence towards optimal loss. Experimental results on the MNIST dataset with non-identical distributions demonstrate the algorithm's effectiveness. FedLayerWise outperforms existing algorithms (FedAdp and FedAvg) under highly skewed non-IID data distribution, achieving faster convergence. The proposed modifications showcase improvements and converge faster to non-IID data distributions, reducing computation rounds by about 4 0 % compared to FedAdp and about 5 0 % compared to FedAvg in such scenarios.
AB - Federated learning (FL), a decentralized approach utilizing multiple devices for training machine learning algorithms, presents an innovative solution to the challenges associated with handling sensitive personal data in neural network training. This research explores the application of FL for training neural network models, emphasizing its benefits in safeguarding privacy, optimizing efficiency, and addressing communication costs. While FL offers advantages such as enhanced data efficiency, promotion of heterogeneity, and scalability, challenges arise from the presence of non-independent and non-identically distributed data in the real world. This research introduces the FedLayerWise algorithm, a novel approach that dynamically assigns weights to each layer of clients' models based on their contributions to the global model. The algorithm leverages the relationship between gradients of the client model and the global model to adaptively modify weights post-training, expediting convergence towards optimal loss. Experimental results on the MNIST dataset with non-identical distributions demonstrate the algorithm's effectiveness. FedLayerWise outperforms existing algorithms (FedAdp and FedAvg) under highly skewed non-IID data distribution, achieving faster convergence. The proposed modifications showcase improvements and converge faster to non-IID data distributions, reducing computation rounds by about 4 0 % compared to FedAdp and about 5 0 % compared to FedAvg in such scenarios.
UR - http://www.scopus.com/inward/record.url?scp=85207838751&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85207838751&partnerID=8YFLogxK
U2 - 10.1109/IRI62200.2024.00036
DO - 10.1109/IRI62200.2024.00036
M3 - Conference contribution
AN - SCOPUS:85207838751
T3 - Proceedings - 2024 IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024
SP - 126
EP - 131
BT - Proceedings - 2024 IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024
PB - Institute of Electrical and Electronics Engineers Inc.
T2 - 25th IEEE International Conference on Information Reuse and Integration for Data Science, IRI 2024
Y2 - 7 August 2024 through 9 August 2024
ER -