Federated Learning With Generalization To New Domains
Loading...
Authors
Soltany, Milad
Date
2024-11-29
Type
thesis
Language
eng
Keyword
Federated Learning , Computer Vision , Deep Learning , Domain Generalization , Federated Domain Generalization , Self Supervised Learning
Alternative Title
Abstract
Federated Learning (FL) is an area of research that focuses on training machine learning models in a decentralized fashion without having the need to store all data on one central server.
In this thesis, we address the challenges of data heterogeneity and label scarcity in FL by proposing two novel approaches for federated domain generalization in both unsupervised and supervised settings.
First, to tackle federated domain generalization in an unsupervised setting, we introduce Federated Unsupervised Domain Generalization using Global and Local Alignment of Gradients. We establish a connection between domain shifts and gradient alignment in unsupervised federated learning, demonstrating that aligning gradients at both the client and server levels facilitates the generalization of the model to new, unseen domains. FedGaLA performs gradient alignment locally to encourage clients to learn domain-invariant features, and globally at the server to obtain a more generalized aggregated model. Extensive experiments on four multi-domain datasets—PACS, OfficeHome, DomainNet, and TerraInc—show that FedGaLA outperforms comparable baselines. Ablation and sensitivity studies highlight the impact of different components and hyper-parameters in our approach.
Second, to address data heterogeneity in a supervised federated learning framework, we propose Federated Domain Generalization with Label Smoothing and Balanced Decentralized Training (FedSB). FedSB utilizes label smoothing at the client level to prevent overfitting to domain-specific features, thereby enhancing generalization capabilities across diverse domains when aggregating local models into a global model. Additionally, FedSB incorporates a decentralized budgeting mechanism that balances training among clients, improving the performance of the aggregated global model. Experiments on four commonly used multi-domain datasets—PACS, VLCS, OfficeHome, and TerraInc—demonstrate that FedSB outperforms competing methods, achieving state-of-the-art results on three out of four datasets.
Collectively, these contributions address critical challenges in FL by enhancing model generalization across diverse and unseen domains in both unsupervised and supervised settings. The effectiveness of FedGaLA and FedSB in addressing data heterogeneity is evidenced by their superior performance in extensive empirical evaluations.
Description
Citation
Publisher
License
Queen's University's Thesis/Dissertation Non-Exclusive License for Deposit to QSpace and Library and Archives Canada
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
Attribution 4.0 International
ProQuest PhD and Master's Theses International Dissemination Agreement
Intellectual Property Guidelines at Queen's University
Copying and Preserving Your Thesis
This publication is made available by the authority of the copyright owner solely for the purpose of private study and research and may not be copied or reproduced except as permitted by the copyright laws without written authority from the copyright owner.
Attribution 4.0 International