DEPARTMENT OF COMPUTATIONAL AND DATA SCIENCES
Ph.D. Thesis Colloquium
Speaker: Mr. Yashwanth. M
S.R. Number: 06-18-01-10-12-21-1-19272
Title: “Collaboration amid Differences: Generalization, Adaptation, Incentivized Participation and Minimax Optimization in Federated Learning”
Research Supervisor: Prof. Anirban Chakraborty
Date & Time : October 23, 2026 (Friday), 11:00 AM
Venue : #102, CDS Seminar Hall
ABSTRACT
Federated Learning (FL) is a distributed machine learning paradigm, where multiple clients train a model collaboratively under the orchestration of a central server while keeping the training data private to respective clients. Since its introduction, FL has
been enabling numerous privacy-preserving applications across major industries like healthcare and big-tech. Despite its advantages, the federated framework introduces several critical algorithmic challenges under practical constraints. This thesis studies
federated machine learning tasks under a number of these challenges, as described below.
A key challenge in cross-device and cross-silo FL is that the data held by individual clients is typically not independent and identically distributed (non-IID). Under this heterogeneity, each client’s model drifts toward its own local minimum, even when all
clients participate in the federation. The global model, aggregated from these drifted local models, then performs poorly. A single global model does have one clear advantage: it can be deployed to clients that were never seen during training. But because
no single model suits diverse datasets well, many methods turn to partial model personalization. Relying on local parameters, however, leaves the server unable to serve new, unseen clients that never joined training. Heterogeneity thus produces both the ‘client-drift’
problem and a personalization-generalization trade-off. A second challenge originates from a practical point that traditional FL tends to ignore: clients are assumed to participate whether or not they benefit. This is often unrealistic, and encouraging participation
would need a suitable incentive mechanism. FL under client rationality with non-monetary incentives remains unstudied and existing algorithms do little to prevent federation collapse in such settings, especially when data is highly heterogeneous. A third challenge
arises from the data locality constraint itself. Because data stays strictly on each client, objectives like minimax optimization introduce variables tied to individual data points that the server cannot aggregate, leaving their aggregation undefined. This
thesis takes up each of these challenges in turn, formulating and addressing practical, non-trivial FL problems that arise from them.
First, to address client-drift due to label heterogeneity, we introduce Adaptive Self-Distillation (ASD), a client-side regularizer that adapts to each client’s local dataset using the global model’s prediction entropy together with the client’s label distribution.
We justify theoretically that ASD reduces client drift and show empirically that it improves generalization as it results in the global model converging to a ‘flat’ minimum. Because ASD is plug-and-play, integrating it into standard FL methods is straightforward,
and across multiple real-world benchmarks it leads to substantial gains. Building on this connection of generalizability to flatness of loss topology, we next aim to achieve the generalization objective explicitly through a flatness-constrained optimization
framework that controls the largest eigenvalue of the Hessian of training-loss. To keep this tractable during local training, we propose FedMAN, a computationally efficient regularizer that minimizes the per-layer activation norm of the client-side models.
We provide theoretical guarantees for neural nets with fully connected and convolutional layers, showing that minimizing this layer-wise activation norm bounds the top Hessian eigenvalue of the client’s loss, driving the aggregated model toward a flat minimum.
Integrating this regularizer with existing FL methods delivers significant gains in accuracy.
We now explore how we can adapt a pre-trained model in a federated setup, with clients training on local datasets that are unlabeled and exhibit significant domain gaps from one client to another. Towards this end, we address the Federated Source-Free Domain
Adaptation (FFreeDA) problem, where clients train locally on such unlabeled training samples from widely heterogeneous data distributions. The client models are initialized with a pre-trained server model trained on a source domain that is significantly different
from every client’s local data distribution and also unavailable during the federated training. Adapting Source-Free Domain Adaptation (SFDA) methods to this federated setting struggles with client drift: the extreme heterogeneity from the domain gaps results
in unreliable pseudo-labels. We introduce FedSCAl, an FL framework built on our Server-Client Alignment (SCAl) mechanism, which regularizes client updates by matching the client and server model predictions on weak and strong augmentations of the input data,
thereby achieving strong server-client alignment, reduced client-drift as well as more reliable pseudo labels to train local models on. Extensive experiments on benchmark vision datasets show that FedSCAl outperforms the competing FL methods.
In the works described above, we have trained a global model in which the clients share all model parameters during the training. However, to combat data heterogeneity, partial model personalization methods have been proposed. The main drawback here is that
the server model cannot be used for clients which did not participate in the federation. This creates a trade-off in balancing personalization and generalization. In the context of Vision transformer models, when the training paradigm is prompt tuning, we
propose PEP-FedPT, a framework achieving both generalization and personalization in federated prompt tuning of ViTs. At its core is the Class-Contextualized Mixed Prompt (CCMP) – class-specific prompts maintained alongside a globally shared prompt, which CCMP
adaptively combines per input using weights from global class prototypes and client class priors. This gives per-sample prompt personalization without storing any client-dependent trainable parameters and the prompts are optimized collaboratively via federated
averaging. We demonstrate the efficiency of our method on CIFAR-100, TinyImageNet, DomainNet and iNaturalist datasets.
So far, we have assumed that clients take part in the federation willingly under server orchestration and without conditions. In practice, this is often not the case. Clients are usually self-interested and a client may care more about how well the shared
global model performs on its held-out data relative to its own local model performance or any desired threshold. If the clients do not see the benefit they drop out and the whole federation can fall apart. One way to keep the clients engaged is to pay them,
and such monetary incentives have been studied before. In this work we take a different approach with FedUCA, a non-monetary framework built on a non-payment channel between server and clients. The server first shares the exact aggregation weights with each
client, from which the client designs a utility and signals how willing it is to participate. The server then chooses aggregation parameters satisfying each client’s individual-rationality (IR) constraint, sustaining global model performance. Extensive experiments
on standard datasets show that, by prioritizing feasibility of participation, FedUCA achieves higher client retention and, in turn, a stronger global model. As an extension, we design FedLoPers, a lottery-based persuasion scheme that randomizes the information
shared with clients. Rather than committing to the true aggregation weights, the server shares a distribution over the same. This lottery-inspired design further improves client participation.
In all of the above works, the clients minimize their respective loss functions. We now consider the federated minimax problems, a class of problems that includes distributionally robust optimization and adversarial training. Such problems carry an inner maximization
variable that is a worst-case reweighting of a client’s own samples or a perturbation of its own inputs, constructed entirely from local data. Averaging such variables across clients is not just suboptimal but ill-posed. We therefore study the federated minimax
problem in which the minimization variable is shared and aggregated while each client retains a personalized maximization variable that is never shared. In the nonconvex-concave regime, our proposed projected personalized ascent scheme attains the standard
sublinear rate. In the nonconvex-PL regime, the same scheme incurs a stepsize-invariant noise floor from the stochastic ascent updates. To remove it, we propose VR-SGDA, a variance-reduced ascent step. This eliminates the floor and attains the sublinear rate.
Experiments on subpopulation-robustness and adversarial training confirm our theoretical findings and contributions.
ALL ARE WELCOME
