Federated Class-Incremental Learning under class heterogeneity

Abstract

Modern intelligent systems are increasingly deployed across distributed devices that operate on continuously evolving data, where neither centralised data collection nor static training assumptions are realistic. Federated Continual Learning (FCL), and specifically Federated Class-Incremental Learning (FCIL), combines the privacy-preserving collaboration of Federated Learning with the lifelong adaptability of Continual Learning. Yet FCL inherits and amplifies the difficulties of both fields, of which heterogeneity is the most pervasive: heterogeneous data and systems across clients, constrained communication, asynchronous task arrival, privacy leakage, and the stability–plasticity tension that drives catastrophic forgetting.

This thesis frames the open challenges of FCIL through the lens of heterogeneity and advances the field on three fronts. First, it presents a structured survey that traces each FCL challenge back to its origin in Federated or Continual Learning and reviews methodologies along four families: replay-based, regularisation-based, parameter-isolation and decomposition, and knowledge-distillation methods.

Second, it proposes FedLoCA, a low-rank class-adaptive framework for class-imbalance heterogeneity, where classes are unevenly distributed across clients. Client updates are decomposed into a globally shared low-rank basis, which aligns client geometries and reduces communication, and class-specific adaptation memories, which preserve dedicated knowledge pathways. Aggregation is performed class-by-class, with bounded shared-basis drift and class-wise normalisation limiting logit imbalance between recent and older classes.

Third, it introduces FCIL-VC, the first framework, to our knowledge, studying task-cardinality heterogeneity in FCIL, where successive tasks introduce a variable number of new classes. It combines a non-parametric cosine prototype classifier, a permutation-invariant Cardinality-Agnostic Task Encoder (CATE) mapping tasks of arbitrary size to fixed-dimensional Gaussian embeddings, a closed-form 2-Wasserstein inter-task separator, and bounded exemplar replay with old-model knowledge distillation.

Experiments on CIFAR-10, CIFAR-100 and TinyImageNet for FedLoCA, and on CIFAR-10, MNIST and Fashion-MNIST under variable cardinality schedules for FCIL-VC, show that the two methods consistently outperform representative FCIL baselines in final retained accuracy and average continual accuracy, and that the advantage widens precisely in the regimes where heterogeneity is most severe. Together, the survey, FedLoCA and FCIL-VC provide both a conceptual map and two concrete, communication-efficient pathways for scalable, privacy-aware lifelong learning under realistic federated heterogeneity.