Federated Learning for Large Scale Distributed Systems

Simulation and detailed description publicly available at:

Related to the papers:

``Field-Based Coordination for Federated Learning ’’ published at International Conference on Coordination Models and Languages (DOI: 10.1007/978-3-031-62697-5_4).
``Proximity-based Self-Federated Learning ’’ published at International Conference on Autonomic Computing and Self-Organizing Systems (DOI: 10.1109/ACSOS61780.2024.00033).
``FBFL: A Field-Based Coordination Approach for Data Heterogeneity in Federated Learning ’’ submitted at Journal Logical Methods in Computer Science (DOI: arXiv.2502.08577).
``Decentralized Proximity-Aware Clustering for Collective Self-Federated Learning ’’ submitted at Journal Internet of Things.

Abstract

In the era of pervasive devices and edge intelligence, Federated Learning enables multiple distributed nodes to collaboratively train machine learning models without exchanging raw data. This fosters privacy preservation and bandwidth efficiency. Yet, real-world deployments face challenges of data heterogeneity, network dynamics, and lack of centralized control.

In our work, we harness the Alchemist simulator as a playground for exploring field-based, self-organizing, and spatial coordination paradigms for federated learning. Through three complementary simulation experiments, we investigate how devices can autonomously organize, aggregate, and adapt models in a fully decentralized manner, leveraging computational fields and spatial interaction patterns.

Simulations

1. Field-Based Federated Learning (FBFL)

This simulation explores the emergence of personalized model zones using computational fields as a distributed coordination mechanism. Devices in spatial proximity exchange model parameters through local diffusion processes, forming hierarchical regions governed by dynamically elected leaders. These leaders act as aggregators for their respective zones, enabling localized learning and self-stabilizing coordination.

Methodology & Implementation

Distributed spatial leader election: nodes autonomously select aggregators within neighborhoods
Hierarchical aggregation: aggregation occurs at multiple levels based on spatial zones
Field coordination primitives: implemented via ScaFi over Alchemist, using constructs like diffusion, convergence, and feedback loops
Datasets used: MNIST, FashionMNIST, Extended MNIST
Simulation parameters: (here you can insert the number of nodes, communication radius, number of rounds, etc.)

Key Outcomes

Accuracy under IID settings comparable to centralized FedAvg
Superior robustness to node failures and topology changes
Emergent spatial clusters aligned to local data distributions
No reliance on any central server

Proximity-based Self-Federated Learning

This experiment introduces a collective intelligence paradigm for federated IoT systems. Devices autonomously cluster into self-federations based on both spatial proximity and model similarity, forming specialized local models that better represent underlying regional data distributions.

Goal: Enable adaptive, decentralized FL in non-IID IoT environments.
Method: Field-based coordination using Self-Organizing Coordination Regions (SCR) and space-fluid sparse choice for dynamic federation formation.
Implementation: Each node locally computes similarity with neighbors, elects a regional leader, and participates in decentralized aggregation cycles.
Dataset: Extended MNIST and CIFAR-100.

Federation evolution under aggregator failure

Start of the learning.

Federations stabilization.

Learning.

Aggregators failure.

Federations re-stabilization.

Learning resumes.