Machine learning plays a central role in crucial applications that require learning tasks to be carried out in a decentralized fashion. In real-world decentralized settings, hardware components may crash, the software can be buggy, communications can be slowed down, machines can be hacked, and data can be corrupted, e.g., user-generated data may be crafted maliciously. In distributed computing, "nodes" can be Byzantine, i.e., they can behave arbitrarily maliciously. To the best of our knowledge, at the current time the field of Byzantine resilience within decentralized learning arguably rests on fragile foundations. Current techniques are hard to prove correct and rely on assumptions that are unrealistic making it difficult to compare approaches.
In the context of this PhD project, the PHD Candidate will investigate Byzantine faults and resilience mechanisms for decentralized learning. Byzantine faults are the most generic type of faults caused involuntarily (e.g., a node crashes, has bugs) or voluntarily (e.g., a node exhibits selfish behavior, or performs attacks such as data/model poisoning) by a node in a distributed system. The PHD candidate will investigate both novel Byzantine attacks and resilience mechanisms, including resilient communication protocols and resilient model aggregation strategies. We will further consider attacks performed by external Byzantine adversaries. For instance, the PHD candidate will revisit accountability mechanisms to deter Byzantine faults and generate proof of misbehavior for convincing third parties.