Thesis of Julien Nicolas


Subject:
Resilient and Privacy-Preserving Decentralized Recommender System

Start date: 24/11/2023
End date (estimated): 24/11/2026

Advisor: Sonia Ben Mokhtar

Summary:

There is a strong momentum towards data-driven services at all layers of society and industry. This started from large scale web-based applications such as Web search engines (e.g., Google, Bing), social networks (e.g., Facebook, TikTok, Twitter, Instagram) and recommender systems (e.g., Amazon, Netflix) and is becoming increasingly pervasive thanks to the adoption of handheld devices and the advent of the Internet of Things.
Recent initiatives such as Web 3.0 are coming with the promise of decentralizing such services for empowering users with the ability to gain back control over their personal data, and prevent a few economic actors from over concentrating decision power. However decentralizing online services calls for decentralizing the machine learning algorithms on which they heavily rely. In this context, recent research works have investigated various strategies for distributing machine learning task (e.g., Federated Learning [1], Gossip Learning[2]). Applied to recommender systems, these strategies have led to the emergence of recommender systems of two types: Federated Recommender Systems (e.g., [3]) and Gossip Learning-based Recommender Systems (e.g., Pepper[4])
In this PhD project we will investigate decentralized recommender systems by focusing on three main aspects: (i) personalization; (ii) privacy and (iii) resilience. Specifically, we will investigate how effective are decentralized recommender systems in providing personalized recommendation to the users. We will further 
study the effectiveness of existing privacy and Byzantine attacks on decentralized recommender systems (e.g., data property inference attack, membership attack, poisoning attacks) that have been successfully tested in the context of centralized or federated machine learning [5,6,7,8]. The objective is then to assess whether existing attacks are still effective in a decentralized context. If privacy/resilience vulnerabilities are detected, novel model exchange strategies and model aggregation algorithms shall be explored. 

The PhD candidate shall follow the following agenda:

•    Carry out a state-of-the-art analysis on Federated Recommender Systems and Gossip Learning-based Recommender Systems.
•    Carry out state-of-the-art analysis on privacy and Byzantine attacks in distributed learning systems.
•    Study how decentralization affects the performance of recommender systems.
•    Assess the resilience of decentralized recommender systems to privacy and Byzantine attacks.
•    Investigate model exchange strategies and model aggregation functions that improve the personalization, privacy and resilience of decentralized recommender systems.
Bibliography

[1] Brendan McMahan, Eider Moore, Daniel Ramage, Seth Hampson, and Blaise Aguera y Arcas. Communication- efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics, pages 1273– 1282. PMLR, 2017. 
[2] István Hegedu ̋s, Gábor Danner, and Márk Jelasity. Gos- sip learning as a decentralized alternative to federated learning. In Distributed Applications and Interoperable Systems: 19th IFIP WG 6.1 International Conference, DAIS 2019, Held as Part of the 14th International Feder- ated Conference on Distributed Computing Techniques, DisCoTec 2019, Kongens Lyngby, Denmark, June 17–21, 2019, Proceedings 19, pages 74–90. Springer, 2019. 
 [3] Muhammad Ammad-Ud-Din, Elena Ivannikova, Suleiman A Khan, Were Oyomno, Qiang Fu, Kuan Eeik Tan, and Adrian Flanagan. Federated collaborative filtering for privacy-preserving personalized recom- mendation system. arXiv preprint arXiv:1901.09888, 2019. 
[4] Yacine Belal, Aurélien Bellet, Sonia Ben Mokhtar, and Vlad Nitu. Pepper: Empowering user-centric recom- mender systems over gossip learning. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiqui- tous Technologies, 6(3):1–27, 2022. 
[5] Chen Chen, Lingjuan Lyu, Han Yu, and Gang Chen. Practical attribute reconstruction attack against feder- ated learning. IEEE Transactions on Big Data, 2022. 
[6] Chong Fu, Xuhong Zhang, Shouling Ji, Jinyin Chen, Jingzheng Wu, Shanqing Guo, Jun Zhou, Alex X Liu, and Ting Wang. Label inference attacks against vertical federated learning. In 31st USENIX Security Symposium (USENIX Security 22), pages 1397–1414, 2022. 
[7] Yuhao Gu, Yuebin Bai, and Shubin Xu. Cs-mia: Mem- bership inference attack based on prediction confidence series in federated learning. Journal of Information Security and Applications, 67:103201, 2022. 
[8] Lingjuan Lyu and Chen Chen. A novel attribute recon- struction attack in federated learning. arXiv preprint arXiv:2108.06910, 2021.