ADAPTATIVE STRATEGIES TO IMPROVE COMMUNICATION EFFICIENCY AND AVOID OVERFITTING IN FEDERATED LEARNING

Federated Learning (FL) is a data privacy-aware distributed machine learning framework, which the client utilizes its own private data to train a local model and transmit it to the server side. FL trains machine learning models using decentralized data, especially unbalanced and non-iid. It has been widely used considering legal requirements of privacy in Brazil and Europe. Recently, more attention has been paid to the privacy and security of user data. Despite its advantages, the communication overhead is the main bottleneck of FL.  It also has to handle the challenges of data, model and resources heterogeneity. Adaptive methods can be used to accelerate convergence, reducing the number of rounds of local computation and communication to a centralized server. This thesis proposes a set of adaptive methods using client selection and local updating to preserve the fast convergence and the quality of the models performance. We evaluate the impact of an increasing number of epochs using algorithms such as FedAVG and FedADAM combined with Knowledge distillation (KD), in order to reduce the model size transfering, KD transfer the information of a large model to a small model preserving the model’s precision and convergence.