Federated Learning - FourWeekMBA

Federated learning is the decentralized form of machine learning where the model is trained on decentralized data across multiple edge devices.

Understanding federated learning

Traditional machine learning techniques require the training data from edge devices to be aggregated and centralized in a data center or machine. Machine learning algorithms then train themselves on the data and run the model on a cloud server where it can be accessed via various applications.

However, the traditional technique is subject to privacy concerns. Since tech giants such as Amazon, Microsoft, and Google offer cloud-based AI solutions, sensitive user data is sent to their servers where the models are trained.

Federated learning is one way to remedy this issue. Born at the intersection of blockchain, on-device AI, and edge computing, the approach involves training a centralized machine learning model on decentralized data.

How does federated learning work?

To understand how the process works, consider a smartphone. Federated learning enables smartphones to learn a shared prediction without the training data leaving the device. In other words, machine learning can take place without the need to store the data in the cloud.

Note that federated learning moves beyond local models that already make predictions on smartphones like the Mobile Vision API. This is because they enable model training to occur on the device as well.

When a smartphone downloads the current model, it improves it with data from the phone, and that improvement is summarized as a small update. Importantly, only the update is sent to the cloud and in any case, is encrypted and averaged with other user updates.

The three types of federated learning

There are three main types of federated learning:

Horizontal – where the central model is trained on similar datasets.
Vertical – where datasets are complementary. For example, book and movie reviews can be combined to predict someone’s music interests, and
Federated transfer learning – where a pre-trained model that performs one task is trained on a different dataset to perform another task. For example, banks could train an AI model to detect fraud and then repurpose it elsewhere.

Benefits of federated learning

Since models are trained on the device, applications can continue to function even when the device has no internet access. Users who are on metered connections will also appreciate the ability of federated learning to save them bandwidth.

What’s more, in many cases, on-device inference is far more energy-efficient than constantly sending data to the cloud. Since training data remains on the device, it can also be used to train models to deliver a personalized experience.

More detail is provided on this in the next section.

Federated learning and Gboard

Google is currently testing federated learning in Gboard on Android – otherwise known as the Google Keyboard. When Gboard shows a user a suggested query, the smartphone stores information on the current context and whether the query was clicked on.

Federated learning then processes a user’s search history and behavior on-device to deliver improvements the next time Gboard displays suggestions.

Key takeaways:

Federated learning is the decentralized form of machine learning where the model is trained on decentralized data across multiple edge devices.
When a device downloads the current model, it improves it with data with that improvement summarized as an update. Only the update is sent to the cloud and in any case, it is encrypted and averaged with other user updates to improve the model.
Federated learning has several benefits. Since models are trained on the device, applications can continue to function even when the device has no internet access. There are also improvements in bandwidth usage and the ability to deliver personalized experiences on devices.