Transfer Learning in Deep Learning
Source: Hacker News
As humans, we can transfer our knowledge to learn other tasks. For example, if we learn how to ski we can use the acquired skills or knowledge to learn perhaps snowboarding. It is easy for us to transfer our knowledge to similar tasks. Basically what we are doing when we learn a similar task is to improve what we learned in the past.
So what is transfer learning in Convolutional Neural Networks (CNN)?
Transfer learning is a machine learning technique where a pre-trained model is built and reused as a base architecture for another model. This approach is most commonly used in computer vision and neutral language processing.
Why is this important?
Well these models are bigger architectures with many layers and parameters that I will be mostly unable to train in our local computer. These models can take days or weeks to train on modern hardware. The models are already trained so we can use their pre-trained parameters and use them without wasting the time and computation resources on doing it.
Transfer learning for image classification
Transfer learning is a very common approach in computer vision. It’s normally used for image recognition tasks where we try to predict the type of an image. Our data can not only be images, it can be pictures or videos. However, it’s difficult to find the right amount of data to train these models because they required a huge amount of data to train. For this type of problem, pre-trained models are the way to go.
Normally companies that do research for image classification and that have the resources to develop these models attend this competition called ImageNet where many huge deep learning models are developed. Once the competition ends the company releases a model to the public as open source.
Some examples of this pre-trained models are:
With Keras API you can download for example VGG16 and VGG19 models that are already pre- trained and work with them.
