For deep learning, to do Transfer Learning we take out and replace a few end layers of the network. We can then train just the last layer of weights of a neural network.

The number of layers to remove and then added from pretrained depends on the similarity between tasks. Higher layers in networks are able to recognise higher detail components.