Convenience method for learning rate factor

[alanakbik]

This PR adds a parameter to set a factor on the learning rate of the decoder, if fine-tuning a model.

Usage:

trainer.fine_tune(f"path/to/output/folder",
                  mini_batch_size=4,
                  learning_rate=5e-5,
                  decoder_lr_factor=10,
                  )

[helpmefindaname]

Hi @alanakbik,
if I understand this PR correctly, this is to set a different LR for pretrained weights and randomly initialized weights on all DefaultClassifier's?

I suppose this could be extended even further to for example SequenceTagger, by filtering by embedding e.g:

embedding_parameters = [param for name, param in self.model.named_parameters() if "embedding" in name]
model_parameters = [param for name, param in self.model.named_parameters() if "embedding" not in name]

Such that we could train a Transformer-Bert model with a higher LT for the CRF part

[alanakbik]

Yes, it is for training the non-pretrained (i.e. randomly initialized) parts with a higher LR. Since the decoder is always randomly initialized, it is handled here. Extending this to the LSTM-CRF of the SequenceTagger would be great, but some embeddings (like CharacterEmbeddings) are randomly initialized, while others are not. So I think it's not easy to come up with a good heuristic to identify those parts.

Edit: I'll merge this now for experimentation, but any ideas to improve are welcome!