Fine-tuning transformers

Resources

• You will need a HF🤗 account to upload models
• We will use WandB for monitoring - get an account if you like to follow that part

Notebooks

• Transformer-based Translation (inference)

• Finetuning Sequence Classification with Bert

Homework - As a preparation for final assignment‼️

Finetune a model for token-classification

• Follow this tutorial
• Cheating option 1: NERDA, Thanks to the lovely people at Ekstrabladet/PIN
• Cheating option 2: Simpletransformers
• Try custom Named Entities Skills, Scientific Terms
• Make your own NER dataset using Argilla and 0-shot labeling with Flair - you can deploy argilla with 1-click on 🤗 spaces and connect to it via e.g. Colab.

Nerd-out options

• GPT-2 finetuning. Make it produce text in a certain style. Follow this tutorial
• Build a retreival based Question-Answering system with [🦜⛓️LangChain][https://langchain.readthedocs.io/en/latest/]