Transformer models in 2023

In Part 4 of this course, students will learn about training and publishing transformer models in 2023. Session 7 will cover the finetuning and inference of common transformer-based language models like BERT, and will include classification and token classification tasks such as named entity recognition. Session 8 will focus on using transformer-based models for time series data, including training timeseries transformers and making predictions on data like stock prices or sales data. In the exercise session, students will train and publish their own transformer model, while the group assignment will involve finetuning a transformer model for a business application of their choice, publishing the model on HF, and building a gradio demo.

Literature