Introduction to transformers

Literature

Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to sequence learning with neural networks. Advances in neural information processing systems, 27.

Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., … & Polosukhin, I. (2017). Attention is all you need. Advances in neural information processing systems, 30.

The illustrated transformer

Simple transformer LM

Notebooks

• Seq2Seq - Neural Machine Translation

• Simple transformer LM

Slides