20-00-0947-iv Deep Learning für Natural Language Processing

Lehrende: Ph.D. Ivan Habernal

Veranstaltungsart: Integrierte Veranstaltung

Orga-Einheit: FB20 Informatik

Anzeige im Stundenplan: DL4NLP

Fach:

Anrechenbar für:

Semesterwochenstunden: 4

Unterrichtssprache: Deutsch

Min. | Max. Teilnehmerzahl: - | -

Lehrinhalte:
All you need to know about contemporary natural langauge processing (NLP) using deep learning. More about foundations, less about particular frameworks or implementations.

Content:

- Deep learning foundations (learning from data, learning problem formalization, loss functions, training with backpropagation, evaluation)
- NLP as supervised task learning
- Language representation (word embeddings, multi-lingual embeddings)
- Prominent architectures (convoluational neural networks, recurrent neural networks)
- Contemporary architectures and foundational models (transformers and BERT)
- Applications (text classification, text generation, translation)

Literatur:
Literature will be announced during the lectures but here are some great textbooks that are freely available.

Goldberg (2017). Neural Network Methods for Natural Language Processing. Morgan & Claypool Publishers.
All TU-Da students can download the PDF at https://www.morganclaypool.com/doi/10.2200/S00762ED1V01Y201703HLT037 (use VPN outside the campus)

Goodfellow et al. (2016). Deep Learning. MIT Press.
HTML freely accessible at https://www.deeplearningbook.org/

Deisenroth et al. (2020). Mathematics for Machine Learning. Cambridge University Press.
PDF freely accessible at https://mml-book.github.io/ (updated continuosly)

Voraussetzungen:
Mathematics (calculus, esp. derivatives and gradients; basic linear algebra; basic probability theory)
Python 3 programming

Online-Angebote:
Lectures and links to YouTube videos available at https://github.com/dl4nlp-tuda/deep-learning-for-nlp-lectures