Kursöversikt
The purpose of this course is to provide an introduction to probabilistic modeling and statistical machine learning techniques in terms of their use within the field of language technology. We will review foundational concepts in mathematics and apply them to basic techniques and applications in statistical NLP. We will also learn practical skills in programming text processing pipelines and analytical tools.
The course syllabus in full as adopted by the head of department is available at the following address:
Teachers
Course organizer & lecturer: Sharid Loáiciga (sharid.loaiciga@gu.se), webpage.
Teaching assistants: Maria Szawerna (maria.szawerna@gu.se)
Administrators
Education coordinator: Madelaine Miller (e-mail: madelaine.miller@.gu.se)
Student administration: flov@flov.gu.se
Schedule
https://cloud.timeedit.net/gu/web/schema/ri1X5015Z9107vQQ5wZ6779Y55yY5Y27QQ.html
For remote attendance
Remote attendance is possible through Zoom using the following information
Meeting ID: 637 1914 2218
Passcode: 472925
https://gu-se.zoom.us/j/63719142218?pwd=bwhrTX2Zg7KUA2OrsyJN0eNZqRC1ZU.1
Examination policy
There is no written examination for the course (please ignore any scheduling information to the contrary).
The course will instead be examined through a series of 4 assignments that focus on organizing and transforming data for NLP machine learning and statistical modeling. There won't be any graded labs or quizzes, just the assignments. Each assignment will have equal weight, even if the score denominator is different between assignments.
Extensive bonus points may be offered in the programming assignments and calculated proportionately into the grade.
A grade of G will be given for a 50% of the whole course.
A grade of VG will be given for 90% of the whole course.
Literature V25
- Daniel Jurafsky and James Martin (2008) An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition, Third Edition. Online draft.
- Delip Rao, Brian McMahan (2018) Natural Language Processing with PyTorch: Build Intelligent Language Applications Using Deep Learning. O'Reilly. Available from library and bookstore.
Online guides that will be useful during the course
- Code companion to Rao and McMahan's book
- Python Programming Language
- Scipy/Numpy Quickstart Tutorial
- Pandas Tutorials
- Scikit Learn: Machine Learning in Python
- PyTorch documentation
- NLTK documentation
Note: Canvas Student Guide available at the following Link
Content Plan*
*NB: Contents and pace will be adjusted as needed
| Week | Content | Comment |
Materials | Published | Due |
| 4 (Jan 20-26) |
Introduction, high-level classification, black-box classification
Math review |
2_handout_with_class_notes.pdf Blog post on information entropy |
|||
| 5 (Jan 27-Feb2) | Dimensionality reduction, svd, vector space classification, perceptron, SVMs |
Optional: Tutorial on the math behind SVD
Example: simple perceptron.py |
Assignment 1 | ||
| 6 (Feb 3-9) | Logistic regression, learning process overview | NO LAB SESSION ON 5.02.2025 | |||
| 7 (Feb 10-16) | MaxEnt | EXTRA LAB SESSION ON 13.02.2025 |
|
Assignment 1 | |
| 8 (Feb 17-23) | Self-study |
no lecture session on 2025-02-18 |
Assignment 2 | ||
| 9 (Feb 24-Mar 2) | FFNs |
Reading material: J&M ch 7 or NMT ch 5 by Philipp Koehn (also available in the GU library) Jupyter notebook on Pytorch basics (from Rao and McMahan's book linked above)
|
|||
| 10 (Mar 3-9) | Perplexity, backpropagation, regularization | NO LAB SESSION ON 2025-03-05 |
Reading material: Backpropagation explanation 1 or explanation 2 Recommended extra reading: NMT ch 10 by Philipp Koehn. |
Assignment 3 | Assignment 2 |
| 11 (Mar 10-16) | RNNs, LSTMs, Attention |
Video: Stanford lecture introducing RNNs Stanford slides on LSTMs, MT and attention
Recommended extra reading: |
|||
| 12 (Mar 17-23) | Transformer |
Optional: Original transformer paper by Vaswani et al 2017 |
Assignment 4 | Assignment 3 | |
| 13 (Mar 24-30) | Final thoughts, ethics in NLP | last lecture on 2025-03-25 |
|
||
| 14 (Mar 31 - Apr 6) | Assignment 4 |