Build Your Own Neural Networks

Course Description

In this course, we will build a toy deep learning framework from scratch (tensorslow😅), and students could understand the essence of deep learning through such experience. This course is a project seminar for senior undergraduate students, and highlights learning by doing.

Instructor: Meng Li

Time: Fridays, 10:00 - 15:00 (Oct 17, 24; Nov 07, 14; Dec 19)

Room: 2.14.0.32

Course Management System: Moodle

Syllabus

• Day 1 (2025-10-17):
  • 10:15 - 12:00. From Perceptron to Feed-Forward Neural Networks
  • 13:00 - 15:00. Backpropagation and Learning Tricks
• Day 2 (2025-10-24):
  • 10:15 - 12:00. Automatic Differentiation
  • 13:00 - 15:00. Expand and Optimize Code
• Day 3 (2025-11-07):
  • 10:15 - 12:00. Calculate Higher-order Derivatives
  • 13:00 - 15:00. Create Neural Networks
• Day 4 (2025-11-14):
  • 10:15 - 12:00. RNN
  • 13:00 - 15:00. Project Discussion
• Day 5 (2025-12-19):
  • 10:15 - 12:00. Project Presentation
  • 13:00 - 15:00. Project Presentation

Requirement

Prerequisites. You are expected to be proficient in Python programming.

Format. In the first three blocks, I will introduce the basic knowledge of neural networks and how to build a toy deep learning framework step-by-step. In the fourth block, I will introduce the theory of RNN, and you are expected to build RNNs from scratch (with python and tensorslow in this course) individually. In the final block, each student should present their project and explain the code. I will ask random questions to check if you really understand your code.

Grading

• For bachelor students in Computational Linguistics: this seminar belongs to “Methods of computational linguistics and subject hours/Methoden der Computerlinguistik und Ver-suchspersonenstunden (LIN-BS-070)”.
  • To complete this seminar, you are expected to complete a software project on grammatical error correction, and give a presentation to explain your code at the end of this course.
  • (Optional) To pass the LIN-BS-070 module examination, you need to complete a term paper related to one of two seminars. If you would like to write a term paper on grammatical error correction, please talk to me.
• For bachelor students in Linguistics: this seminar belongs to “Computational linguistics (LIN-BS-034)”.
  • To complete this seminar and pass the module, you are expected to complete a software project on grammatical error correction, and write a term paper on NLP.

Resource

• Alice’s Adventures in a differentiable wonderland An intuitive, well-motivated reader on neural networks and their math foundation.
• Mathematical Engineering of Deep Learning More rigorous math description of deep learning.

Contact

Feel free to contact Meng at meng.li (at) uni-potsdam.de for any questions.